See: https://koji.fedoraproject.org/koji/taskinfo?taskID=28657220 in anaconda-packaging.log: ... Problem 1: conflicting requests - package chrony-3.3-4.fc29.x86_64 is disabled Problem 2: conflicting requests - package dracut-live-048-14.git20180726.fc29.x86_64 requires tar, but none of the providers can be installed - package tar-2:1.30-5.fc29.x86_64 is disabled Problem 3: conflicting requests - package anaconda-install-env-deps-29.22-1.fc29.x86_64 requires rsync, but none of the providers can be installed - package rsync-3.1.3-4.fc29.x86_64 is disabled Problem 4: conflicting requests - package kernel-modules-extra-4.18.0-0.rc6.git1.1.fc29.x86_64 requires kernel-uname-r = 4.18.0-0.rc6.git1.1.fc29.x86_64, but none of the providers can be installed - package kernel-core-4.18.0-0.rc6.git1.1.fc29.x86_64 requires /usr/bin/kernel-install, but none of the providers can be installed - package systemd-udev-239-3.fc29.x86_64 is disabled Problem 5: conflicting requests - package kernel-modules-4.18.0-0.rc6.git1.1.fc29.x86_64 requires kernel-uname-r = 4.18.0-0.rc6.git1.1.fc29.x86_64, but none of the providers can be installed - package kernel-core-4.18.0-0.rc6.git1.1.fc29.x86_64 requires /usr/bin/kernel-install, but none of the providers can be installed - package systemd-udev-239-3.fc29.x86_64 is disabled Problem 6: conflicting requests - package kernel-4.18.0-0.rc6.git1.1.fc29.x86_64 requires kernel-core-uname-r = 4.18.0-0.rc6.git1.1.fc29.x86_64, but none of the providers can be installed - package kernel-core-4.18.0-0.rc6.git1.1.fc29.x86_64 requires /usr/bin/kernel-install, but none of the providers can be installed - package systemd-udev-239-3.fc29.x86_64 is disabled Problem 7: conflicting requests - package kernel-4.18.0-0.rc6.git1.1.fc29.x86_64 requires kernel-core-uname-r = 4.18.0-0.rc6.git1.1.fc29.x86_64, but none of the providers can be installed - package kernel-core-4.18.0-0.rc6.git1.1.fc29.x86_64 requires /usr/bin/kernel-install, but none of the providers can be installed - package systemd-udev-239-3.fc29.x86_64 is disabled Problem 8: conflicting requests - package anaconda-29.22-1.fc29.x86_64 requires anaconda-install-env-deps = 29.22-1.fc29, but none of the providers can be installed - package anaconda-install-env-deps-29.22-1.fc29.x86_64 requires rsync, but none of the providers can be installed - package rsync-3.1.3-4.fc29.x86_64 is disabled Problem 9: conflicting requests - package firewalld-0.6.0-1.fc29.noarch requires systemd, but none of the providers can be installed - package systemd-239-3.fc29.x86_64 requires dbus >= 1.9.18, but none of the providers can be installed - package systemd-239-3.fc29.i686 requires dbus >= 1.9.18, but none of the providers can be installed - package dbus-1:1.12.8-3.fc29.x86_64 is disabled - package dbus-1:1.12.8-3.fc29.i686 is disabled ... There should be only 2 repos enabled, the "Everything" repo and the "Workstation" repo. There shouldn't be any modules involved. Not sure why it's reporting things as 'disabled'. I have untagged this build for now so we can get composes going again. Let me know if you need more info.
Also had to untag anaconda-29.22-1.fc29 Adding anaconda-maint-list
Isn't there module metadata available in any of these repos? In that case DNF filters RPMs according to module defaults or streams requested by Anacoda. We'll work with Anaconda team to track this down and get it fixed.
Created attachment 1471532 [details] Combined log from the crash reproduced via kickstart test I've managed to reproduce the crash with a kickstart test and this is the combined log from the installation run. To look for what's normally packaging.log just search for "anaconda:packaging" and similarly for other logs.
Also this is the fairly humongous %packages section for the live image kickstart: %packages @anaconda-tools @base-x @core @firefox @fonts @gnome-desktop @guest-desktop-agents @hardware-support @libreoffice @multimedia @networkmanager-submodules @printing @workstation-product aajohan-comfortaa-fonts anaconda anaconda-install-env-deps dracut-live glibc-all-langpacks kernel kernel-modules kernel-modules-extra memtest86+ syslinux -@dial-up -@input-methods -@standard -gfs2-utils -reiserfs-utils %end I guess feeding an equivalent of this to a DNF transaction could be enough to reproduce the issue as well ?
Also these are the lists Anaconda passes to the install_specs() function as seen in the combined log: 12:35:26,108 DEBUG anaconda:packaging: transaction include list 12:35:26,108 DEBUG anaconda:packaging: ['@core', '@anaconda-tools', '@base-x', '@core', '@firefox', '@fonts', '@gnome-desktop', '@guest-desktop-agents', '@hardware-support', '@libreoffice', '@multimedia', '@networkmanager-submodules', '@printing', '@workstation-product', 'aajohan-comfortaa-fonts', 'anaconda', 'anaconda-install-env-deps', 'dracut-live', 'glibc-all-langpacks', 'kernel', 'kernel-modules', 'kernel-modules-extra', 'memtest86+', 'syslinux', 'kernel', 'e2fsprogs', 'grub2', 'grub2-tools', 'firewalld', 'langpacks-en', 'authselect-compat', 'chrony'] 12:35:26,108 DEBUG anaconda:packaging: transaction exclude list 12:35:26,109 DEBUG anaconda:packaging: ['@dial-up', '@input-methods', '@standard', 'gfs2-utils', 'reiserfs-utils']
-@standard excludes 'standard' group incl. packages such as 'tar' which is one of the reported errors. I'd say it's quite expected behavior. Could you remove this exclude and try again? When I removed -@standard, transaction passed as expected. Unless I'm missing something, it looks like group excludes never worked properly and the new anaconda + dnf do it right and that caused this issue.
Well, it depends on what: @foo -@foo is intended to do, and what historically it did, doesn't it? I suspect the intent here was that the workstation "-@foo" countermanded the base "@foo": the intent was that they should essentially cancel out and tell dnf "don't do anything specifically related to group foo". So the packages in @foo won't be pulled in purely due to membership of @foo, but they *would* be in the transaction if they're part of some other selected group, or they're dependencies of some other selected package, or whatever. I *don't* think the intent was that "-@foo" should be interpreted as "explicitly remove all packages that are part of @foo from the package set you'd calculated so far". Before declaring this "expected" or "right" behaviour I think it'd be imperative to go back and check how a) yum and b) old-DNF interpreted this.
(Also note that the behaviour I described is behaviour we clearly have a reasonable use case for: if not like this, how *else* is a kickstart meant to inherit from another kickstart, but countermand one of its group selections in this way?)
Fixed by https://github.com/rpm-software-management/dnf/pull/1152
Created attachment 1473561 [details] traceback with the proposed fix I've tried to use the proposed fix on a recent Rawhide nightly and got this: anaconda 29.21 exception report Traceback (most recent call first): File "/tmp/updates/dnf/base.py", line 226, in _setup_excludes_includes self.sack.filter_modules(self._moduleContainer, hot_fix_repos, File "/tmp/updates/dnf/base.py", line 486, in fill_sack self._setup_excludes_includes() File "/tmp/updates/pyanaconda/payload/dnfpayload.py", line 973, in gatherRepoMetadata self._base.fill_sack(load_system_repo=False) File "/tmp/updates/pyanaconda/payload/__init__.py", line 1689, in _runThread payload.gatherRepoMetadata() File "/usr/lib64/python3.7/threading.py", line 865, in run self._target(*self._args, **self._kwargs) File "/tmp/updates/pyanaconda/threading.py", line 286, in run threading.Thread.run(self) AttributeError: 'Sack' object has no attribute 'filter_modules' (see the attached traceback file for more information) Could it be I just have an outdated version of libdnf or something like that ? According to the /root/lorax-packages.log the image has: - libdnf 0.16.1-2 - dnf 3.0.4-1 And I've put the latests DNF master on top of that via updates image.
So according to Dan Mach the root cause of the live image build issue should be fixed in dnf 3.2 and I've released anaconda-29.23-1 that requires dnf >=3.2 due to some changes in how platform id for modules is set. During my local testing the "conflicting requests" issue indeed no longer shows up even though the resulting image seems to be a bit bigger, which might be something to investigate later on and even possibly (temporarily ?) bump the maximum image size. If the live image size increase manifests also in Koji it might be good to check for the root cause - it might very well be that some packages have been silently ignored in the past but are now correctly pulled in.
Martin and me have just tested following builds: anaconda-29.23-2.fc29 dnf-3.2.0-1.fc29 libdnf-0.17.0-1.fc29 We can confirm that this combination of builds works as expected (no crashes, excludes work fine). Unfortunately there's an image size increase that leads to an insufficient space error. It is not clear to me what's the root cause, if it's a different repository or change in dnf code. Kevin, could you eventually exclude additional packages or groups by hand in he kickstart? Could you also somehow diff package lists of the old and new images and tell us the difference? That would help us to understand the root cause of the size increase.
We can diff package lists if you give us the log from the failed compose attempt...it ought to include a full list of the packages it tried to include.
In fact we've had successful composes of the critical live images (Workstation and KDE x86_64) in the last few Rawhide compose attempts, so I don't *think* we have a problem there: https://koji.fedoraproject.org/koji/taskinfo?taskID=28920822 https://koji.fedoraproject.org/koji/taskinfo?taskID=28920439 I will check and see if they include more packages than they used to.
Both 20180802.n.0 and 20180808.n.0 Workstation x86_64 live composes included 1604 packages: https://kojipkgs.fedoraproject.org//work/tasks/822/28920822/root.log https://kojipkgs.fedoraproject.org//work/tasks/7650/28787650/root.log so I don't think we have a problem with installing more packages than we used to.