Created attachment 999424 [details] kickstart file generated by anaconda Description of problem: The package selection section of Anaconda kickstart file, I specify several packages to exclude. However, these packages are actually installed. Version-Release number of selected component (if applicable): Fedora 22 Server Alpha TC8 How reproducible: Steps to Reproduce: 1. install fedora 22 by using kickstart file. 2. 3. Actual results: Packages explicitly excluded in the kickstart file are installed. Expected results: These packages are not installed. Additional info:
Created attachment 999425 [details] anaconda.log
Created attachment 999426 [details] dnf.log
Created attachment 999427 [details] dnf.rpm.log
Created attachment 999428 [details] journal.log of the installation process
Created attachment 999429 [details] packaging.log of anaconda
Created attachment 999430 [details] program.log of anaconda
In package.log I attached above, there exists the following INFO messages %%%%%%%%%%%%%%%% 07:24:06,454 INFO packaging: skipped removing nonexistant package: realmd 07:24:06,469 INFO packaging: skipped removing nonexistant package: abrt-cli 07:24:06,485 INFO packaging: skipped removing nonexistant package: sssd 07:24:06,501 INFO packaging: skipped removing nonexistant package: fprintd-pam 07:24:06,517 INFO packaging: skipped removing nonexistant package: mlocate 07:24:06,533 INFO packaging: skipped removing nonexistant package: mcelog %%%%%%%%%%%%%%%% which program output these messages? If Anaconda output them, it might be Anaconda's bug.
TBH, I have no idea how Anaconda uses DNF. It would be nice if someone can tell us. Depending on what it does, this bug may be a duplicate of bug 1185408.
The message is not from DNF. Reassigning to Anaconda. If you find it's really DNF bug, put there some reproducer calling DNF so we can fix it. Thanks.
Thank you for reassigning to the suitable package. I've tested Fedora 22 Server Alpha. However, the situation is the same. %%%%%%%%%%%% 01:56:58,945 INFO packaging: skipped removing nonexistant package: realmd 01:56:58,961 INFO packaging: skipped removing nonexistant package: abrt-cli 01:56:58,977 INFO packaging: skipped removing nonexistant package: sssd 01:56:58,993 INFO packaging: skipped removing nonexistant package: fprintd-pam 01:56:59,008 INFO packaging: skipped removing nonexistant package: mlocate 01:56:59,024 INFO packaging: skipped removing nonexistant package: mcelog %%%%%%%%%%%% package.log say above. However, these package actually exist on the repository, and are installed by this installation process.
Can you please try again with F23 beta? I think I fixed this in anaconda-23.1-1, which would be a version of anaconda that will be in F23.
I got the same result on F23 beta TC4. Excluded packages in ks file are still installed.
Created attachment 1072068 [details] anaconda-ks.cfg
Created attachment 1072069 [details] anaconda.log
Created attachment 1072070 [details] dnf.log
Created attachment 1072071 [details] dnf.rpm.log
Created attachment 1072072 [details] packaging.log
Created attachment 1072073 [details] program.log
Okay, which packages that you requested for removal are being installed? Also note that when you list that you want a package removed, you really are just requesting it. If it is required by some other package, it will still get pulled in. anaconda doesn't break dependencies to force removals.
As you can see in the attached ks file, https://bugzilla.redhat.com/attachment.cgi?id=1072068 I've tried to exclude the following packages: -NetworkManager -abrt-cli -cifs-utils -coolkey -cryptsetup -cyrus-sasl-plain -fprintd-pam -irda-utils -mlocate -ntfs-3g -ntfsprogs -pam_krb5 -pam_pkcs11 -pcmciautils -plymouth -policycoreutils -realmd -selinux-policy-targeted -sssd -vconfig -wireless-tools I have not check that all the above packages are not required by the selected packages, which are, also as you can see in the attached ks file, @core, @standard, and mainly development tools. However, at least, realmd is not required. After the installation, realmd can be removed independently.
I confirm that irda-utils, mlocate and vconfig can be independently removed on the installed system.
I'm able to reproduce this with %packages @standard -abrt-cli %end which installs abrt-cli. Anaconda is calling dnf's base.group_install(<group object>, {'default', 'mandatory'}, exclude=['abrt-cli']), which does not appear to be working as expected.
Created attachment 1072636 [details] exclude-test.py Reproducer. The output of packages selected in the transaction includes: ('abrt-cli', 'x86_64', '0', '2.6.2', '7.fc24')
*** Bug 1266952 has been marked as a duplicate of this bug. ***
Discussed at 2015-09-28 blocker review meeting: https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2015-09-28/f23-blocker-review.2015-09-28-16.01.html . We are (still...last discussed at F18...sigh) lacking detailed kickstart criteria, so we're still operating on the basis that we evaluate kickstart issues case-by-case with the intent of setting some precedent for writing better criteria. There was a consensus at the meeting that this is a sufficiently core and commonly-used element of kickstart functionality that it should be release blocking, hence this bug is accepted as a blocker. We noted that the Cloud images in fact rely on this feature to trim some unneeded packages, and thus may be larger than intended at present.
Hmm, so I think I see a possible cause here. I think it may not be possible to exclude 'mandatory' packages...and libcomps considers any package whose type is not explicitly stated to be 'mandatory'. Viz libcomps comps_elem.c: tmp = comps_dict_get(elem->attrs, "type"); if (!tmp) package->type = parsed->def_options->default_pkgtype; else package->type = comps_package_get_type(tmp); and comps_default.c: .default_pkgtype = COMPS_PACKAGE_MANDATORY Now in dnf, you have to get all the way to dnf/comps.py Solver.group_install() , which is ultimately what gets called by base's group_install (it calls dnf.comps.install_or_skip() which is a thin wrapper around the function that's passed as the first argument, that function is solver.group_install, which is Solver.group_install). Solver.group_install() does this: types = pkg_types & MANDATORY mandatory = self._pkgs_of_type(group, types, exclude) types = pkg_types & (DEFAULT | OPTIONAL) trans.install_opt = self._pkgs_of_type(group, types, exclude) ...so we go look at _pkgs_of_type(), which does this: def _pkgs_of_type(group, pkg_types, exclude): pkgs = set() if pkg_types & MANDATORY: pkgs.update(pkg.name for pkg in group.mandatory_packages) if pkg_types & DEFAULT: pkgs.update(pkg.name for pkg in group.default_packages if pkg.name not in exclude) if pkg_types & OPTIONAL: pkgs.update(pkg.name for pkg in group.optional_packages if pkg.name not in exclude) return pkgs there's some bitwise crap going on there which I'm kinda handwaving, but point is basically that, when that first condition is hit, exclude is *ignored*: presumably the idea is you shouldn't be able to exclude 'mandatory' packages, but I don't think that's compatible with current expectations.
I take it then that a packagereq without a type is "mandatory" in comps? apper is: <packagereq>apper</packagereq> and I can't exclude it. I'm think it's reasonable to not be able to exclude "mandatory" packages from comps groups, but if that's going to be the case we're going to need to change a lot of packages in comps to "default".
"I take it then that a packagereq without a type is "mandatory" in comps?" That's part of my theory, yeah. "I'm think it's reasonable to not be able to exclude "mandatory" packages from comps groups, but if that's going to be the case we're going to need to change a lot of packages in comps to "default"." That's *only* my theory. I might be wrong on any point, I haven't tested it for sure.
(In reply to awilliam from comment #28) > "I take it then that a packagereq without a type is "mandatory" in comps?" > > That's part of my theory, yeah. Probably. If you think this behavior is wrong -> reassign it to libcomps component, please. > "I'm think it's reasonable to not be able to exclude "mandatory" packages > from comps groups, but if that's going to be the case we're going to need to > change a lot of packages in comps to "default"." It's exactly like that. If you want to exclude mandatory package you can't use group_install. You have to use the workaround of iterating over packages in groups and add call "base.install" for non-excluded packages. This was already discussed in some bug report and AFAIK Anaconda uses this workaround. Keeping the mandatory packages from `group_install` is intentional.
Hmm, so looking at dnfpayload.py anaconda is indeed doing something like that workaround in one place, as part of `_apply_selections`: for pkg_name in set(self.data.packages.packageList) - set(self.data.packages.excludedList): try: self._install_package(pkg_name) log.info("selected package: '%s'", pkg_name) except packaging.NoSuchPackage as e: self._miss(e) but several *other* places in the same file call `self._select_group()`, which calls `self._base.group_install(grp, types, exclude=exclude)` . The most recent change which added a `self._select_group()` call is https://github.com/rhinstaller/anaconda/commit/079e643683d3673cac2ee5cc8481bb9ecc269c99 , from April - the others have all been there since last December or earlier, I think.
jsilhan: I can see the thinking, there, but it *is* causing anaconda quite a lot of trouble, especially since dnf doesn't communicate that there's any kind of issue with the 'exclude' list, it simply silently ignores what you told it. couldn't group_install have some sort of 'really-exclude' arg, or some other mechanism to allow a caller to force exclusion of 'mandatory' packages? I think the historical context here is valuable. AIUI, the 'mandatory'/'default'/'optional' thing was originally intended for *anaconda*, it really wasn't so much for yum: it defined what packages on the old package selection screen were selected by default and could not be unselected, which were selected by default but be unselected, and which were not selected by default (for each group). Pre-dnf, including all the time anaconda had the old package selection UI, you could always de-select whatever you liked with a kickstart (assuming dependencies were satisfied). 'mandatory' was never enforced to the extent dnf is now trying to enforce it, it was more a cue for the anaconda UI.
IMHO, being able to say, I want group @foo without packages bar and baz, no matter whether bar and baz are considered "mandatory" in @foo (which is a pretty much arbitrary decision), is essential kickstart functionality.
Hmm... now we even let the users to be able to exclude mandatory packages from the group in CLI (dnf install @<group> -x <mandatory_pkg> --setopt=strict=0). Maybe we can allow it in python API too. I will throw it into discussion during software management meeting - whats the meaning of mandatory for package manager. PR: https://github.com/rpm-software-management/dnf/pull/369
(In reply to awilliam from comment #28) > "I take it then that a packagereq without a type is "mandatory" in comps?" > > That's part of my theory, yeah. > > "I'm think it's reasonable to not be able to exclude "mandatory" packages > from comps groups, but if that's going to be the case we're going to need to > change a lot of packages in comps to "default"." Got to say, I would never have guessed that packagereqs are not "default" by default....
dnf-1.1.3-1.fc23 dnf-plugins-extras-0.0.11-1.fc23 dnf-plugins-core-0.1.13-1.fc23 hawkey-0.6.2-1.fc23 libsolv-0.6.14-2.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-a011ac001a
dnf-1.1.3-1.fc22 dnf-plugins-extras-0.0.11-1.fc22 dnf-plugins-core-0.1.13-1.fc22 hawkey-0.6.2-1.fc22 libsolv-0.6.14-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-02efefba0e
dnf-1.1.3-1.fc23, dnf-plugins-core-0.1.13-1.fc23, dnf-plugins-extras-0.0.11-1.fc23, hawkey-0.6.2-1.fc23, libsolv-0.6.14-2.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with $ su -c 'dnf --enablerepo=updates-testing update hawkey dnf-plugins-core dnf-plugins-extras dnf libsolv' You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-a011ac001a
dnf-1.1.3-1.fc22, dnf-plugins-core-0.1.13-1.fc22, dnf-plugins-extras-0.0.11-1.fc22, hawkey-0.6.2-1.fc22, libsolv-0.6.14-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with $ su -c 'dnf --enablerepo=updates-testing update hawkey dnf-plugins-core dnf-plugins-extras dnf libsolv' You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-02efefba0e
Should this start working once a new test compose has the new dnf, or are anaconda changes required as well?
Anaconda changes should not be necessary.
Looks fixed to me with F23_TC11
yeah, upstream DNF basically changed to allow what anaconda's trying to do to work, so this should work without anaconda changes. Kamil, can you confirm it works for you? Then we'll get the DNF update pushed stable and this can be closed.
dnf-1.1.3-1.fc23, dnf-plugins-core-0.1.13-1.fc23, dnf-plugins-extras-0.0.11-1.fc23, hawkey-0.6.2-1.fc23, libsolv-0.6.14-2.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
dnf-1.1.3-1.fc22, dnf-plugins-core-0.1.13-1.fc22, dnf-plugins-extras-0.0.11-1.fc22, hawkey-0.6.2-1.fc22, libsolv-0.6.14-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.