Red Hat Bugzilla – Bug 1473859
DNF not removing all installed dependencies ( test case with wine )
Last modified: 2018-05-04 03:26:53 EDT
Created attachment 1302561 [details]
tar of debugdata for installation and removal of wine per the reproducible steps
Description of problem:
Niether DNF-1 nor DNF-2 completely remove all installed dependencies when installing and then immediately removing wine. This seems to happen across all DNF versions, as well as Fedora 26 and 25 (two which i've tested, but I suspect all). The exact number of missed packages differ between systems and conditions, but the story basically remains the same:
Version-Release number of selected component (if applicable):
Suspect all DNF versions on F25 and F26 at least
Always, tested on F25 and F26 fresh VM installations, as well as an initial report sent to me from another user on a non-stock installation.
Steps to Reproduce:
1. Install wine with `dnf install wine`
2. Observer amount of packages installed, usually around ~188-197 on a fresh install.
3. Remove wine with `dnf remove wine`
4. Observe that dnf will only remove between ~43-90 packages on a fresh install
Hinted to this in the observations during reproduction steps, but basically the actual results are that DNF does NOT uninstall the same amount of packages that were installed as dependencies.
This occurs at least for the 'wine' package (implying x86_64 on my system) and 'wine.i686' which I have tested.
'wine.i686' specific: Here the problem is compounded in that `dnf remove wine.i686` removes 43 packages and _another 53_ are then removed by `dnf autoremove` and still yet 101 packages are left on the system unattached to the original installtion package. This specific problem doesn't occur with the x86_64 packages on my system (i.e. needing to be autoremoved as well) where installing and removing just the 'wine.x86_64' package results in some packages being removed, but none are slated for `dnf autoremove`. In other words, I find it to be an additional sub-bug that `wine.i686` is not only broken in the same way - but also orphans packages slating them for autoremoving them when clean_requirements_on_remove is on! What's with that? The reproducible steps for this remain the same except use 'wine.i686' instead on an x86_64 system.
My current understanding of how DNF is supposed to work, and that of my fellow colleague as well, is that DNF is supposed to remove unused dependencies upon removal of the leaf package that needed them.
If this is a "packaging error" then what about starting a discussion around how to not rely on packaging to ensure that dependencies are actually removed?
A discussion around how to find other affected packages?
What to do about people unaware of being affected by this bug?
Attached is a tar of --debugsolver output from F26 with the following folders
1. 'wine_debugdata_install' = output of `dnf install wine --debugsolver`
2. 'wine_debugdata_uninstall' = output of `dnf remove wine --debugsolver`
One particular point of worry for me is that 101 packages are left over after `dnf remove wine` (on x86_64, and i686 after autoremove) thus 101 packages are sitting on people's systems with no way for them to realize they're potentially no longer needed. They do NOT show up in autoremove.
Created attachment 1303710 [details]
rpmreaper of leftover package
Upon further investigation the majority of leftover packages are due to glibc.i686 being left behind because some of the i686 packages are left as leaves (rpmreaper output below):
2.2M ├ < glibc-headers 2.25-7.fc26.x86_64
o 3.4M ├ <+glibc-langpack-en 2.25-7.fc26.x86_64
39K ├ < libaio 0.3.110-7.fc26.x86_64
8.6M ├ < python3-libs 3.6.2-1.fc26.x86_64
l 1.4M ├─< alsa-lib 220.127.116.11-1.fc26.i686
266K ├─< audit-libs 2.7.7-1.fc26.i686
145K ├─< avahi-libs 0.6.32-7.fc26.i686
68K ├─< bzip2-libs 1.0.6-22.fc26.i686
1.8M ├─< cairo 1.14.10-1.fc26.i686
L 840K ├─< cups-libs 2.2.2-6.fc26.i686
384K ├─< cyrus-sasl-lib 2.1.26-32.fc26.i686
370K ├─< dbus-libs 1.11.14-1.fc26.i686
921K ├─< elfutils-libelf 0.169-1.fc26.i686
230K ├─< expat 2.2.1-1.fc26.i686
554K ├─< flac-libs 1.3.2-2.fc26.i686
598K ├─< fontconfig 2.12.1-4.fc26.i686
740K ├─< freetype 2.7.1-9.fc26.i686
11.3M ├─< glib2 2.52.3-1.fc26.i686
598K ├─< gmp 6.1.2-4.fc26.i686
2.3M ├─< gnutls 3.5.14-1.fc26.i686
243K ├─< graphite2 1.3.10-1.fc26.i686
55K ├─< gsm 1.0.17-1.fc26.i686
660K ├─< harfbuzz 1.4.4-1.fc26.i686
104K ├─< jbigkit-libs 2.1-6.fc26.i686
42K ├─< keyutils-libs 1.5.10-1.fc26.i686
1.8M ├─< krb5-libs 1.15.1-15.fc26.i686
151K ├─< libICE 1.0.9-9.fc26.i686
83K ├─< libSM 1.2.2-5.fc26.i686
1.3M ├─< libX11 1.6.5-2.fc26.i686
50K ├─< libXau 1.0.8-7.fc26.i686
29K ├─< libXdamage 1.1.4-9.fc26.i686
88K ├─< libXext 1.3.3-5.fc26.i686
26K ├─< libXfixes 5.0.3-2.fc26.i686
122K ├─< libXft 2.3.2-5.fc26.i686
72K ├─< libXi 1.7.9-2.fc26.i686
L 47K ├─< libXrandr 1.5.1-2.fc26.i686
45K ├─< libXrender 0.9.10-2.fc26.i686
29K ├─< libXtst 1.2.3-2.fc26.i686
25K ├─< libXxf86vm 1.1.4-4.fc26.i686
55K ├─< libasyncns 0.8-11.fc26.i686
310K ├─< libblkid 2.30.1-1.fc26.i686
99K ├─< libcap 2.25-5.fc26.i686
45K ├─< libcap-ng 0.7.8-3.fc26.i686
58K ├─< libcom_err 1.43.4-2.fc26.i686
35K ├─< libcrypt-nss 2.25-7.fc26.i686
58K ├─< libdatrie 0.2.9-4.fc26.i686
1.9M ├─< libdb 5.3.28-24.fc26.i686
371K ├─< libdrm 2.4.82-1.fc26.i686
43K ├─< libffi 3.1-12.fc26.i686
896K ├─< libgcrypt 1.7.8-1.fc26.i686
359K ├─< libglvnd 0.2.999-19.20170620gitd850cdd.fc26.i686
o 77K ├─< libglvnd-egl 0.2.999-19.20170620gitd850cdd.fc26.i686
o 460K ├─< libglvnd-glx 0.2.999-19.20170620gitd850cdd.fc26.i686
637K ├─< libgpg-error 1.25-2.fc26.i686
222K ├─< libidn2 2.0.2-1.fc26.i686
531K ├─< libjpeg-turbo 1.5.1-0.fc26.i686
371K ├─< libmount 2.30.1-1.fc26.i686
35K ├─< libogg 1.3.2-6.fc26.i686
L 333K ├─< libpcap 1.8.1-3.fc26.i686
44K ├─< libpciaccess 0.13.4-4.fc26.i686
224K ├─< libpng 1.6.28-2.fc26.i686
219K ├─< libselinux 2.6-6.fc26.i686
673K ├─< libsepol 2.6-1.fc26.i686
542K ├─< libsndfile 1.0.28-3.fc26.i686
1.9M ├─< libstdc++ 7.1.1-3.fc26.i686
160K ├─< libtasn1 4.12-1.fc26.i686
740K ├─< libthai 0.1.25-2.fc26.i686
l 494K ├─< libtiff 4.0.8-1.fc26.i686
1.5M ├─< libunistring 0.9.7-1.fc26.i686
21K ├─< libuuid 2.30.1-1.fc26.i686
22K ├─< libverto 0.2.6-7.fc26.i686
771K ├─< libvorbis 1.3.5-2.fc26.i686
49K ├─< libwayland-client 1.13.0-1.fc26.i686
70K ├─< libwayland-server 1.13.0-1.fc26.i686
968K ├─< libxcb 1.12-3.fc26.i686
l 1.7M ├─< libxml2 2.9.4-2.fc26.i686
7K ├─< libxshmfence 1.2-4.fc26.i686
46.7M ├─< llvm-libs 4.0.0-2.fc26.i686
L 57K ├─< lockdev 1.0.4-0.23.20111007git.fc26.i686
89K ├─< lz4-libs 1.7.5-4.fc26.i686
l 36.8M ├─< mesa-dri-drivers 17.1.5-1.fc26.i686
o 225K ├─< mesa-libEGL 17.1.5-1.fc26.i686
o 518K ├─< mesa-libGL 17.1.5-1.fc26.i686
56K ├─< mesa-libgbm 17.1.5-1.fc26.i686
195K ├─< mesa-libglapi 17.1.5-1.fc26.i686
821K ├─< ncurses-libs 6.0-8.20170212.fc26.i686
605K ├─< nettle 3.3-2.fc26.i686
297K ├─< nspr 4.14.0-2.fc26.i686
o 2.3M ├─< nss 3.30.2-1.1.fc26.i686
1.6M ├─< nss-softokn 3.30.2-1.0.fc26.i686
476K ├─< nss-softokn-freebl 3.30.2-1.0.fc26.i686
174K ├─< nss-util 3.30.2-1.0.fc26.i686
L 1.0M ├─< openldap 2.4.45-1.fc26.i686
2.8M ├─< openssl-libs 1.1.0f-7.fc26.i686
1.3M ├─< p11-kit 0.23.5-3.fc26.i686
L 722K ├─< pango 1.40.7-1.fc26.i686
511K ├─< pcre 8.41-1.fc26.i686
712K ├─< pixman 0.34.0-3.fc26.i686
L 3.2M ├─< pulseaudio-libs 10.0-4.fc26.i686
l 470K ├─< readline 7.0-5.fc26.i686
935K ├─< sqlite-libs 3.19.3-1.fc26.i686
1.3M ├─< systemd-libs 233-6.fc26.i686
127K ├─< tcp_wrappers-libs 7.6-85.fc26.i686
172K ├─< xz-libs 5.2.3-2.fc26.i686
194K ├─< zlib 1.2.11-2.fc26.i686
o 14.6M glibc 2.25-7.fc26.i686
Subsequently removing `glibc.i686` by itself removes most of the left-overs, except:
The first of which ends up being required by fontconfig, and the last of which is left as a leaf with no dependencies or dependents, but is not picked up by `dnf autoremove`! (why was it left behind in the first place?)
A look at the reason file shows that it's a dep, too:
I find the above issues particularly distressing from the viewpoint of a long-term user's system who may not be paying attention or looking for such issues.
Think of it from the perspective of someone who installed wine a long time ago, and this person goes to remove it and all of the i686 dependencies are left behind. Then consider that they do remove glibc.i686 but don't think to look for the other packages that remained.
The way I discovered which were left behind was a tedious process of taking a `rpm -qa` snapshot of each state from a VM that is snapshotted, so it was relatively easy for me.
What worries me is how many other packages are like this and how many other dependencies are left behind on MY long-standing systems as a result of this dependency mismanagement?
In addition, `dnf autoremove` doesn't seem to be picking up some leaf packages without dependents as it should be, packages for with rpm --whatrequires can't find anything for either. (though granted it could be some generic Requires)
I really think this behavior in general needs to be re-factored so that we're not leaving cruft behind. I think that's an important tenant to package management and something that is imperative for the user to trust doesn't happen when installing/removing packages.
I recall we talked about this on IRC a bit and you said it was probably a packaging error and that you'd take a look at the specific packages.
Is there any way to get visibility on this to the point where what I said in comment #3 about there being an initiative to ensure that this level of package mismanagement (leading to circular dependencies, or what seems more like dependencies that become dependent on packages after the fact?) isn't dependent on the ability of the packager to know better?
It seems more like a bug that a packager can potentially craft a package that causes packages to be left behind.
It looks like that the problem here is that dnf does't remove packages that have provide that is provided by other package on the system. If this is the only issue I have to disappoint you because this is wanted behavior due to incorrect packaging aspecially in 3rd party repos.
Example: I have package Pa that requires Pb. Pb also on system. I install Xa that requires Xb. Both installed. But Xb also provides Pb. When I remove Xa, there is only removed Xa package. Why - solver can remove Pb or Xb because both provides Pb. In past solver really removed one of them, but often happened that it resulted in destroyed system, because not all packages that provides something really provided it (like skype in past). To prevent such an issues we decide to make auto-remove of dependencies more secure by keeping both providers. It means that system getting little bit bigger, but functional after remove.
If X* is x86_64, and P* is i686, you assert that removing Xa should not remove Pb or Xb, but it probably *should* remove Xb.
Is the error here not that x86_64 packages and i686 packages can even provide the same thing? Isn't it not supposed to do that?
If given the above scenario, Xa is removed, I would argue that Xb should be removed also. Pb should be left alone because it's i686.
I still maintain that this is a bug as this situation shouldn't happen, because what this ticket is about originally here is a problem with i686 and x86_64 packages. (granted it took a bit to decipher that from the information I had when I posted the bug initially)
To address your reply specifically about this behavior being desirable and bad packaging - I *totally understand* what you're saying ... if we were talking about two packages of the same arch (which I think is a critical point), but we aren't.
Thus, if the package to be removed is x86_64 and in its removal chain there are x86_64 and i606 packages with the same provides, the x86_64 package should probably be considered for removal.
Somewhat off topic to this specific ticket, but worth mentioning:
The other overarching issue is that there's _no way to track_ (it seems) this situation where two packages provide things twice. How do you manage this?
I would think dnf shouldn't silently keep something that it ordinarily would remove, and let the user know that it couldn't choose which because there were two providers.
It seems to me that these packages basically become invisible to dnf when considering removals, without there being a command to reveal these conflicts.
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora 'version'
Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.
Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.
Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.