Bug 1473859 - DNF not removing all installed dependencies ( test case with wine )
DNF not removing all installed dependencies ( test case with wine )
Status: NEW
Product: Fedora
Classification: Fedora
Component: dnf (Show other bugs)
26
Unspecified Linux
unspecified Severity unspecified
: ---
: ---
Assigned To: Igor Gnatenko
Fedora Extras Quality Assurance
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-21 17:43 EDT by Mike Goodwin
Modified: 2017-09-21 03:51 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
tar of debugdata for installation and removal of wine per the reproducible steps (7.70 MB, application/x-gzip)
2017-07-21 17:43 EDT, Mike Goodwin
no flags Details
rpmreaper of leftover package (230.62 KB, image/png)
2017-07-24 10:38 EDT, Mike Goodwin
no flags Details

  None (edit)
Description Mike Goodwin 2017-07-21 17:43:15 EDT
Created attachment 1302561 [details]
tar of debugdata for installation and removal of wine per the reproducible steps

Description of problem:

  Niether DNF-1 nor DNF-2 completely remove all installed dependencies when installing and then immediately removing wine. This seems to happen across all DNF versions, as well as Fedora 26 and 25 (two which i've tested, but I suspect all). The exact number of missed packages differ between systems and conditions, but the story basically remains the same: 


Version-Release number of selected component (if applicable):

  Suspect all DNF versions on F25 and F26 at least


How reproducible:

  Always, tested on F25 and F26 fresh VM installations, as well as an initial report sent to me from another user on a non-stock installation. 


Steps to Reproduce:

  1. Install wine with `dnf install wine` 
  2. Observer amount of packages installed, usually around ~188-197 on a fresh install. 
  3. Remove wine with `dnf remove wine` 
  4. Observe that dnf will only remove between ~43-90 packages on a fresh install


Actual results:

  Hinted to this in the observations during reproduction steps, but basically the actual results are that DNF does NOT uninstall the same amount of packages that were installed as dependencies.

  This occurs at least for the 'wine' package (implying x86_64 on my system) and 'wine.i686' which I have tested. 

  'wine.i686' specific: Here the problem is compounded in that `dnf remove wine.i686` removes 43 packages and _another 53_ are then removed by `dnf autoremove` and still yet 101 packages are left on the system unattached to the original installtion package. This specific problem doesn't occur with the x86_64 packages on my system (i.e. needing to be autoremoved as well) where installing and removing just the 'wine.x86_64' package results in some packages being removed, but none are slated for `dnf autoremove`. In other words, I find it to be an additional sub-bug that `wine.i686` is not only broken in the same way - but also orphans packages slating them for autoremoving them when clean_requirements_on_remove is on! What's with that? The reproducible steps for this remain the same except use 'wine.i686' instead on an x86_64 system.


Expected results:

  My current understanding of how DNF is supposed to work, and that of my fellow colleague as well, is that DNF is supposed to remove unused dependencies upon removal of the leaf package that needed them. 


Additional info:

  If this is a "packaging error" then what about starting a discussion around how to not rely on packaging to ensure that dependencies are actually removed? 

  A discussion around how to find other affected packages? 

  What to do about people unaware of being affected by this bug?

----

Attached is a tar of --debugsolver output from F26 with the following folders

1. 'wine_debugdata_install'   = output of `dnf install wine --debugsolver` 
2. 'wine_debugdata_uninstall' = output of `dnf remove wine --debugsolver`
Comment 1 Mike Goodwin 2017-07-21 17:52:59 EDT
One particular point of worry for me is that 101 packages are left over after `dnf remove wine` (on x86_64, and i686 after autoremove) thus 101 packages are sitting on people's systems with no way for them to realize they're potentially no longer needed. They do NOT show up in autoremove.
Comment 2 Mike Goodwin 2017-07-24 10:38 EDT
Created attachment 1303710 [details]
rpmreaper of leftover package

Upon further investigation the majority of leftover packages are due to glibc.i686 being left behind because some of the i686 packages are left as leaves (rpmreaper output below):

        2.2M ├ < glibc-headers             2.25-7.fc26.x86_64
  o     3.4M ├ <+glibc-langpack-en         2.25-7.fc26.x86_64
         39K ├ < libaio                    0.3.110-7.fc26.x86_64
        8.6M ├ < python3-libs              3.6.2-1.fc26.x86_64
 l      1.4M ├─< alsa-lib                  1.1.4.1-1.fc26.i686
        266K ├─< audit-libs                2.7.7-1.fc26.i686
        145K ├─< avahi-libs                0.6.32-7.fc26.i686
         68K ├─< bzip2-libs                1.0.6-22.fc26.i686
        1.8M ├─< cairo                     1.14.10-1.fc26.i686
 L      840K ├─< cups-libs                 2.2.2-6.fc26.i686
        384K ├─< cyrus-sasl-lib            2.1.26-32.fc26.i686
        370K ├─< dbus-libs                 1.11.14-1.fc26.i686
        921K ├─< elfutils-libelf           0.169-1.fc26.i686
        230K ├─< expat                     2.2.1-1.fc26.i686
        554K ├─< flac-libs                 1.3.2-2.fc26.i686
        598K ├─< fontconfig                2.12.1-4.fc26.i686
        740K ├─< freetype                  2.7.1-9.fc26.i686
       11.3M ├─< glib2                     2.52.3-1.fc26.i686
        598K ├─< gmp                       6.1.2-4.fc26.i686
        2.3M ├─< gnutls                    3.5.14-1.fc26.i686
        243K ├─< graphite2                 1.3.10-1.fc26.i686
         55K ├─< gsm                       1.0.17-1.fc26.i686
        660K ├─< harfbuzz                  1.4.4-1.fc26.i686
        104K ├─< jbigkit-libs              2.1-6.fc26.i686
         42K ├─< keyutils-libs             1.5.10-1.fc26.i686
        1.8M ├─< krb5-libs                 1.15.1-15.fc26.i686
        151K ├─< libICE                    1.0.9-9.fc26.i686
         83K ├─< libSM                     1.2.2-5.fc26.i686
        1.3M ├─< libX11                    1.6.5-2.fc26.i686
         50K ├─< libXau                    1.0.8-7.fc26.i686
         29K ├─< libXdamage                1.1.4-9.fc26.i686
         88K ├─< libXext                   1.3.3-5.fc26.i686
         26K ├─< libXfixes                 5.0.3-2.fc26.i686
        122K ├─< libXft                    2.3.2-5.fc26.i686
         72K ├─< libXi                     1.7.9-2.fc26.i686
 L       47K ├─< libXrandr                 1.5.1-2.fc26.i686
         45K ├─< libXrender                0.9.10-2.fc26.i686
         29K ├─< libXtst                   1.2.3-2.fc26.i686
         25K ├─< libXxf86vm                1.1.4-4.fc26.i686
         55K ├─< libasyncns                0.8-11.fc26.i686
        310K ├─< libblkid                  2.30.1-1.fc26.i686
         99K ├─< libcap                    2.25-5.fc26.i686
         45K ├─< libcap-ng                 0.7.8-3.fc26.i686
         58K ├─< libcom_err                1.43.4-2.fc26.i686
         35K ├─< libcrypt-nss              2.25-7.fc26.i686
         58K ├─< libdatrie                 0.2.9-4.fc26.i686
        1.9M ├─< libdb                     5.3.28-24.fc26.i686
        371K ├─< libdrm                    2.4.82-1.fc26.i686
         43K ├─< libffi                    3.1-12.fc26.i686
        896K ├─< libgcrypt                 1.7.8-1.fc26.i686
        359K ├─< libglvnd                  0.2.999-19.20170620gitd850cdd.fc26.i686
  o      77K ├─< libglvnd-egl              0.2.999-19.20170620gitd850cdd.fc26.i686
  o     460K ├─< libglvnd-glx              0.2.999-19.20170620gitd850cdd.fc26.i686
        637K ├─< libgpg-error              1.25-2.fc26.i686
        222K ├─< libidn2                   2.0.2-1.fc26.i686
        531K ├─< libjpeg-turbo             1.5.1-0.fc26.i686
        371K ├─< libmount                  2.30.1-1.fc26.i686
         35K ├─< libogg                    1.3.2-6.fc26.i686
 L      333K ├─< libpcap                   1.8.1-3.fc26.i686
         44K ├─< libpciaccess              0.13.4-4.fc26.i686
        224K ├─< libpng                    1.6.28-2.fc26.i686
        219K ├─< libselinux                2.6-6.fc26.i686
        673K ├─< libsepol                  2.6-1.fc26.i686
        542K ├─< libsndfile                1.0.28-3.fc26.i686
        1.9M ├─< libstdc++                 7.1.1-3.fc26.i686
        160K ├─< libtasn1                  4.12-1.fc26.i686
        740K ├─< libthai                   0.1.25-2.fc26.i686
 l      494K ├─< libtiff                   4.0.8-1.fc26.i686
        1.5M ├─< libunistring              0.9.7-1.fc26.i686
         21K ├─< libuuid                   2.30.1-1.fc26.i686
         22K ├─< libverto                  0.2.6-7.fc26.i686
        771K ├─< libvorbis                 1.3.5-2.fc26.i686
         49K ├─< libwayland-client         1.13.0-1.fc26.i686
         70K ├─< libwayland-server         1.13.0-1.fc26.i686
        968K ├─< libxcb                    1.12-3.fc26.i686
 l      1.7M ├─< libxml2                   2.9.4-2.fc26.i686
          7K ├─< libxshmfence              1.2-4.fc26.i686
       46.7M ├─< llvm-libs                 4.0.0-2.fc26.i686
 L       57K ├─< lockdev                   1.0.4-0.23.20111007git.fc26.i686
         89K ├─< lz4-libs                  1.7.5-4.fc26.i686
 l     36.8M ├─< mesa-dri-drivers          17.1.5-1.fc26.i686
  o     225K ├─< mesa-libEGL               17.1.5-1.fc26.i686
  o     518K ├─< mesa-libGL                17.1.5-1.fc26.i686
         56K ├─< mesa-libgbm               17.1.5-1.fc26.i686
        195K ├─< mesa-libglapi             17.1.5-1.fc26.i686
        821K ├─< ncurses-libs              6.0-8.20170212.fc26.i686
        605K ├─< nettle                    3.3-2.fc26.i686
        297K ├─< nspr                      4.14.0-2.fc26.i686
  o     2.3M ├─< nss                       3.30.2-1.1.fc26.i686
        1.6M ├─< nss-softokn               3.30.2-1.0.fc26.i686
        476K ├─< nss-softokn-freebl        3.30.2-1.0.fc26.i686
        174K ├─< nss-util                  3.30.2-1.0.fc26.i686
 L      1.0M ├─< openldap                  2.4.45-1.fc26.i686
        2.8M ├─< openssl-libs              1.1.0f-7.fc26.i686
        1.3M ├─< p11-kit                   0.23.5-3.fc26.i686
 L      722K ├─< pango                     1.40.7-1.fc26.i686
        511K ├─< pcre                      8.41-1.fc26.i686
        712K ├─< pixman                    0.34.0-3.fc26.i686
 L      3.2M ├─< pulseaudio-libs           10.0-4.fc26.i686
 l      470K ├─< readline                  7.0-5.fc26.i686
        935K ├─< sqlite-libs               3.19.3-1.fc26.i686
        1.3M ├─< systemd-libs              233-6.fc26.i686
        127K ├─< tcp_wrappers-libs         7.6-85.fc26.i686
        172K ├─< xz-libs                   5.2.3-2.fc26.i686
        194K ├─< zlib                      1.2.11-2.fc26.i686
  o    14.6M glibc                     2.25-7.fc26.i686

Subsequently removing `glibc.i686` by itself removes most of the left-overs, except:

liberation-narrow-fonts-1.07.4-8.fc26.noarch
‎libgcc-7.1.1-3.fc26.i686     

The first of which ends up being required by fontconfig, and the last of which is left as a leaf with no dependencies or dependents, but is not picked up by `dnf autoremove`! (why was it left behind in the first place?)

A look at the reason file shows that it's a dep, too:

cat /var/lib/dnf/yumdb/l/a9f51a47ade707a4e80c710c2e44efaa6b20df62-libgcc-7.1.1-3.fc26-i686/reason
dep
Comment 3 Mike Goodwin 2017-07-24 10:52:09 EDT
I find the above issues particularly distressing from the viewpoint of a long-term user's system who may not be paying attention or looking for such issues. 

Think of it from the perspective of someone who installed wine a long time ago, and this person goes to remove it and all of the i686 dependencies are left behind. Then consider that they do remove glibc.i686 but don't think to look for the other packages that remained.

The way I discovered which were left behind was a tedious process of taking a `rpm -qa` snapshot of each state from a VM that is snapshotted, so it was relatively easy for me.

What worries me is how many other packages are like this and how many other dependencies are left behind on MY long-standing systems as a result of this dependency mismanagement? 

In addition, `dnf autoremove` doesn't seem to be picking up some leaf packages without dependents as it should be, packages for with rpm --whatrequires can't find anything for either. (though granted it could be some generic Requires) 

I really think this behavior in general needs to be re-factored so that we're not leaving cruft behind. I think that's an important tenant to package management and something that is imperative for the user to trust doesn't happen when installing/removing packages.
Comment 4 Mike Goodwin 2017-08-31 13:17:32 EDT
I recall we talked about this on IRC a bit and you said it was probably a packaging error and that you'd take a look at the specific packages. 

Is there any way to get visibility on this to the point where what I said in comment #3 about there being an initiative to ensure that this level of package mismanagement (leading to circular dependencies, or what seems more like dependencies that become dependent on packages after the fact?) isn't dependent on the ability of the packager to know better?

It seems more like a bug that a packager can potentially craft a package that causes packages to be left behind.

Note You need to log in before you can comment on or make changes to this bug.