Bug 1723126 - dnf remove --duplicates does not work after system crash during an update
Summary: dnf remove --duplicates does not work after system crash during an update
Keywords:
Status: CLOSED DUPLICATE of bug 1674296
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 30
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jaroslav Mracek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1091702
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-06-23 04:23 UTC by Oleg Drokin
Modified: 2019-09-07 14:47 UTC (History)
15 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2019-09-07 14:47:49 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
full dnf remove --duplicates output (184.86 KB, text/plain)
2019-06-27 16:51 UTC, Oleg Drokin
no flags Details
rpm -qa output (78.50 KB, text/plain)
2019-06-27 17:40 UTC, Oleg Drokin
no flags Details
dnf repoquery --duplicates (3.92 KB, text/plain)
2019-06-27 17:40 UTC, Oleg Drokin
no flags Details
rpm -qa output after the two mentioned dnf steps and dnf update (49.35 KB, text/plain)
2019-06-27 18:07 UTC, Oleg Drokin
no flags Details

Description Oleg Drokin 2019-06-23 04:23:18 UTC
Description of problem:
Had a system crash during an update (in the middle of scripts execution) that left me with a bunch of duplicate packages.

Now dnf remove --duplicates does not work.
dnf history redo $lasttransaction and dnf history rollback $lasttransaction don't work either.


remove --duplicates at first was complaining like this:

  file /usr/share/man/man1/gapplication.1.gz from install of glib2-2.60.4-1.fc30.i686 conflicts with file from package glib2-2.60.3-1.fc30.x86_64
  file /usr/share/man/man1/gdbus.1.gz from install of glib2-2.60.4-1.fc30.i686 conflicts with file from package glib2-2.60.3-1.fc30.x86_64
  file /usr/share/man/man1/gio-querymodules.1.gz from install of glib2-2.60.4-1.fc30.i686 conflicts with file from package glib2-2.60.3-1.fc30.x86_64
  file /usr/share/man/man1/gio.1.gz from install of glib2-2.60.4-1.fc30.i686 conflicts with file from package glib2-2.60.3-1.fc30.x86_64
  file /usr/share/man/man1/glib-compile-schemas.1.gz from install of glib2-2.60.4-1.fc30.i686 conflicts with file from package glib2-2.60.3-1.fc30.x86_64
  file /usr/share/man/man1/gsettings.1.gz from install of glib2-2.60.4-1.fc30.i686 conflicts with file from package glib2-2.60.3-1.fc30.x86_64
  file /usr/share/xml/fontconfig/fonts.dtd from install of fontconfig-2.13.1-8.fc30.i686 conflicts with file from package fontconfig-2.13.1-6.fc30.x86_64

(basically a lot of what looked like same version i686 vs x86_64 packages.

history redo was unhappy about the same, rollback was unhappy about some missing kernel packages (installing them did not help, it then became unhappy about some kmod-nvidia that got uninstalled by installation of the said kernel packages).

Now I did dng remove glibc.i686 to remove all the i686 packages in hopes it'll clear the dnf remove --duplicates, but now it's still unhappy about same duplicate versions sharing files (duh, of course they share files, they are duplicates after all):

(really lots of output like below so I am not including it all):
Error: Transaction check error:
...
  file /usr/share/man/man7/nmcli-examples.7.gz from install of NetworkManager-1:1.16.2-1.fc30.x86_64 conflicts with file from package NetworkManager-1:1.16.0-1.fc30.x86_64
  file /usr/share/man/man8/NetworkManager.8.gz from install of NetworkManager-1:1.16.2-1.fc30.x86_64 conflicts with file from package NetworkManager-1:1.16.0-1.fc30.x86_64
  file /usr/share/man/man8/nm-initrd-generator.8.gz from install of NetworkManager-1:1.16.2-1.fc30.x86_64 conflicts with file from package NetworkManager-1:1.16.0-1.fc30.x86_64


Version-Release number of selected component (if applicable):
# dnf --version
4.2.5
  Installed: dnf-0:4.2.5-1.fc30.noarch at Fri 10 May 2019 09:12:35 PM GMT
  Built    : Fedora Project at Tue 07 May 2019 03:11:38 PM GMT

  Installed: rpm-0:4.14.2.1-4.fc30.1.x86_64 at Wed 01 May 2019 03:04:34 AM GMT
  Built    : Fedora Project at Sat 02 Feb 2019 06:46:11 PM GMT


Additional info:

So I now have this semi-hosed system that I hope could be recovered.

any ideas?

I looked at #12750525 that claims ther was a bunch of improvements made recently and apparently none of them help. I have no problems downloading same version of packages I have duplicate installs because they are the current packages I just downloaded for that update anyway.

Comment 1 Jaroslav Mracek 2019-06-27 16:44:41 UTC
I am sorry but it failed in RPM during the transaction. Probably we can ask RPM team for some suggestion. In general dnf identify duplicates and then reinstall the newer package and remove all older packages. Probaly additional information about duplicates (including their exact versions) will be required.

Comment 2 Oleg Drokin 2019-06-27 16:51:09 UTC
Created attachment 1585292 [details]
full dnf remove --duplicates output

Hm, ok, how can I gather the information?

here's the full output of the dnf remove --duplicates that sohuld list all the packages it wanted to uninstall and the files conflicting.

any idea why dnf history redo/undo did not work too?

Comment 3 Jaroslav Mracek 2019-06-27 17:30:06 UTC
`rpm -qa`
It provide a list of all packages

`sudo dnf repoquery --duplicates`
provide a list of all packages


You can also try `sudo dnf remove %(sudo dnf repoquery --duplicates --latest-limit=-1)`

and then `sudo dnf reinstall "*"` to ensure all dependency is present on the system
and sudo dnf upgrade

Comment 4 Oleg Drokin 2019-06-27 17:40:20 UTC
Created attachment 1585312 [details]
rpm -qa output

this is rpm -qa output

Comment 5 Oleg Drokin 2019-06-27 17:40:54 UTC
Created attachment 1585313 [details]
dnf repoquery --duplicates

this is dnf repoquery --duplicates output

Comment 6 Oleg Drokin 2019-06-27 17:45:09 UTC
sudo dnf remove $(sudo dnf repoquery --duplicates --latest-limit=-1)

removed a lot of stuff! (875 packages, 5.1G space freed)

dnf reinstall "*" wants to only install 47 packages (and then proceed to download 1379 packages, huh?), 4.3G space to use.

We'll see how it ends.

meanwhile I also have lvm snapshot of the system before I performed those steps for the RPM guys to get more data if needed.

Comment 7 Oleg Drokin 2019-06-27 18:05:46 UTC
ok, so it finished. the end result is it wiped a bunch of stuff including my gnome DE, openssh-server and such.

going to attach rpm -qa after the above two steps and dnf update.

dnf groupinstall "Gnome Desktop Environment" installed another 220+ packages, but attempting to start gdm still does not work with a "Something did not work, please logout" window, so I bet it is missing more stuff.

is there a quick way to identify what packages I had that I don't anymore and install just those? Thanks.

Comment 8 Oleg Drokin 2019-06-27 18:07:05 UTC
Created attachment 1585315 [details]
rpm -qa output after the two mentioned dnf steps and dnf update

Gnome, openssh-server and a bunch of stuff is wiped now.

Comment 9 Panu Matilainen 2019-06-28 07:38:45 UTC
> remove --duplicates at first was complaining like this:
>
>  file /usr/share/man/man1/gapplication.1.gz from install of glib2-2.60.4-1.fc30.i686 conflicts with file from package
> glib2-2.60.3-1.fc30.x86_64

Removal does not cause file conflicts, ever. So dnf is trying to install something on duplicate removal.
The second thing here is that it's trying to install different versions of i686 vs x86_64 package, that's generally not going to work. That's basically bug 1653683.

In the latter part:
> file /usr/share/man/man7/nmcli-examples.7.gz from install of NetworkManager-1:1.16.2-1.fc30.x86_64 conflicts with file from package NetworkManager-1:1.16.0-1.fc30.x86_64

I guess dnf is trying to reinstall a different version of a package than what's installed. That's not a reinstall, it's an upgrade/downgrade.

Comment 10 Jaroslav Mracek 2019-07-04 07:20:08 UTC
Please could you provide files generated by 'dnf remove --duplicates --debugsolver'. Thanks a lot.

But according to the latest comments it looks like that your system was restored, therefore no additional information. Am I correct?

Comment 11 Oleg Drokin 2019-07-04 13:08:37 UTC
I made an lvm snapshot specifically for this, so I am able to gather debug info for you.

Of course now that some time has passed some packages are no longer available:

# dnf remove --duplicates --debugsolver 
Last metadata expiration check: 0:03:26 ago on Thu 04 Jul 2019 09:03:21 AM EDT.
Installed package mesa-libEGL-19.0.6-1.fc30.x86_64 not available.
Installed package mesa-libgbm-19.0.6-1.fc30.x86_64 not available.
Installed package flatpak-selinux-1.4.1-1.fc30.x86_64 not available.
Installed package mesa-libglapi-19.0.6-1.fc30.x86_64 not available.
Error: 
 Problem 1: problem with installed package mesa-libOpenCL-19.0.5-2.fc30.x86_64
  - package mesa-libOpenCL-19.0.2-3.fc30.x86_64 requires mesa-libgbm(x86-64) = 19.0.2-3.fc30, but none of the providers can be installed
  - package mesa-libOpenCL-19.0.8-1.fc30.x86_64 requires mesa-libgbm(x86-64) = 19.0.8-1.fc30, but none of the providers can be installed
  - package mesa-libOpenCL-19.0.5-2.fc30.x86_64 requires mesa-libgbm(x86-64) = 19.0.5-2.fc30, but none of the providers can be installed
  - conflicting requests
 Problem 2: package xorg-x11-drv-nvidia-libs-3:430.26-1.fc30.x86_64 requires mesa-libGLES(x86-64) >= 13.0.3-3, but none of the providers can be installed
  - problem with installed package xorg-x11-drv-nvidia-libs-3:418.74-1.fc30.x86_64
  - package mesa-libGLES-19.0.2-3.fc30.x86_64 requires mesa-libglapi(x86-64) = 19.0.2-3.fc30, but none of the providers can be installed
  - package mesa-libGLES-19.0.8-1.fc30.x86_64 requires mesa-libglapi(x86-64) = 19.0.8-1.fc30, but none of the providers can be installed
  - package mesa-libGLES-19.0.5-2.fc30.x86_64 requires mesa-libglapi(x86-64) = 19.0.5-2.fc30, but none of the providers can be installed
  - package xorg-x11-drv-nvidia-libs-3:418.56-1.fc30.x86_64 requires libnvidia-fatbinaryloader.so.418.56()(64bit), but none of the providers can be installed
  - package xorg-x11-drv-nvidia-libs-3:418.74-1.fc30.x86_64 requires libnvidia-fatbinaryloader.so.418.74()(64bit), but none of the providers can be installed
  - conflicting requests
(try to add '--skip-broken' to skip uninstallable packages)


additional annoyance, when I mount my environment readonly, dnf is unhappy:
[root@mnt ~]# dnf remove --duplicates
Config error: [Errno 30] Read-only file system: '/var/log/dnf.log': '/var/log/dnf.log'

Comment 12 Jaroslav Mracek 2019-07-23 11:59:46 UTC
I believe that the issue will be fully resolved by adding complete transaction functionality.

Probably it can be resolved by "dnf shell" just add:
remove --duplicates
upgrade

Comment 13 Jaroslav Mracek 2019-09-07 14:47:49 UTC

*** This bug has been marked as a duplicate of bug 1674296 ***


Note You need to log in before you can comment on or make changes to this bug.