Bug 1844533

Summary: yum debug-restore removes all but one kernel even though the dump has multiple kernels.
Product: Red Hat Enterprise Linux 8 Reporter: jcastran
Component: dnfAssignee: Marek Blaha <mblaha>
Status: CLOSED ERRATA QA Contact: Eva Mrakova <emrakova>
Severity: medium Docs Contact:
Priority: high    
Version: 8.2CC: dmach, james.antill, joedward, kwalker, lagordon, mblaha, nsella, tom.giunta
Target Milestone: rcKeywords: Triaged
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: dnf-plugins-core-4.0.17-4.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-04 01:53:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description jcastran 2020-06-05 16:10:36 UTC
Description of problem:
yum debug-restore will remove kernels even if the dump includes them.



Version-Release number of selected component (if applicable):


How reproducible:
Everytime

Steps to Reproduce:
1. Have multiple kernels installed
2. yum debug-dump
3. yum debug-restore /root/dnf_debug_dump*

[root@r8 ~]# rpm -q kernel
kernel-4.18.0-147.0.3.el8_1.x86_64
kernel-4.18.0-80.el8.x86_64
kernel-4.18.0-147.5.1.el8_1.x86_64

[root@r8 ~]# dnf debug-dump
Output written to: /root/dnf_debug_dump-r8-2020-06-05_12:02:07.txt.gz

[root@r8 ~]# yum debug-restore /root/dnf_debug_dump-r8-2020-06-05_12:02:07.txt.gz

Actual results:
============================================================================================
 Package          Arch     Version                   Repository                        Size
============================================================================================
Removing:
 kernel           x86_64   4.18.0-80.el8             @rhel-8-for-x86_64-baseos-rpms     0
 kernel           x86_64   4.18.0-147.0.3.el8_1      @rhel-8-for-x86_64-baseos-rpms     0
 kernel-core      x86_64   4.18.0-80.el8             @anaconda                         57 M
 kernel-core      x86_64   4.18.0-147.0.3.el8_1      @rhel-8-for-x86_64-baseos-rpms    58 M
 kernel-modules   x86_64   4.18.0-80.el8             @rhel-8-for-x86_64-baseos-rpms    19 M
 kernel-modules   x86_64   4.18.0-147.0.3.el8_1      @rhel-8-for-x86_64-baseos-rpms    20 M

Transaction Summary
============================================================================================
Remove  6 Packages


Expected results:
It should restore the database dump. If 3 kernels were installed, it should restore 3. 

Additional info:

Comment 6 Jonathan Edwards 2020-06-17 03:09:49 UTC
Issue also appears in upstream dnf:
[root@aragorn t]# rpm -q kernel
kernel-5.6.15-300.fc32.x86_64
kernel-5.6.16-300.fc32.x86_64
kernel-5.6.18-300.fc32.x86_64
[root@aragorn t]# dnf --version
4.2.23
  Installed: dnf-0:4.2.23-1.fc32.noarch at Tue 09 Jun 2020 01:21:38 AM GMT
  Built    : Fedora Project at Tue 02 Jun 2020 07:50:03 PM GMT

  Installed: rpm-0:4.15.1-3.fc32.1.x86_64 at Fri 08 May 2020 12:39:43 PM GMT
  Built    : Fedora Project at Mon 04 May 2020 07:32:48 AM GMT
[root@aragorn t]# dnf debug-dump
Updating Subscription Management repositories.
Last metadata expiration check: 2:52:34 ago on Tue 16 Jun 2020 08:12:01 PM EDT.
Output written to: /home/jone/Downloads/dnf_debug_problem/t/dnf_debug_dump-aragorn-2020-06-16_23:04:36.txt.gz
[root@aragorn t]# dnf debug-restore dnf_debug_dump-aragorn-2020-06-16_23:04:36.txt.gz --output
Updating Subscription Management repositories.
Last metadata expiration check: 2:53:24 ago on Tue 16 Jun 2020 08:12:01 PM EDT.
remove    kernel-0:5.6.15-300.fc32.x86_64
remove    kernel-0:5.6.16-300.fc32.x86_64
remove    kernel-core-0:5.6.15-300.fc32.x86_64
remove    kernel-core-0:5.6.16-300.fc32.x86_64
remove    kernel-modules-0:5.6.15-300.fc32.x86_64
remove    kernel-modules-0:5.6.16-300.fc32.x86_64
remove    kernel-modules-extra-0:5.6.15-300.fc32.x86_64
remove    kernel-modules-extra-0:5.6.16-300.fc32.x86_64

note that the previous yum-debug-restore in RHEL7 had additional logic in pkg_data2list to deal with installonly packages

Comment 8 Marek Blaha 2020-06-17 14:07:47 UTC
PR to fix multiple problems with `dnf debug-restore`: https://github.com/rpm-software-management/dnf-plugins-core/pull/402

Comment 9 Marek Blaha 2020-06-18 11:59:34 UTC
PR with debug-dump / debug-restore tests: https://github.com/rpm-software-management/ci-dnf-stack/pull/854

Comment 12 Tom Giunta 2020-06-25 15:05:19 UTC
why is the default behavior to remove install-only packages? See line 229-230 here: https://github.com/rpm-software-management/dnf-plugins-core/pull/402/files

This behavior causes issues with the tool attempting to remove the running kernel. Shouldn't the script take into account the 'installonly_limit' set in the dnf conf? (changing True to False on line 230 allows the transaction to go through, but I'm not sure it accounts for installonly_limit) - see output below using pdb for debugging the patched debug.py script:


[root@752916v81 ~]# dnf debug-restore dnf_debug_dump.txt
Updating Subscription Management repositories.
Unable to read consumer identity
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Last metadata expiration check: 0:00:13 ago on Wed 24 Jun 2020 08:18:19 PM EDT.
> /usr/lib/python3.6/site-packages/dnf-plugins/debug.py(201)run()
-> if self.opts.filter_types:
(Pdb) n
> /usr/lib/python3.6/site-packages/dnf-plugins/debug.py(202)run()
-> self.opts.filter_types = set(
(Pdb) n
> /usr/lib/python3.6/site-packages/dnf-plugins/debug.py(203)run()
-> self.opts.filter_types.replace(",", " ").split())
(Pdb) n
> /usr/lib/python3.6/site-packages/dnf-plugins/debug.py(205)run()
-> dump_pkgs = self.read_dump_file(self.opts.filename[0])
(Pdb) n
> /usr/lib/python3.6/site-packages/dnf-plugins/debug.py(207)run()
-> self.process_installed(dump_pkgs, self.opts)
(Pdb) n
> /usr/lib/python3.6/site-packages/dnf-plugins/debug.py(209)run()
-> self.process_dump(dump_pkgs, self.opts)
(Pdb) n
> /usr/lib/python3.6/site-packages/dnf-plugins/debug.py(211)run()
-> if not self.opts.output:
(Pdb) n
> /usr/lib/python3.6/site-packages/dnf-plugins/debug.py(212)run()
-> self.base.resolve()
(Pdb) n
dnf.exceptions.DepsolveError:
Problem: The operation would result in removing the following protected packages: kernel-core
> /usr/lib/python3.6/site-packages/dnf-plugins/debug.py(212)run()

Comment 14 Marek Blaha 2020-06-26 07:51:49 UTC
Yes, attempt to remove the running kernel (protection of which is hard coded in the dnf) is just one of the problems with this command. The root cause is that command itself is far too simple for replicating installed packages between two systems (and was not originally designed to be one). There are other issues with this tool:

- it does not replicate groups installed (even does not dump them)
- it is completely unaware of modules
- the restoring on target system is kind of fragile (depends on the same repositories set enabled on both systems)
- does not preserve the reason why original packages were installed (the reason could be user-installed, dependency or part of the group). On target system all new packages are considered user-installed.

I'll try modify the patch to workaround the problem with removal of the running kernel - your suggestion about not removing the install-only packages at all is probably the way to go. Once the patch is ready, I will update the copr repository with the new version for testing.
As far as the other issues are concerned it looks like we need a new, proper tool designed to do "system replication".

Comment 19 Marek Blaha 2020-07-02 13:53:52 UTC
New PR which prevents attempts to remove the running kernel (and also with improved error messages from debug-restore command): https://github.com/rpm-software-management/dnf-plugins-core/pull/403
Tests: https://github.com/rpm-software-management/ci-dnf-stack/pull/862

Comment 28 errata-xmlrpc 2020-11-04 01:53:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (yum bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4510