Bug 2148470
| Summary: | [leapp][PES] IPU 7->8: conflict with *some* libvirt packages | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Christophe Besson <cbesson> |
| Component: | leapp-repository | Assignee: | Leapp team <leapp-notifications> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Upgrades and Supportability <upgrades-and-supportability> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.9 | CC: | awaltlov, egolov, jdickers, lob+redhat, lpramuk, pdudley, pkubica, pstodulk, shane.seymour, soutteri, system-engineering, vsokol, ymankad |
| Target Milestone: | rc | Keywords: | Reproducer |
| Target Release: | --- | Flags: | pm-rhel:
mirror+
|
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-03-07 19:25:11 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Christophe Besson
2022-11-25 15:19:20 UTC
Hi Christophe, thanks for the report. Most likely it's related to PES data, but in this case it's maybe more complicated. We will investigate it later. It seems like a lot of Satellite's will hit this issue as these are Deps for commonly installed packages. Details from a failure here from a Satellite: ~~~ 11973:2022-11-29 09:03:04.826 DEBUG PID: 5241 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has finished: ['systemd-nspawn', '--register=no', '--quiet', '-D', '/var/lib/leapp/scratch/mounts/root_/system_overlay', '--setenv=LEAPP_UPGRADE_PATH_TARGET_RELEASE=8.6', '--setenv=LEAPP_NO_RHSM=0', '--setenv=LEAPP_EXPERIMENTAL=0', '--setenv=LEAPP_UPGRADE_PATH_FLAVOUR=default', '--setenv=LEAPP_COMMON_TOOLS=:/etc/leapp/repos.d/system_upgrade/el7toel8/tools', '--setenv=LEAPP_COMMON_FILES=:/etc/leapp/repos.d/system_upgrade/common/files:/etc/leapp/repos.d/system_upgrade/el7toel8/files', '--setenv=LEAPP_IPU_IN_PROGRESS=7to8', '--setenv=LEAPP_UNSUPPORTED=0', '--setenv=LEAPP_EXECUTION_ID=7fa01d2a-4e32-421b-9854-4255bea2e667', '--setenv=LEAPP_HOSTNAME=hostname.example.com', 'dnf', 'install', '-y', '--nogpgcheck', '--setopt=module_platform_id=platform:el8', '--setopt=keepcache=1', '--releasever', u'8.6', '--installroot', '/el8target', '--disablerepo', '*', '--enablerepo', u'satellite-maintenance-6.11-for-rhel-8-x86_64-rpms', '--enablerepo', u'advanced-virt-for-rhel-8-x86_64-rpms', '--enablerepo', u'satellite-6.11-for-rhel-8-x86_64-rpms', '--enablerepo', u'rhel-8-for-x86_64-appstream-rpms', '--enablerepo', u'rhel-8-for-x86_64-highavailability-rpms', '--enablerepo', u'rhel-8-for-x86_64-baseos-rpms', 'dnf', 'dnf-command(config-manager)'] 12188:2022-11-29 09:03:20.443 DEBUG PID: 5835 leapp.workflow.TargetTransactionCheck.dnf_transaction_check: Cannot enable multiple streams for module 'virt' 12191:2022-11-29 09:03:20.913 DEBUG PID: 5835 leapp.workflow.TargetTransactionCheck.dnf_transaction_check: broken groups or modules: virt:rhel ~~~ To solve (specific to Satellite in this case) - # rpm -e --nodeps `rpm -qa libvirt\*` - the same as the suggestion above. At the moment still reviewing the system after, but critical Satellite packages are still intact (including the Satellite package) and only other issues not related to this exist. Will update as I find more info if needed. My Satellite leapp upgrade performed successfully: # hammer ping >>> all components say OK # yum --disableplugin foreman-protector list libvirt-libs --showd Updating Subscription Management repositories. Last metadata expiration check: 0:10:44 ago on Wed 30 Nov 2022 11:10:38 AM EST. Installed Packages libvirt-libs.x86_64 8.0.0-5.5.module+el8.6.0+16828+96e76c36 @System >>> libvirt-libs got upgraded from el7 to el8 Satellites which have foreman-discovery-image rpm installed are affected ! (as foreman-discovery-image requires libvirt-daemon-kvm) In order to proceed with leapp upgrade I had to run # rpm -e --nodeps $(rpm -qa libvirt-daemon*) libvirt-libs rpm is not blocking the upgrade, only libvirt-daemon* rpms are I tried to install all libvirt* packages mentioned in this ticket and I cannot reproduce the issue at all. Do you have someone the minimal reproducer? Or a machine for the investigation? reach me on irc or gchat > '--enablerepo', u'advanced-virt-for-rhel-8-x86_64-rpms', '
> '--enablerepo', u'rhel-8-for-x86_64-highavailability-rpms',
those repos are not supported on a Satellite system!
(I'm investigating the rest, but wanted to give this small headsup)
and after installing fdi, I see libvirt upgraded correctly? libvirt-daemon x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 419 k libvirt-daemon-driver-interface x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 210 k libvirt-daemon-driver-network x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 235 k libvirt-daemon-driver-nodedev x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 220 k libvirt-daemon-driver-nwfilter x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 235 k libvirt-daemon-driver-qemu x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 923 k libvirt-daemon-driver-secret x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 198 k libvirt-daemon-driver-storage x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 66 k libvirt-daemon-driver-storage-core x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 253 k libvirt-daemon-driver-storage-disk x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 76 k libvirt-daemon-driver-storage-gluster x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 78 k libvirt-daemon-driver-storage-iscsi x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 73 k libvirt-daemon-driver-storage-logical x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 77 k libvirt-daemon-driver-storage-mpath x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 71 k libvirt-daemon-driver-storage-rbd x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 81 k libvirt-daemon-driver-storage-scsi x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 73 k libvirt-daemon-kvm x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 65 k libvirt-libs x86_64 8.0.0-10.module+el8.7.0+16689+53d59bc2 AppStream 4.7 M (In reply to Evgeni Golov from comment #15) > > '--enablerepo', u'advanced-virt-for-rhel-8-x86_64-rpms', ' > > '--enablerepo', u'rhel-8-for-x86_64-highavailability-rpms', > > those repos are not supported on a Satellite system! > > (I'm investigating the rest, but wanted to give this small headsup) if these are not supported on a satellite system, then possibly customer has multiple SKUs and uses SCA so they see all repositories even when they should not see them otherwise? anyway, it is not answering the question whether PES data is correct regarding the conflicts, which could be seen on non-satellite systems also. So, the problem is in PES data. The issue is caused when trying to enable two different streams (rhel, av) for the virt module:
~~~ snippet from the output ~~~
Cannot enable multiple streams for module 'virt'
Unable to resolve argument virt:rhel
Error: Problems in request:
broken groups or modules: virt:rhel
~~~~
~~~ snippet from the data for the dnf plugin ~~~
"enable_repos": [
"rhel-8-for-x86_64-baseos-rpms",
"advanced-virt-for-rhel-8-x86_64-rpms",
"rhel-8-for-x86_64-appstream-rpms"
],
"modules_to_enable": [
"virt:av",
"virt:rhel"
],
~~~~
Below attaching the list of "pkgname:repoid{module_stream}" based on PES data that cause problems on my system (repoids could differe a little bit on various systems).
From the list can be seen that PES events for qemu pkgs requires virt:rhel however libvirt packages require virt:av.
~~~~~~~~
{(u'virt', u'av'): [libvirt-daemon:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-network:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-bash-completion:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-client:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-config-network:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-nwfilter:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-storage-gluster:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-secret:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-nodedev:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-storage-iscsi:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-storage-scsi:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-qemu:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-storage-logical:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-libs:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-storage-mpath:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-storage-rbd:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-config-nwfilter:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-interface:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-kvm:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-storage:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-storage-core:advanced-virt-for-rhel-8-x86_64-rpms{virt,av},
libvirt-daemon-driver-storage-disk:advanced-virt-for-rhel-8-x86_64-rpms{virt,av}],
(u'virt', u'rhel'): [qemu-kvm-block-ssh:rhel-8-for-x86_64-baseos-rpms{virt,rhel},
qemu-kvm:rhel-8-for-x86_64-appstream-rpms{virt,rhel},
qemu-kvm-block-rbd:rhel-8-for-x86_64-appstream-rpms{virt,rhel},
qemu-kvm-block-iscsi:rhel-8-for-x86_64-appstream-rpms{virt,rhel},
qemu-kvm-core:rhel-8-for-x86_64-appstream-rpms{virt,rhel},
qemu-kvm-block-gluster:rhel-8-for-x86_64-appstream-rpms{virt,rhel},
qemu-kvm-block-curl:rhel-8-for-x86_64-appstream-rpms{virt,rhel}]}
~~~~~~~~~~
It seems to me that particular libvirt packages should not be probably pointing to the advanced virtualisation.
Also, the list I printed above does not have to reflect all problematic packages. most likely there are others libvirt, qemu, ... packages in this conflict. Keeping on the overview of PES data to check the whole set. (In reply to Petr Stodulka from comment #17) > (In reply to Evgeni Golov from comment #15) > > > '--enablerepo', u'advanced-virt-for-rhel-8-x86_64-rpms', ' > > > '--enablerepo', u'rhel-8-for-x86_64-highavailability-rpms', > > > > those repos are not supported on a Satellite system! > > > > (I'm investigating the rest, but wanted to give this small headsup) > > if these are not supported on a satellite system, then possibly customer has > multiple SKUs and uses SCA so they see all repositories even when they > should not see them otherwise? I thought leapp would only enable "needed" repositories? And libvirt *can* be satisfied by virt:rhel, so I kinda thought one would get virt:rhel for libvirt from rhel7-base and virt:av for libvirt from rhel7-rhev. (But that's clearly not what is in PES right now, and that's also what you write in your second comment.) > anyway, it is not answering the question whether PES data is correct > regarding the conflicts, which could be seen on non-satellite systems also. Yeah, absolutely. I am just worried that we might end up having Satellite systems upgraded into an unsupported configuration. According to https://access.redhat.com/solutions/6959344, virt:av should not be used in 8.6+, as everything is in virt:rhel. You can hit this issue with SCA if any subscription in your account provides advanced-virt-for-rhel-8-x86_64-rpms repo: # subscription-manager status +-------------------------------------------+ System Status Details +-------------------------------------------+ Overall Status: Disabled Content Access Mode is set to Simple Content Access. This host has access to content, regardless of subscription status. System Purpose Status: Disabled # subscription-manager list --consumed No consumed subscription pools were found. # leapp preupgrade ... 2022-12-05 10:52:51.637527 [ERROR] Actor: dnf_transaction_check Message: DNF execution failed with non zero exit code. STDOUT: Last metadata expiration check: 0:01:03 ago on Mon Dec 5 10:51:42 2022. STDERR: Cannot enable multiple streams for module 'virt' Unable to resolve argument virt:rhel Error: Problems in request: broken groups or modules: virt:rhel I fixed the problem by limiting subscription options and attaching Satellite specific subscription # subscription-manager attach --pool <pool_id> RE comment#11 >> libvirt-libs rpm is not blocking the upgrade, only libvirt-daemon* rpms are This statement is not true. All libvirt-* rpms are blocking the upgrade. So it means Satellite doesn't have to have FDI installed (>libvirt-deamon) to be affected Two workarounds: # rpm -e --nodeps $(rpm -qa libvirt-*) Or # subscription-manager attach --pool <satellite_specific_pool_id> (to avoid consuming AV repo in SCA enabled case or subscription is too generic and provides AV) (In reply to Lukas Pramuk from comment #22) > You can hit this issue with SCA if any subscription in your account provides > advanced-virt-for-rhel-8-x86_64-rpms repo: > > > # leapp preupgrade > ... > 2022-12-05 10:52:51.637527 [ERROR] Actor: dnf_transaction_check > Message: DNF execution failed with non zero exit code. > STDOUT: > Last metadata expiration check: 0:01:03 ago on Mon Dec 5 10:51:42 2022. > > STDERR: > Cannot enable multiple streams for module 'virt' > Unable to resolve argument virt:rhel > Error: Problems in request: > broken groups or modules: virt:rhel Right, that part I can reproduce. But that aborts the upgrade, and does not lead to Satellite removal as described in BZ#2149690? This BZ and BZ#2149690 are not related (Though I thought they are when filed it) (In reply to Lukas Pramuk from comment #26) > This BZ and BZ#2149690 are not related (Though I thought they are when filed > it) Ah, great! RE comment#24 >> Two workarounds: No there is only one. Just # rpm -e --nodeps $(rpm -qa libvirt-*) >> # subscription-manager attach --pool <satellite_specific_pool_id> (to avoid consuming AV repo in SCA enabled case or subscription is too generic and provides AV) This doesn't work at all. Setting subscription explicitly on RHEL7 has no effect on actual RHEL8 subscription in SCA case (=> preupgrade still fails) The current internal leapp data fixes the problem. I will update the ticket once with the new leapp data release. we sent the latest pes-events.json on the (new) attached case and the customer reported it worked for them. The new data has been published everywhere (leapp-data-21.tar.gz). Closing. |