Bug 1850219 - No way to install non-modular libvirt component via Kickstart [NEEDINFO]
Summary: No way to install non-modular libvirt component via Kickstart
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: dnf
Version: 8.2
Hardware: All
OS: Linux
high
medium
Target Milestone: rc
: 8.3
Assignee: Packaging Maintenance Team
QA Contact: swm-qe
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-06-23 18:29 UTC by Jonathan Mills
Modified: 2021-02-16 14:08 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-14 11:06:46 UTC
Type: Bug
Target Upstream Version:
jmracek: needinfo? (jonmills)


Attachments (Terms of Use)

Description Jonathan Mills 2020-06-23 18:29:59 UTC
Description of problem:  

No way to install newer libvirt versions via Kickstart


Version-Release number of selected component (if applicable): 

RHEL 8.2


How reproducible:  

Always


Steps to Reproduce:

1.  Write a kickstart file with the following url and repo tags:

url --url http://<yourserver>/install/centos/8.2/BaseOS/x86_64/os/

# Yum repos
repo --name="advanced-virtualization" --baseurl=http://<yourserver>/install/centos/8.2/virt/x86_64/advanced-virtualization/ --cost 100 --install
#repo --name="AppStream" --baseurl=http://<yourserver>/install/centos/8.2/AppStream/x86_64/os --cost=101 --install
repo --name="extras" --baseurl=http://<yourserver>/install/centos/8.2/extras/x86_64/os/ --cost=500 --install
repo --name="openstack-train" --baseurl=http://<yourserver>/install/centos/8.2/cloud/x86_64/openstack-train/ --cost=101 --install

2.  Note that the AppStream repo is commented out; also note the cost settings, wherein we attempt to make the advanced-virtualization repo have the lowest cost. This ought to result in the installer choosing a newer version of libvirt* from the advanced virtualization repo (such as libvirt 5.6 or 6.0).

3. In the kickstart file, include these groups in your package list:

@base
@base-x
@container-management
@core
@development
@hardware-monitoring
@network-file-system-client
@network-tools
@performance
@platform-devel
@remote-system-management
@security-tools
@virtualization-client
@virtualization-hypervisor
@virtualization-tools

4. PXE --> Kickstart your test node

Actual results:

Anaconda installer always chooses libvirt* 4.5.* from the BaseOS repo, ignoring advanced-virtualization repo completely, ignoring cost settings, and ignoring the comparison of RPM version numbers on the libvirt RPM files

Expected results:

Expected result should be anaconda installer choosing to install libvirt* ver 6.0.* from the advanced-virtualization repo


Additional info:

While I tested with CentOS, I fully expect to see the same result in RHEL 8.  I believe this is a bug with yum/dnf's "streams" support.  I believe similar issues exist with other repos, not just advanced-virtualization. For example, the RDO repos, and I would also point you to this page:  

https://jira.mariadb.org/browse/MDEV-20673?focusedCommentId=142533&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-142533

and suggest that you search in that page for "module_hotfixes=true".  They also make reference to https://dnf.readthedocs.io/en/latest/modularity.html#hotfix-repositories.

My feeling is that somehow this is related to why I can't install a version of libvirt on CentOS 8 newer than 4.5 via kickstart.

I've been able to install from the advanced-virtualization repo, but only by disabling the BaseOS repo (not possible inside a kickstart) or by using yum priority settings (also not possible inside a kickstart).

Any assistance/advice would be much appreciated.

Comment 1 Jonathan Mills 2020-06-23 22:14:59 UTC
From the %post macro inside a kickstart file, I was able to get the latest libvirt* packages to install into CentOS 8.2, using the following:

# Install newer virt pkgs from advanced-virtualization repo
echo "module_hotfixes=true" >> /etc/yum.repos.d/advanced-virtualization.repo
yum clean all
yum -y --nogpgcheck group upgrade virtualization-client virtualization-hypervisor virtualization-tools


However, this seems like a real hackish way of dealing with it.

Comment 2 Jiri Denemark 2020-06-24 06:32:19 UTC
Hmm, this is not a bug in libvirt, it's more about the way libvirt or rather
the whole virt module is shipped. Danilo, can you help with this either
directly or by pointing to someone who can help?

Comment 3 yalzhang@redhat.com 2020-06-24 07:01:35 UTC
For virt module on rhel8.2, there are 2 streams, one is virt:rhel provides libvirt with * 4.5.*, another is virt:8.2 provides libvirt with *6.0.0*. And the virt:rhel is enabled by default, this is why we can only install libvirt with version "4.5" by default. To update the default setting, you can:

1) run "yum module reset virt -y" to clear the default setting and make them not disabled or enabled;
2) run "yum module install virt:8.2 -y" directly;

or 
1) run "yum module reset virt -y" to clear the default setting;
2) run "yum module enable virt:8.2 -y";
3) run "yum module install virt -y"

Comment 4 Jonathan Mills 2020-06-24 12:45:38 UTC
(In reply to Jiri Denemark from comment #2)
> Hmm, this is not a bug in libvirt, it's more about the way libvirt or rather
> the whole virt module is shipped. Danilo, can you help with this either
> directly or by pointing to someone who can help?

Understood.  My apologies for putting it in the wrong category.

Comment 5 Jonathan Mills 2020-06-24 12:51:15 UTC
(In reply to yalzhang from comment #3)
> For virt module on rhel8.2, there are 2 streams, one is virt:rhel provides
> libvirt with * 4.5.*, another is virt:8.2 provides libvirt with *6.0.0*. And
> the virt:rhel is enabled by default, this is why we can only install libvirt
> with version "4.5" by default. To update the default setting, you can:
> 
> 1) run "yum module reset virt -y" to clear the default setting and make them
> not disabled or enabled;
> 2) run "yum module install virt:8.2 -y" directly;
> 
> or 
> 1) run "yum module reset virt -y" to clear the default setting;
> 2) run "yum module enable virt:8.2 -y";
> 3) run "yum module install virt -y"


Right, I understand this is how you'd do it post-install.  However, my ticket was about kickstart.  What is the supported method of achieving this inside of a kickstart?  In a massive server farm, I want all the servers to come up fully, correctly installed, rather than doing a lot of post-boot fixups.


My second problem with this solution is that, for CentOS 8.2 at least, it doesn't work.  There is no virt:8.2 module:

[root@gputest ~]# yum module reset virt -y
Last metadata expiration check: 0:00:39 ago on Wed 24 Jun 2020 08:46:29 AM EDT.
Dependencies resolved.
===============================================================================================================================================
 Package                           Architecture                     Version                            Repository                         Size
===============================================================================================================================================
Resetting modules:
 virt                                                                                                                                         

Transaction Summary
===============================================================================================================================================

Complete!

[root@gputest ~]# yum module list virt
Last metadata expiration check: 0:03:32 ago on Wed 24 Jun 2020 08:46:29 AM EDT.
AppStream
Name                          Stream                           Profiles                           Summary                                      
virt                          rhel [d]                         common [d]                         Virtualization module                        

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled


[root@gputest ~]# yum module enable virt:8.2 -y
Last metadata expiration check: 0:00:49 ago on Wed 24 Jun 2020 08:46:29 AM EDT.
Error: Problems in request:
missing groups or modules: virt:8.2

Comment 6 Jiri Denemark 2020-06-24 14:56:36 UTC
It works a bit differently in CentOS as the virt:8.* module is built for Virt
SIG as advanced-virtualization. And since Danilo is building this, he might
know how to properly consume the bits.

Comment 7 Jonathan Mills 2020-06-24 16:41:49 UTC
UPDATE:  so my apologies, but there does in fact seem to be a supported way of manipulating modules inside kickstart, as per section B.3.7. "module" of this guide: 

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/performing_an_advanced_rhel_installation/kickstart-commands-and-options-reference_installing-rhel-as-an-experienced-user#module_kickstart-commands-for-system-configuration


So the only real issue that remains is that, for CentOS, the virt module appears to be broken for the 'advanced-virtualization' stream (i don't know if I'm using the right syntax yet, this is still so new to me).  But as I described above, there's no virt:8.* visible to us; unless I'm also doing that wrong.

Comment 8 Jaroslav Suchanek 2020-06-25 15:37:13 UTC
Moving to distribution component for further discussion.

Comment 10 John Ferlan 2020-07-14 11:07:49 UTC
Jeff - this has been dormant now for a bit too long and seems it should end up with someone on your team or perhaps even be transferred to a different team for resolution per Danilo's comment 9.

Comment 11 Danilo de Paula 2020-07-14 18:00:33 UTC
So, there are at least three options

1- OP has to live with the fact that modularity is broken and keep using module_hotfix=true
2- We assign this to the DNF team to allow the instalation of a package with a higher NVR, at least when the available module stream are disabled.
3- We ask for modularity support for SIGs and then create a virt module there.

From my perspective: OP will have to deal with option 1 for the foreseeable future, no matter what.
Then we should pursue option two. Reassign this to DNF and expect them to provide a fix for this. Depending on how this is implemented, could be our way out of modularity.
But, as far as I know, this was mentioned as one of those 'by design' decisions that dnf/modularity won't change.

As for option 3: I don't even know who takes care of the build tools in centos, but my instinct tells me that want less modularity/modules, not more.

Comment 12 Jeff Nelson 2020-07-14 19:02:03 UTC
Summary: cannot override modular content with non-modular content
or
Summary: cannot build modular content with CentOS SIG build process

Problem: In CentOS, the virt:rhel stream contains qemu-kvm, libvirt and other components. The Advanced Virtualization stream, however, cannot be built in CentOS due to the fact that the CentOS SIG builds do not support modules. This forces the Advanced Virtualization SIG to build it's qemu-kvm, libvirt and other components outside of a module. But, these components do not override the modular content that is in the virt:rhel stream, even when the virt:rhel stream is disabled.

Solution 1: when a stream is disabled, dnf should allow the non-modular content to be selected
Solution 2: enhance the CentOS SIG build process to build modular content

Note that both solutions are independent and can--and probably should--coexist.

I'm going to reassign this to the dnf team for additional investigation for solution 1.

Comment 13 Daniel Mach 2020-07-20 11:06:12 UTC
(In reply to Jonathan Mills from comment #7)
> But as I described above, there's no virt:8.* visible to us; unless I'm also doing that wrong.

Could you try disabling 'virt:rhel' stream and try again?
Disabling the stream should make the non-modular packages available again.

Comment 15 Jaroslav Mracek 2020-08-11 12:50:08 UTC
I am really sorry, I set needinfos without a proper explanation. I realized that in Comment 13 the question `Could you try disabling 'virt:rhel' stream and try again?` was asked without needinfo.

Did the suggestion from Comment 13 resolved your issue?

Comment 16 Jeff Nelson 2020-08-12 13:54:25 UTC
Hi Jaroslav,

Thanks for clarifying the needinfo. I'll let the original reporter, Jonathan Mills, answer the question that was asked in comment 13.

Comment 17 jeff.taylor 2020-09-05 20:46:21 UTC
Not kicksart, but this helped me get past "missing groups or modules: virt:8.2": 
# subscription-manager repos --enable=advanced-virt-for-rhel-8-x86_64-rpms

Comment 18 Daniel Mach 2020-09-14 11:06:46 UTC
Closing based on comment#17 which describes that dnf errors out when disabling a module that is not available, which is expected.


Note You need to log in before you can comment on or make changes to this bug.