Bug 1897130

Hi Karen, are you experience this issue in a Red Hat Enterprise Linux Host (RHEL + rhv-packages) or on the Red Hat Virtualization Host (RHVH)?

Checking this on RHVH-4.4 (RHVH-4.4-20200930.0-RHVH-x86_64-dvd1.iso) the request working fine for me.


# rpm -q --queryformat "%{VENDOR}\n" libvirt 
Red Hat, Inc.

Could you please provide more information on your correct host-setup.
Thanks

I'm using Rhel Hosts (RHEL + rhv-packages)

Hi Karen,

according to the Package Browser on the Customer Portal there should be a 'libvirt' package for RHEL8:
https://access.redhat.com/downloads/content/libvirt/6.0.0-28.module+el8.3.0+7827+5e65edd7/x86_64/fd431d51/package

Therefore I'm a bit surprised that rpm claims that there is no libvirt package installed.

To get a better idea whats going on can you provide thev output of the following command from your RHEL host:
# rpm -qa libvirt

The output for that exact command is nothing and as you can see the output for the other:
[root@proton10 ~]# rpm -qa libvirt
[root@proton10 ~]# rpm -qa libvirt*
libvirt-daemon-driver-storage-iscsi-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-network-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-client-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-qemu-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-storage-logical-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-interface-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-bash-completion-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-admin-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-nwfilter-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-storage-rbd-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-storage-scsi-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-lock-sanlock-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-glib-3.0.0-1.el8.x86_64
libvirt-daemon-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-config-nwfilter-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-storage-iscsi-direct-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-storage-disk-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-secret-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-storage-core-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-storage-mpath-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-nodedev-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-kvm-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-libs-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-storage-gluster-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-driver-storage-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64
libvirt-daemon-config-network-6.0.0-25.4.module+el8.2.1+8060+c0c58169.x86_64


These all come from advanced-virt-for-rhel-8-x86_64-rpms
I just checked on a regular rhel system, yes you can find the libvirt package on the rhel-8-for-x86_64-appstream-rpms

The necessary packages that are installed are handled by the engine though, so this wasn't manually done.

I can reproduce this issue on rhel8.3 host (not RHV) if the libvirt package has not been installed.  After installing the libvirt package, there's no problem.

I could manually install libvirt also and i presume this issue would be gone too but when adding a host to rhv, the engine installs what it requires on the host. 
It seems it doesn't add this package, or at least it didn't in my case, just the ones i listed above. 
I don't know if this is an issue or just a random occurrence. If anyone else has a rhv 4.4 environment with a host setup of rhel 8.2 + rhv packages, could they also check this.
If it works for you then we can close this bug otherwise we should fix this to reflect all occurrences of libvirt packages.

The reason why the libvirt package is used to get the information for the virtualization vendor is because this package was always required so far when using any form of XEN or KVM virtualization on RHEL, and we couldn't find any other method to get this information that worked across different virtualization setups (bare RHEL+XEN, bare RHEL+KVM, RHEV 3.x, RHV4.x, ...)

So if RHV4.4 no longer requires the libvirt packge to be installed on the hypervisor then it doesn't make sense for me to install the package just for the command to get the info about the virtualization vendor in vhostmd to work correctly.

Therefore my recommendation would be to try reach out to the colleagues responsible for RHV4.4 to find out if it is indeed the case that the libvirt package is no longer needed, or if it is due to a bug or something else that the package isn't installed on your system.

If it is the former then the RHV colleagues should provide suggestions for what method to use instead of the current one to retrieve the info for the virtualization vendor metric on the hypervisor.

If it is the later, and the libvirt package is missing due to a bug or something else then we should try to get this fixed, because if the package is missing by accident then there could potentially be amany other issues with RHV since functionality depending on that package will be broken.

I reproduced this in my RHV-4.4 environment.
The libvirt-package is a meta-package, which does not provide any files. It only have all additional libvirt-* packages as dependency, to ease an installation.

A RHEL-8.2 host is failing, while a RHV-H 4.4 succeed. This means, on RHV-H (imagebased hypervisor) the libvirt-package is installed. 
On the RHEL-8.2 it is not. 

@Karen: for your test I would propose to install it manually.

The host-deploy mechanism in RHV should install this packages as well. I will change this bug to have this addressed. I also sent a mail to RHV-folks.

Changing Product/Component

During `ovirt-host-deploy` the package `libvirt` is not installed.

Metrics in component `vhostmd` require this package to identify Virtualization Vendor. This is required for virtualized SAP systems.
RHV-H does have libvirt package installed.

RHV itself doesn't require libvirt package and it works fine, so it makes sense to add libvirt dependency only to VDSM vhostmd hook package only. As vhostmd hook is specialized component not mandatory for RHV, lowering severity to medium and targeting to 4.4.4

(In reply to Martin Perina from comment #11)
> RHV itself doesn't require libvirt package and it works fine, so it makes
> sense to add libvirt dependency only to VDSM vhostmd hook package only. As
> vhostmd hook is specialized component not mandatory for RHV, lowering
> severity to medium and targeting to 4.4.4

Agree in this solution. 
As SAP workload is not certified on RHV-4.4 environment by today, applying workaround of manually installing libvirt-package until 4.4.4 is released should be fine.

Karen, would this be suitable to you?

Hi Steffen, 
Yea sounds good! I informed SAP about the issue, they said it's minor and they gave the go ahead to verifying support now for NW on rhv 4.4.

With confirmation from Karen, this resolution can go ahead.

For QE, following test needs to be successful on RHEL-8.2 hosts.

  # yum install -y vhostmd
  # rpm -q --queryformat "%{VENDOR}\n" libvirt | sort -u
  Red Hat, Inc.

> The reason why the libvirt package is used to get the information for the virtualization vendor is because this package was always required so far when using any form of XEN or KVM virtualization on RHEL

This is not true. The 'vhostmd' version included in 4.3 didn't require 'libvirt' package. The change that introduced the requirement is here [1]. This is basically adding an implicit package dependency without reflecting it properly on the RPM level. This is not a bug on RHV side.

> RHV itself doesn't require libvirt package and it works fine, so it makes sense to add libvirt dependency only to VDSM vhostmd hook package only

I really don't want to do this. We always had pretty complex requirements on libvirt packages in vdsm's spec [2] and I don't want to add some metapackage we never required before out of the blue, since I can't fully predict the consequences.

> For QE, following test needs to be successful on RHEL-8.2 hosts.

>  # yum install -y vhostmd
>  # rpm -q --queryformat "%{VENDOR}\n" libvirt | sort -u
>  Red Hat, Inc.

This use case especially won't be satisfied by adding 'libvirt' metapackage as a dependency to 'vdsm-hook-vhostmd' - 'vhostmd' doesn't require the hook, it's the other way around.

I see this as 'vhostmd' bug. It's clear that it's not 'libvirt' metapackage that defines a machine's capability of acting as a virtualization host. If anything I would be aiming at 'libvirt-libs' or 'libvirt-daemon', since you can't really function without them. So changing [3] (and other 'libvirt' RPM references in the XML) to 'libvirt-libs/daemon' is my short-term fix proposal.

In the long run, a prettier solution would be to produce 'vhostmd-libvirt' and 'vhostmd-xen' packages that would require both 'vhostmd' and virtualization-vendor-specific packages. Each of them would only query the RPMs that are really expected to be there.

[1] https://github.com/vhostmd/vhostmd/commit/d1a230d6d3ca4ded376f6522688d379bac347fbe
[2] https://github.com/oVirt/vdsm/blob/8cf1da2e2b0d95ad774f8ae56b9445d30408f8a1/vdsm.spec.in#L208
[3] https://github.com/vhostmd/vhostmd/blob/99995e4ba138f43b277620bd43a096c72f354548/vhostmd.xml#L56

Comment 17 makes much more sense than the original idea, so moving the bug to platform to fix that dependency on vhostmd package

(In reply to Marcin Sobczyk from comment #17)
[...]
> I really don't want to do this. We always had pretty complex requirements on
> libvirt packages in vdsm's spec [2] and I don't want to add some metapackage
> we never required before out of the blue, since I can't fully predict the
> consequences.

for the record, we required "libvirt" in RHEL 6 (RHV 3.4)
vhostmd is fairly dated, so no wonder...

This was changes in RHEL7, "libvirt" does not require qemu-kvm, "libvirt-daemon-kvm" does.

(In reply to Michal Skrivanek from comment #20)
> (In reply to Marcin Sobczyk from comment #17)
> [...]
> > I really don't want to do this. We always had pretty complex requirements on
> > libvirt packages in vdsm's spec [2] and I don't want to add some metapackage
> > we never required before out of the blue, since I can't fully predict the
> > consequences.
> 
> for the record, we required "libvirt" in RHEL 6 (RHV 3.4)
> vhostmd is fairly dated, so no wonder...
> 
> This was changes in RHEL7, "libvirt" does not require qemu-kvm,
> "libvirt-daemon-kvm" does.

So to me changing the dependencies of vhostmd seems the correct path forward.

Rich - assigning to you directly as it seems you have dealt with vhostmd issues previously.

  Requires: libvirt

I suspect no one will be overjoyed about vhostmd having a hard
dependency on libvirt, since it pulls in a lot of stuff.  How about:

  Suggests: libvirt

It will install libvirt in the default configuration of dnf, but it
could be uninstalled.

I honestly think the real solution is to fix the command in vhostmd.xml,
but I'm not sure there is a good alternative at the moment (asking on
virt-devel).

Anyway it's fine if you want me to add either Requires/Suggests as above
for z-stream.

Dan suggested that we change the XML file instead so it does something like:

$ rpm -q --qf '%{VENDOR}\n' -qf /etc/os-release 
Fedora Project

(other files might be used).  I will need to discuss this upstream.  We
can go with the simpler z-stream fix in the meantime.

rhel 8.4.0: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=33403581

(In reply to Richard W.M. Jones from comment #25)
> rhel 8.4.0:
> https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=33403581

Hi rjones,
libvirt package has been added for the latest vhostmd(1.1-5) as dependency. Is it not proper to use 'rpm -q --qf '%{VENDOR}\n' -qf /etc/os-release '?

This would only be a simple, interim fix.

How to fix it long term is being discussed upstream
(https://github.com/vhostmd/vhostmd/issues/7) which might be using
/etc/os-release.  Actually the real problem upstream is what to do
about "VirtualizationProductInfo".

(In reply to Richard W.M. Jones from comment #27)
> This would only be a simple, interim fix.
> 
> How to fix it long term is being discussed upstream
> (https://github.com/vhostmd/vhostmd/issues/7) which might be using
> /etc/os-release.  Actually the real problem upstream is what to do
> about "VirtualizationProductInfo".

Hi Richard, I contacted SAP and I'm just waiting for feedback here for you.

actually vhostmd itself uses libvirt functions, that's part of the changes for the virtio support

from a RHEL 8.2

# rpm -qf /usr/sbin/vhostmd
vhostmd-1.1-4.el8.x86_64
# ldd -d /usr/sbin/vhostmd | grep virt
	libvirt.so.0 => /lib64/libvirt.so.0 (0x00007f9be758d000)
# rpm -qf /lib64/libvirt.so.0
libvirt-libs-4.5.0-42.module+el8.2.0+6024+15a2423f.x86_64

I guess, changing the xml would not solve the dependency issue

libvirt is a meta-package which pulls in many unrelated features, including
parts of the Xen hypervisor.

libvirt-libs just includes the client library.

So we very much do want to avoid the libvirt dependency if at all possible.

There is one failed test which I'm unable to waive:

https://dashboard.osci.redhat.com/#/artifact/brew-build/aid/33403581

Some more info from the SAP side: basically any query is fine for us that:
- returns Hypervisor product-related info/version
- works on SLES, RHEL and RHV,
- is able to get the info from the default or even minimum installation pattern
Qemu information was the next best thing to KVM-related info we could get.
Would libvirt-libs - or any other package - match these conditions ?

Verified with package:
vhostmd-1.1-5.el8.x86_64


Steps:

1. 
# yum localinstall -y vhostmd-1.1-5.el8.x86_64.rpm 
Updating Subscription Management repositories.
Unable to read consumer identity

This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.

Last metadata expiration check: 0:00:16 ago on Tue 15 Dec 2020 09:35:40 PM EST.
Dependencies resolved.
============================================================================================================================================================================================
 Package                             Architecture                       Version                                                              Repository                                Size
============================================================================================================================================================================================
Installing:
 vhostmd                             x86_64                             1.1-5.el8                                                            @commandline                              56 k
Installing dependencies:
 libvirt                             x86_64                             6.0.0-30.module+el8.4.0+8705+34397d87                                rhel8-app                                 56 k

Transaction Summary
============================================================================================================================================================================================
...


The libvirt package will be installed as the dependency of vhostmd.

2.
# rpm -q --queryformat "%{VENDOR}\n" libvirt | sort -u
Red Hat, Inc.

The zstream flag has been set, but It seems there is no corresponding bug for rhel8.3.0.z.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (vhostmd bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1681