Bug 1658652
| Summary: | Missing librbd1 dependecy for libvirt-daemon-driver-storage-rbd is causing problem with RHEL host upgrade | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Jaroslav Spanko <jspanko> | ||||||
| Component: | libvirt | Assignee: | Jiri Denemark <jdenemar> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | gaojianan <jgao> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | 7.5 | CC: | berrange, chhu, dsariel, fjin, gfidente, hearvy+f125t689xceks, hhan, jcoscia, jdenemar, johfulto, kchamart, klaas, mtessun, rrasouli, toneata, xuzhang, yafu, yalzhang | ||||||
| Target Milestone: | rc | Keywords: | ZStream | ||||||
| Target Release: | 7.6 | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | libvirt-4.5.0-11.el7 | Doc Type: | If docs needed, set a value | ||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | |||||||||
| : | 1679569 (view as bug list) | Environment: | |||||||
| Last Closed: | 2019-08-06 13:14:35 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 1670798, 1679569 | ||||||||
| Attachments: |
|
||||||||
|
Description
Jaroslav Spanko
2018-12-12 15:44:40 UTC
Created attachment 1513717 [details]
dependencies list + ansible ovirt-host-upgrade.yml output
Created attachment 1513719 [details]
dependencies list + ansible ovirt-host-upgrade.yml output
Moving to RHEL libvirt, seems like libvirt-daemon-driver-storage-rbd is missing proper requirement on librbd1 correct version. Proposing for 7.6 batch update It is caused by librbd API incompatibility. Look at following code:
#if LIBRBD_VERSION_CODE > 265
r = rbd_diff_iterate2(image, snaps[i].name, 0, info.size, 0, 1,
virStorageBackendRBDIterateCb, (void *)&diff);
#else
r = rbd_diff_iterate(image, snaps[i].name, 0, info.size,
virStorageBackendRBDIterateCb, (void *)&diff);
#endif
Libvirt codes can be compiled with LIBRBD_VERSION_CODE > 265 or LIBRBD_VERSION_CODE <= 265. However, it was compiled with librbd version <= 265 on RHEL7.4 while compiled with librbd version > 265 on RHEL7.6. They invoke different librbd API but the update of libvirt doesn't update librbd on the same time. That is why this error happened.
I thinks updating librbd will solve this problem.
For developer and QEs, the question is , how to detect these ABI incompatibility earlier, because it is common on C/C++ compiled software. See also: https://bugzilla.redhat.com/show_bug.cgi?id=1351106
To detect these 'undefined symbol' error, we can do this upgrade testing: 1. Update softwares or the system, it depends by the scenarios of customers 2. Browser all the ELF format files (the executable binary and so files), check them with ldd: # ldd -r ELF_FILE If its output is like following: undefined symbol: rbd_diff_iterate2 (/usr/lib64/libvirt/storage-backend/libvirt_storage_backend_rbd.so) undefined symbol: virStorageBackendRegister (/usr/lib64/libvirt/storage-backend/libvirt_storage_backend_rbd.so) Obviously, we catch the undefined symbol issue here. This is caused by librbd.so library: # rpm -ql librbd1-0.94.5-2.el7.x86_64 /usr/lib/udev/rules.d/50-rbd.rules /usr/lib64/librbd.so.1 /usr/lib64/librbd.so.1.0.0 # rpm -ql librbd1-10.2.5-4.el7.x86_64 /usr/lib/udev/rules.d/50-rbd.rules /usr/lib64/librbd.so.1 /usr/lib64/librbd.so.1.0.0 Even though librdb.so version is the same in both packages, they don't provide the same set of APIs and at the same time the symbols exported by librbd.so are not versioned. Thus when libvirt is built, rpm just adds librbd.so.1()(64bit) dependency, which is unfortunately satisfied by both packages. We have to workaround this by adding an explicit dependency on new enough librbd1 package. *** Bug 1673101 has been marked as a duplicate of this bug. *** Note that this bug is only tracking the resolution of the missing versioned dependency from libvirt to librbd. This assumes that the newer librbd is indeed available from the host's configured yum repository. As such this missing dep should not a problem when doing a full yum update, which would pull in all new packages. It is only a problem if selectively updating individual packages whereby you request a libvirt update but exclude a request for librbd. If the newer librbd is not available via yum that would point to a bad yum repository configuration. Having been bitten by this on Friday I'm quite perplex by Red Hat's stance:
> If the newer librbd is not available via yum that would point to a bad yum repository configuration.
One would think Red Hat reconsiders their descsion when breaking userspace. To wit libvirt should depend on the proper upstream version. There is nothing wrong with our yum config.
As I tried to explain on IRC that we need to know the steps which lead to such situation. If it is a jenkins job, we need to see what steps it is doing so that we can try to reproduce locally without jenkins. If you encounter this issue when upgrading please see https://access.redhat.com/solutions/4090431 Verified on libvirt-4.5.0-17.virtcov.el7.x86_64 [root@localhost ~]# rpm -qa |grep libvirt libvirt-daemon-3.9.0-14.virtcov.el7_5.8.x86_64 libvirt-daemon-config-nwfilter-3.9.0-14.virtcov.el7_5.8.x86_64 ... libvirt-3.9.0-14.virtcov.el7_5.8.x86_64 [root@localhost ~]# rpm -qa librbd1 librbd1-0.94.5-2.el7.x86_64 [root@localhost ~]# yum update libvirt ... librados2 x86_64 1:10.2.5-4.el7 RHEL-Server 1.8 M librbd1 x86_64 1:10.2.5-4.el7 RHEL-Server 2.4 M libvirt-client x86_64 4.5.0-17.virtcov.el7 Libvirt-CI 537 k .... [root@localhost ~]# rpm -qa libvirt librbd1 librbd1-10.2.5-4.el7.x86_64 libvirt-4.5.0-17.virtcov.el7.x86_64 Librbd1 has been updated to version 10.2.5-4.el7.x86_64 Work as expected Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:2294 |