Bug 1724808
Summary: | Compute logs show periodic libvirt rbd errors | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | David Hill <dhill> |
Component: | libvirt | Assignee: | Peter Krempa <pkrempa> |
Status: | CLOSED ERRATA | QA Contact: | Han Han <hhan> |
Severity: | medium | Docs Contact: | |
Priority: | medium | ||
Version: | 7.6 | CC: | chhu, dyuan, gkadam, igarciam, jdenemar, lmen, m.andre, mrunge, pkrempa, xuzhang |
Target Milestone: | rc | ||
Target Release: | 7.6 | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | libvirt-4.5.0-25.el7 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2020-03-31 19:58:29 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
David Hill
2019-06-27 20:58:07 UTC
From the error msg, I guess it was caused by missing the rbd storage backend driver file /usr/lib64/libvirt/storage-backend/libvirt_storage_backend_rbd.so . But I failed to reproduce it. Peter, could you have a look at it? Yes, that is the probable cause, but I'll need to investigate whether it makes sense in the code path to actually even report the error or we can just ignore it. The fix should be quite simple (e.g. add a check whether the backend is supported prior to calling the function which causes the log entry), but I didn't get around to do it yet. Fix proposed upstream: https://www.redhat.com/archives/libvir-list/2019-August/msg00542.html fixed upstream: commit 60b862cf9d6a335db65bbf2b011499dfa729ca2e Author: Peter Krempa <pkrempa> Date: Wed Aug 14 18:46:09 2019 +0200 qemu: Don't report some ignored errors in qemuDomainGetStatsOneBlockFallback The function ignores all errors from qemuStorageLimitsRefresh by calling virResetLastError. This still logs them. Since qemuStorageLimitsRefresh allows suppressing some, do so. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Ján Tomko <jtomko> commit ea26e22f94cea989b566e13e1930d76e06baf2c9 Author: Peter Krempa <pkrempa> Date: Wed Aug 14 18:15:20 2019 +0200 qemu: Allow suppressing errors from qemuStorageLimitsRefresh qemuStorageLimitsRefresh uses qemuDomainStorageOpenStat internally and there are callers which don't care about the error. Propagate the skipInaccessible flag so that we can log less errors. Callers currently don't care about the return value change. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Ján Tomko <jtomko> commit 3d7ea4165b890880f43074e1e5e6e10bf0ad21d6 Author: Peter Krempa <pkrempa> Date: Wed Aug 14 17:57:18 2019 +0200 qemu: driver: Improve error suppression in qemuDomainStorageUpdatePhysical None of the callers of qemuDomainStorageUpdatePhysical care about errors. Use the new flag for qemuDomainStorageOpenStat which suppresses some errors and move the reset of the rest of the uncommon errors into this function. Document what is happening in a comment for the function. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Ján Tomko <jtomko> commit b074363136ddcf1599c36f11b9f5a5fe45c5b3e5 Author: Peter Krempa <pkrempa> Date: Wed Aug 14 17:51:23 2019 +0200 util: storagefile: Don't report errors from virStorageSourceUpdatePhysicalSize virStorageSourceUpdatePhysicalSize is called only from qemuDomainStorageUpdatePhysical and all callers of it reset the libvirt error if -1 is returned. Don't bother setting the error in the first place. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Ján Tomko <jtomko> commit ba6c12df2ceb4df9bfb7ce95deef04f96bf29462 Author: Peter Krempa <pkrempa> Date: Wed Aug 14 16:00:27 2019 +0200 qemu: Allow skipping some errors in qemuDomainStorageOpenStat Some callers of this function actually don't care about errors and reset it. The message is still logged which might irritate users in this case. Add a boolean flag which will do few checks whether it actually makes sense to even try opening the storage file. For local files we check whether it exists and for remote files we at first see whether we even have a storage driver backend for it in the first place before trying to open it. Other problems will still report errors but these are the most common scenarios which can happen here. This patch changes the return value of the function so that the caller is able to differentiate the possibilities. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Ján Tomko <jtomko> commit 68639829c69ba3ed369413771bf0abceb33c3668 Author: Peter Krempa <pkrempa> Date: Tue Aug 13 12:41:40 2019 +0200 util: Export virStorageFileSupportsBackingChainTraversal The function will be reused in the qemu snapshot code. The argument is turned into const similarly to the other virStorageFileSupports* functions. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Ján Tomko <jtomko> commit e776194ad2501c9dcba8b4d9f1272342cd7f7c41 Author: Peter Krempa <pkrempa> Date: Mon Jul 29 17:43:22 2019 +0200 util: storage: Allow checking whether virStorageFileCreate is supported Add virStorageFileSupportsCreate which allows silent check whether virStorageFileCreate is implemented. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Ján Tomko <jtomko> commit d30e0d3abc0cb082d8d97c53dcff2b978e3eb372 Author: Peter Krempa <pkrempa> Date: Mon Jul 29 17:28:34 2019 +0200 util: storage: Refactor logic for using virStorageFileGetBackendForSupportCheck Modify the return value so that callers don't have to repeat logic. Signed-off-by: Peter Krempa <pkrempa> Reviewed-by: Ján Tomko <jtomko> v5.6.0-203-g60b862cf9d Reproduced on libvirt-4.5.0-24.el7.x86_64 qemu-kvm-rhev-2.12.0-42.el7.x86_64 Steps: Prepare a vm with rbd backed disk. scenarios: 1. When vm is active, call `virsh domblkinfo` to get the rbd disk info 2. When vm is inactive, call `virsh domblkinfo` to get the rbd disk info 3. When vm is inactive, call `virsh domstats` to get domain stats Verify on libvirt-4.5.0-31.el7.x86_64: 1. Keep vm active, check domstats and domblkinfo command: Set libvirtd log conf as following: log_level = 3 log_filters="1:qemu 1:libvirt 4:object 4:json 4:event 1:util" ➜ ~ virsh domstats pc --block Domain: 'pc' block.count=2 block.0.name=sda block.0.path=/var/lib/libvirt/images/pc.qcow2 block.0.rd.reqs=7561 block.0.rd.bytes=193991168 block.0.rd.times=1825193494 block.0.wr.reqs=260 block.0.wr.bytes=32218112 block.0.wr.times=54587758 block.0.fl.reqs=66 block.0.fl.times=1063439482 block.0.allocation=866947072 block.0.capacity=10737418240 block.0.physical=832176128 block.1.name=vdb block.1.rd.reqs=110 block.1.rd.bytes=3588608 block.1.rd.times=15953568 block.1.wr.reqs=0 block.1.wr.bytes=0 block.1.wr.times=0 block.1.fl.reqs=0 block.1.fl.times=0 block.1.allocation=0 block.1.capacity=209715200 block.1.physical=209715200 ➜ ~ virsh domblkinfo pc vdb Capacity: 209715200 Allocation: 209715200 Physical: 209715200 No error like 'internal error: missing storage backend for network files using rbd protocol' is found in libvirtd log. 2. Keep vm inactive. Check domstats and domblkinfo command: # virsh domstats pc --block Domain: 'pc' block.count=2 block.0.name=sda block.0.path=/var/lib/libvirt/images/pc.qcow2 block.0.allocation=832176128 block.0.capacity=10737418240 block.0.physical=866975744 block.1.name=vdb # virsh domblkinfo pc vdb error: internal error: missing storage backend for network files using rbd protocol Error only appears on inactive domblkinfo. Verified as comment15. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:1094 |