Created attachment 1642672 [details] libvirtd.log Decription: [blockdev enablement] ‘block commit’ failed when vm having multiple external snapshots Versions: libvirt-5.10.0-1.module+el8.2.0+5040+bd433686.x86_64 qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb.x86_64 How reproducible: 100% Steps 1. Having a running vm (.libvirt-ci-venv-ci-runtest-jUzTYn) [root@libvirt-rhel-8 ~]# virsh start avocado-vt-vm1 Domain avocado-vt-vm1 started 2. Clear libvirtd log (.libvirt-ci-venv-ci-runtest-jUzTYn) [root@libvirt-rhel-8 ~]# echo "" > /var/log/libvirtd-debug.log 3. Create 4 external snapshots for vm (.libvirt-ci-venv-ci-runtest-jUzTYn) [root@libvirt-rhel-8 ~]# for i in {1..4}; do virsh snapshot-create-as avocado-vt-vm1 snap$i --disk-only; done Domain snapshot snap1 created Domain snapshot snap2 created Domain snapshot snap3 created Domain snapshot snap4 created (.libvirt-ci-venv-ci-runtest-jUzTYn) [root@libvirt-rhel-8 ~]# virsh dumpxml avocado-vt-vm1 | awk '/<disk/,/<\/disk/' <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.snap4' index='5'/> <backingStore type='file' index='4'> <format type='qcow2'/> <source file='/var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.snap3'/> <backingStore type='file' index='3'> <format type='qcow2'/> <source file='/var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.snap2'/> <backingStore type='file' index='2'> <format type='qcow2'/> <source file='/var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.snap1'/> <backingStore type='file' index='1'> <format type='qcow2'/> <source file='/var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.qcow2'/> <backingStore/> </backingStore> </backingStore> </backingStore> </backingStore> <target dev='vda' bus='virtio'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> 4. Try to do blockcommit from top to base, error happens (.libvirt-ci-venv-ci-runtest-jUzTYn) [root@libvirt-rhel-8 ~]# virsh blockcommit avocado-vt-vm1 vda --wait --verbose --base vda[1] --active --pivot error: internal error: unable to execute QEMU command 'block-commit': 'libvirt-1-format' is not in this backing file chain 5. Same error happens when do block commit from middle to middle, such as: # virsh blockcommit avocado-vt-vm1 vda --wait --verbose --base vda[2] —top vda[3] Expected result: Block commit should be successful Additional info: Libvirtd debug log as atachment
This is a manifestation of the qemu bug https://bugzilla.redhat.com/show_bug.cgi?id=1763937 The qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb.x86_64 package is based on qemu-4.2-rc1 source and does NOT contain commit v4.2.0-rc1-18-gae0f57f0aa (which was pushed 18 commits after rc1): commit ae0f57f0aa03571045a3accc0b22d91a4775e63c Author: Kevin Wolf <kwolf> Date: Fri Nov 8 09:36:35 2019 +0100 block: Remove 'backing': null from bs->{explicit_,}options bs->options and bs->explicit_options shouldn't contain any options for child nodes. bdrv_open_inherited() takes care to remove any options that match a child name after opening the image and the same is done when reopening. However, we miss the case of 'backing': null, which is a child option, but results in no child being created. This means that a 'backing': null remains in bs->options and bs->explicit_options. A typical use for 'backing': null is in live snapshots: blockdev-add for the qcow2 overlay makes sure not to open the backing file (because it is already opened and blockdev-snapshot will attach it). After doing a blockdev-snapshot, bs->options and bs->explicit_options become inconsistent with the actual state (bs has a backing file now, but the options still say null). On the next occasion that the image is reopened, e.g. switching it from read-write to read-only when another snapshot is taken, the option will take effect again and the node incorrectly loses its backing file. Fix bdrv_open_inherited() to remove the 'backing' option from bs->options and bs->explicit_options even for the case where it specifies that no backing file is wanted. Reported-by: Peter Krempa <pkrempa> Signed-off-by: Kevin Wolf <kwolf> Reviewed-by: Alberto Garcia <berto> Tested-by: Peter Krempa <pkrempa> Please re-test it with appropriate qemu version. An alternative test workaround is to shutdown/destroy the VM fully after creating the snapshots in step 3 and re-starting it as that fixes qemu's internal state. After that the blockjobs shoudl work. I'll keep this BZ open until it's tested. Setting test-only and moving to QA since there's nothing to do from libvirt's PoV
*** Bug 1780713 has been marked as a duplicate of this bug. ***
Test result: PASS [root@hp-dl320eg8-05 ~]# rpm -qa | egrep "^libvirt-6|^qemu-kvm-4" qemu-kvm-4.2.0-10.module+el8.2.0+5740+c3dff59e.x86_64 libvirt-6.0.0-5.module+el8.2.0+5765+64816f89.x86_64 [root@hp-dl320eg8-05 ~]# virsh list Id Name State ---------------------- 1 gls running 3 vm1 running [root@hp-dl320eg8-05 ~]# for i in {1..4}; do virsh snapshot-create-as vm1 snap$i --disk-only; done Domain snapshot snap1 created Domain snapshot snap2 created Domain snapshot snap3 created Domain snapshot snap4 created [root@hp-dl320eg8-05 ~]# virsh blockcommit vm1 vda --wait --verbose --base vda[1] --active --pivot Block commit: [100 %] Successfully pivoted
Changing this TestOnly BZ as CLOSED CURRENTRELEASE. Please reopen if the issue is not resolved.