Bug 1008350

Just wondering if there are any update for this issue? Thanks.

Yearly check-in here. I hit this bug while writing a hot backup script. It has to actually compare the output text against "Successfully pivoted" and conditionally run "rm -f <snapshot>", which makes me very uncomfortable.

This bug is going to be addressed in next major release.

Posted a proposed fix: https://www.redhat.com/archives/libvir-list/2019-November/msg00912.html

Fixed by

7d484ede20 qemu: block: enable the snapshot image deletion feature
73532dadd2 qemu: block: store the delete flag in libvirtd's status XML
9e5c98e84f qemu: block: use the delete flag to delete snapshot images if requested
cb03fd9340 qemu: block: propagate the delete flag to where it can actually be used

v5.10.0-175-gcb03fd9340

As we can see that we have added the --delete function to delete the backing store, why not just delete the display in the snapshot-list directly?
If there is a backing-chain like base<--s1<--s2<--s3<--s4
After we execute "virsh blockcommit demo vdb --wait --verbose  --top /tmp/vdb.s3  --delete --active --pivot" with --delete to delete snapshot s3,but we can still find the snap s3 in the snapshot list.
As i think,we can clear it together with the image.
How do you think of this idea?

(In reply to gaojianan from comment #12)
> As we can see that we have added the --delete function to delete the backing
> store, why not just delete the display in the snapshot-list directly?
> If there is a backing-chain like base<--s1<--s2<--s3<--s4
> After we execute "virsh blockcommit demo vdb --wait --verbose  --top
> /tmp/vdb.s3  --delete --active --pivot" with --delete to delete snapshot
> s3,but we can still find the snap s3 in the snapshot list.
> As i think,we can clear it together with the image.
> How do you think of this idea?

Maybe i have known my wrong,if there are several disks in the guest,but we can onlt delete 1 of the disk's snapshot image,but other disks still use the snapshot name.

I agree that the fact that a snapshot still comes up in snapshot-list output even after the snapshot has been committed and deleted might seem confusing (I was confused myself when I first saw it).

However, the explanation is roughly that the blockcommit command really operates on QEMU level, it just manipulates image chains without understanding the wider context.  In particular, it has no knowledge of libvirt's snapshot metadata (the XMLs normally stored under /var/lib/libvirt/qemu/snapshot) which is what the snapshot-list command uses to list snapshots.

To finish the operation and make the snapshot disappear from snapshot-list, you currently have to issue 'snapshot-delete --metadata' to remove the snapshot metadata left behind by blockcommit.

Obviously this is suboptimal, there should be a single command to handle the whole operation.  That command should be snapshot-delete which however can't do it at the time as its support of external snapshots is incomplete.  However, there's a BZ requesting to fix just that (https://bugzilla.redhat.com/show_bug.cgi?id=1519001) and chances are it might be fixed soon(ish).

Test result: PASS
[root@hp-dl320eg8-05 bz1008350]# rpm -qa | egrep "^libvirt-6|^qemu-kvm-4"
qemu-kvm-4.2.0-10.module+el8.2.0+5740+c3dff59e.x86_64
libvirt-6.0.0-5.module+el8.2.0+5765+64816f89.x86_64


Test scenarios:
1. Commit from middle to middle
2. Commit from middle to base
3. Commit from top to middle
4. Commit from top to base
5. Abort block job and see if files still exist ('blockcommit --timeout' or 'blockjob --abort')
Since snapshots' metadata is not operated by this fix according to above comments, we'll skip snapshot metadata check in this bug verification.



1. Commit from middle to middle
[root@hp-dl320eg8-05 bz1008350]# virsh domblklist vm1
 Target   Source
---------------------------------------------
 vda      /var/lib/libvirt/images/vda.qcow2

[root@hp-dl320eg8-05 bz1008350]# for i in {1..4}; do virsh snapshot-create-as vm1 snap$i --disk-only; done
Domain snapshot snap1 created
Domain snapshot snap2 created
Domain snapshot snap3 created
Domain snapshot snap4 created

[root@hp-dl320eg8-05 bz1008350]# ll /var/lib/libvirt/images/ | grep vda
-rw-r--r--. 1 qemu qemu 2612920603 Feb 20 22:05 vda.qcow2
-rw-------. 1 qemu qemu     196768 Feb 20 22:05 vda.snap1
-rw-------. 1 qemu qemu     196768 Feb 20 22:05 vda.snap2
-rw-------. 1 qemu qemu     196768 Feb 20 22:05 vda.snap3
-rw-------. 1 qemu qemu   32112640 Feb 20 22:05 vda.snap4

[root@hp-dl320eg8-05 bz1008350]# virsh blockcommit vm1 vda --top  /var/lib/libvirt/images/vda.snap3 --base /var/lib/libvirt/images/vda.snap2 --delete --verbose --wait
Block commit: [100 %]
Commit complete
[root@hp-dl320eg8-05 bz1008350]# ll /var/lib/libvirt/images/ | grep vda
-rw-r--r--. 1 qemu qemu 2612920603 Feb 20 22:05 vda.qcow2
-rw-------. 1 qemu qemu     196768 Feb 20 22:05 vda.snap1
-rw-------. 1 qemu qemu     196768 Feb 20 22:05 vda.snap2
-rw-------. 1 qemu qemu   41025536 Feb 20 22:06 vda.snap4
<==== snap3 deleted as expected

[root@hp-dl320eg8-05 bz1008350]# qemu-img info -U /var/lib/libvirt/images/vda.snap4 --backing-chain
image: /var/lib/libvirt/images/vda.snap4
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 70.9 MiB
cluster_size: 65536
backing file: /var/lib/libvirt/images/vda.snap2
...
<==== backing chain is correct

2. Commit from middle to base (start from a clean vm)
[root@hp-dl320eg8-05 bz1008350]# virsh start vm1
Domain vm1 started



[root@hp-dl320eg8-05 bz1008350]# for i in {1..4}; do virsh snapshot-create-as vm1 snap$i --disk-only; done
Domain snapshot snap1 created
Domain snapshot snap2 created
Domain snapshot snap3 created
Domain snapshot snap4 created

[root@hp-dl320eg8-05 bz1008350]# virsh blockcommit vm1 vda --top  /var/lib/libvirt/images/vda.snap3 --base /var/lib/libvirt/images/vda.qcow2 --delete --verbose --wait
Block commit: [100 %]
Commit complete


[root@hp-dl320eg8-05 bz1008350]# ll /var/lib/libvirt/images/ | grep vda
-rw-r--r--. 1 qemu qemu 2612920603 Feb 20 22:11 vda.qcow2
-rw-------. 1 qemu qemu    1835008 Feb 20 22:12 vda.snap4
<==== snap1,2,3 deleted as expected

[root@hp-dl320eg8-05 bz1008350]# qemu-img info -U /var/lib/libvirt/images/vda.snap4 --backing-chain
image: /var/lib/libvirt/images/vda.snap4
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 3.44 MiB
cluster_size: 65536
backing file: /var/lib/libvirt/images/vda.qcow2
<==== backing chain is correct

3. Commit from top to middle (start from a clean vm)
[root@hp-dl320eg8-05 bz1008350]# virsh start vm1
Domain vm1 started

[root@hp-dl320eg8-05 bz1008350]# for i in {1..4}; do virsh snapshot-create-as vm1 snap$i --disk-only; done
Domain snapshot snap1 created
Domain snapshot snap2 created
Domain snapshot snap3 created
Domain snapshot snap4 created

[root@hp-dl320eg8-05 bz1008350]# virsh blockcommit vm1 vda --top  /var/lib/libvirt/images/vda.snap4 --base /var/lib/libvirt/images/vda.snap2 --delete --verbose --wait --active
Block commit: [100 %]
Now in synchronized phase

[root@hp-dl320eg8-05 bz1008350]# ll /var/lib/libvirt/images/ | grep vda
-rw-r--r--. 1 qemu qemu 2612920603 Feb 20 22:15 vda.qcow2
-rw-------. 1 qemu qemu     196768 Feb 20 22:15 vda.snap1
-rw-------. 1 qemu qemu   34144256 Feb 20 22:16 vda.snap2
-rw-------. 1 qemu qemu     196768 Feb 20 22:15 vda.snap3
-rw-------. 1 qemu qemu   34144256 Feb 20 22:16 vda.snap4
<==== file still exist since vm still using snap4 for now, need a pivot action

[root@hp-dl320eg8-05 bz1008350]# virsh blockjob vm1 vda --pivot

[root@hp-dl320eg8-05 bz1008350]# ll /var/lib/libvirt/images/ | grep vda
-rw-r--r--. 1 qemu qemu 2612920603 Feb 20 22:15 vda.qcow2
-rw-------. 1 qemu qemu     196768 Feb 20 22:15 vda.snap1
-rw-------. 1 qemu qemu   35520512 Feb 20 22:16 vda.snap2
<==== now files deleted as expected


[root@hp-dl320eg8-05 bz1008350]# qemu-img info -U /var/lib/libvirt/images/vda.snap2 --backing-chain
image: /var/lib/libvirt/images/vda.snap2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 34.8 MiB
cluster_size: 65536
backing file: /var/lib/libvirt/images/vda.snap1
...
<==== correct backing chain

4. Commit from top to base (start from a clean vm)
[root@hp-dl320eg8-05 bz1008350]# virsh start vm1
Domain vm1 started

[root@hp-dl320eg8-05 bz1008350]# for i in {1..4}; do virsh snapshot-create-as vm1 snap$i --disk-only; done
Domain snapshot snap1 created
Domain snapshot snap2 created
Domain snapshot snap3 created
Domain snapshot snap4 created

[root@hp-dl320eg8-05 bz1008350]# virsh blockcommit vm1 vda --top  /var/lib/libvirt/images/vda.snap4 --delete --verbose --wait --active --pivot
Block commit: [100 %]
Successfully pivoted

[root@hp-dl320eg8-05 bz1008350]# ll /var/lib/libvirt/images/ | grep vda
-rw-r--r--. 1 qemu qemu 2612920603 Feb 20 22:20 vda.qcow2
<==== snaps 1,2,3,4 deleted as expected

5. Abort block job and see if file still exists ('blockcommit --timeout' or 'blockjob --abort') (start from a clean vm)
[root@hp-dl320eg8-05 bz1008350]# virsh start vm1
Domain vm1 started


[root@hp-dl320eg8-05 bz1008350]# for i in {1..4}; do virsh snapshot-create-as vm1 snap$i --disk-only; done
Domain snapshot snap1 created
Domain snapshot snap2 created
Domain snapshot snap3 created
Domain snapshot snap4 created

In vm, create some data to make sure the blockcommit will take long time
[root@localhost ~]# dd if=/dev/urandom of=/tmp/bigfile bs=1M count=99
99+0 records in
99+0 records out
103809024 bytes (104 MB, 99 MiB) copied, 0.457437 s, 227 MB/s
[root@localhost ~]# sync

[root@hp-dl320eg8-05 bz1008350]# virsh blockcommit vm1 vda --top  /var/lib/libvirt/images/vda.snap4 --delete --verbose --wait --active --pivot --timeout 2 --bandwidth 1
Block commit: [  2 %]
Commit aborted
[root@hp-dl320eg8-05 bz1008350]# ll /var/lib/libvirt/images/ | grep vda
-rw-r--r--. 1 qemu qemu 2612920603 Feb 20 22:25 vda.qcow2
-rw-------. 1 qemu qemu     196768 Feb 20 22:22 vda.snap1
-rw-------. 1 qemu qemu     196768 Feb 20 22:22 vda.snap2
-rw-------. 1 qemu qemu     196768 Feb 20 22:22 vda.snap3
-rw-------. 1 qemu qemu  107806720 Feb 20 22:25 vda.snap4
<==== snap files still exist, as expected

[root@hp-dl320eg8-05 bz1008350]# qemu-img info -U /var/lib/libvirt/images/vda.snap4 --backing-chain
image: /var/lib/libvirt/images/vda.snap4
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 115 MiB
cluster_size: 65536
backing file: /var/lib/libvirt/images/vda.snap3
...
<==== correct backing chain

6. Abort block job and see if file still exists ('blockjob --abort') (start from a clean vm)
[root@hp-dl320eg8-05 bz1008350]# virsh start vm1
Domain vm1 started

[root@hp-dl320eg8-05 bz1008350]# for i in {1..4}; do virsh snapshot-create-as vm1 snap$i --disk-only; done
Domain snapshot snap1 created
Domain snapshot snap2 created
Domain snapshot snap3 created
Domain snapshot snap4 created

In vm, create some data to make sure the blockcommit will take long time
[root@localhost ~]#  dd if=/dev/urandom of=/tmp/bigfile bs=1M count=99
99+0 records in
99+0 records out
103809024 bytes (104 MB, 99 MiB) copied, 0.461168 s, 225 MB/s
[root@localhost ~]# sync

[root@hp-dl320eg8-05 ~]# virsh blockjob vm1 vda
Active Block Commit: [  4 %]    Bandwidth limit: 1048576 bytes/s (1.000 MiB/s)

In another terminal [root@hp-dl320eg8-05 ~]# virsh blockjob vm1 vda --abort

Back to original terminal
[root@hp-dl320eg8-05 bz1008350]# virsh blockcommit vm1 vda --top  /var/lib/libvirt/images/vda.snap4 --delete --verbose --wait --active --pivot --bandwidth 1
Block commit: [ 17 %]
Commit aborted

[root@hp-dl320eg8-05 bz1008350]# ll /var/lib/libvirt/images/ | grep vda
-rw-r--r--. 1 qemu qemu 2612920603 Feb 20 22:29 vda.qcow2
-rw-------. 1 qemu qemu     196768 Feb 20 22:28 vda.snap1
-rw-------. 1 qemu qemu     196768 Feb 20 22:28 vda.snap2
-rw-------. 1 qemu qemu     196768 Feb 20 22:28 vda.snap3
-rw-------. 1 qemu qemu  139526144 Feb 20 22:30 vda.snap4
<==== all snap files exist, as expected

[root@hp-dl320eg8-05 bz1008350]# virsh domblklist vm1
 Target   Source
---------------------------------------------
 vda      /var/lib/libvirt/images/vda.snap4

[root@hp-dl320eg8-05 bz1008350]# qemu-img info -U /var/lib/libvirt/images/vda.snap4 --backing-chain
image: /var/lib/libvirt/images/vda.snap4
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 137 MiB
cluster_size: 65536
backing file: /var/lib/libvirt/images/vda.snap3
...
<==== backing chain correct

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2017