Description of problem: When migrating a VM without shared storage when libvirt is orchestrating the disk copy (VIR_MIGRATE_NON_SHARED_DISK/--copy-storage-all) 'block-dirty-bitmaps' are not copied to the destination. This prevents incremental backups from working on the destination. We probably want to ensure that the bitmaps are always copied. Alternatively we could add a opt-in flag to do the copy of bitmaps.
Implemented upstream by: 9f61d14a9e qemu: migration: Migrate block dirty bitmaps corresponding to checkpoints 1475baed36 qemu: migration: Clean up temporary bitmaps when cancelling a migration f7eb3e233d tests: qemumigrationcookie: Add testing for block dirty bitmap migration add6ee37ee tests: qemustatusxml2xml: Add status XML from migration with bitmaps 516906d681 qemu: domain: Store list of temporary bitmaps for migration in status XML 49957f5f69 qemu: migration_cookie: Add helpers for transforming the cookie into migration params f5cd60d478 qemu: migration_cookie: Add XML handling for setting up bitmap migration b0104664c6 qemu: migration_params: Add infrastructure for 'dirty-bitmaps' migration feature b44e3ca306 qemu: blockjob: Use qemuMonitorBitmapRemove for single bitmap removal 7a8d7d75df qemu: monitor: Introduce qemuMonitorBitmapRemove 8da9c7456d qemu: migration: Create qcow2 v3 images for VIR_MIGRATE_NON_SHARED_DISK 253bce93d3 qemu: Probe whether an image is 'qcow2 v2' from query-named-block-nodes 9e855e7589 qemu: capabilities: Introduce QEMU_CAPS_MIGRATION_PARAM_BLOCK_BITMAP_MAPPING
Verified with libvirt-7.0.0-9.module+el8.4.0+10326+5e50a3b6.x86_64 =================================================================================================== Positive test 1. prepare 2 qcow2 image and use them in vm [root@dell-per740-01 ~]# qemu-img create -f qcow2 /var/lib/libvirt/images/vdb.qcow2 100M Formatting '/var/lib/libvirt/images/vdb.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=104857600 lazy_refcounts=off refcount_bits=16 [root@dell-per740-01 ~]# qemu-img create -f qcow2 /var/lib/libvirt/images/vdc.qcow2 100M Formatting '/var/lib/libvirt/images/vdc.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=104857600 lazy_refcounts=off refcount_bits=16 [root@dell-per740-01 ~]# virsh domblklist vm1 Target Source -------------------------------------------------------- vda /var/lib/libvirt/images/jeos-27-x86_64.qcow2 vdb /var/lib/libvirt/images/vdb.qcow2 vdc /var/lib/libvirt/images/vdc.qcow2 [root@dell-per740-01 ~]# virsh start vm1 Domain 'vm1' started 2. create checkpoint 'ck1' for vdb and vdc [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test --name ck1 --diskspec vda,checkpoint=no vdb,checkpoint=bitmap vdc,checkpoint=bitmap Domain checkpoint ck1 created 3. create 1M data in vm to vdb and vdc [root@localhost ~]# dd if=/dev/urandom of=/dev/vdb bs=1M count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210274 s, 49.9 MB/s [root@localhost ~]# dd if=/dev/urandom of=/dev/vdc bs=1M count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214923 s, 48.8 MB/s [root@localhost ~]# sync 4. create checkpoint 'ck2' for vdb [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test --name ck2 --diskspec vda,checkpoint=no vdb,checkpoint=bitmap vdc,checkpoint=no Domain checkpoint ck2 created 5. create 1M data in vm to vdb and vdc with skip=2 [root@localhost ~]# dd if=/dev/urandom of=/dev/vdb bs=1M count=1 skip=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018437 s, 56.9 MB/s [root@localhost ~]# dd if=/dev/urandom of=/dev/vdc bs=1M count=1 skip=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189918 s, 55.2 MB/s [root@localhost ~]# sync 6. create checkpoint 'ck3' for vdc [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test --name ck3 --diskspec vda,checkpoint=no vdb,checkpoint=no vdc,checkpoint=bitmap Domain checkpoint ck3 created 7. create 1M data to in vm to vdb and vdc with skip=4 [root@localhost ~]# dd if=/dev/urandom of=/dev/vdb bs=1M count=1 skip=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184957 s, 56.7 MB/s [root@localhost ~]# dd if=/dev/urandom of=/dev/vdc bs=1M count=1 skip=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195885 s, 53.5 MB/s [root@localhost ~]# sync 8. check vdb and vdc's images' bitmap hash value, as follow: [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"ck1"}}' {"id":"libvirt-388","error":{"class":"GenericError","desc":"Dirty bitmap 'ck1' not found"}} [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"ck2"}}' {"id":"libvirt-389","error":{"class":"GenericError","desc":"Dirty bitmap 'ck2' not found"}} [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"ck3"}}' {"return":{"sha256":"34f3c20909ea13cc63353cd4614c43f9069a984ce024a9128da3ebfdaada67fd"},"id":"libvirt-390"} [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-2-format","name":"ck1"}}' {"return":{"sha256":"34f3c20909ea13cc63353cd4614c43f9069a984ce024a9128da3ebfdaada67fd"},"id":"libvirt-391"} [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-2-format","name":"ck2"}}' {"return":{"sha256":"34f3c20909ea13cc63353cd4614c43f9069a984ce024a9128da3ebfdaada67fd"},"id":"libvirt-392"} [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-2-format","name":"ck3"}}' {"id":"libvirt-393","error":{"class":"GenericError","desc":"Dirty bitmap 'ck3' not found"}} 9. migrate vm to dest host [root@dell-per740-01 ~]# virsh migrate vm1 --live qemu+ssh://lenovo-sr630-10.lab.eng.pek2.redhat.com/system --verbose --copy-storage-all root.eng.pek2.redhat.com's password: Migration: [100 %] 10. check the bitmaps' hash value in dest host [root@lenovo-sr630-10 images]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"ck1"}}' {"id":"libvirt-398","error":{"class":"GenericError","desc":"Dirty bitmap 'ck1' not found"}} [root@lenovo-sr630-10 images]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"ck2"}}' {"id":"libvirt-399","error":{"class":"GenericError","desc":"Dirty bitmap 'ck2' not found"}} [root@lenovo-sr630-10 images]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"ck3"}}' {"return":{"sha256":"34f3c20909ea13cc63353cd4614c43f9069a984ce024a9128da3ebfdaada67fd"},"id":"libvirt-400"} [root@lenovo-sr630-10 images]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-2-format","name":"ck1"}}' {"return":{"sha256":"34f3c20909ea13cc63353cd4614c43f9069a984ce024a9128da3ebfdaada67fd"},"id":"libvirt-401"} [root@lenovo-sr630-10 images]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-2-format","name":"ck2"}}' {"return":{"sha256":"34f3c20909ea13cc63353cd4614c43f9069a984ce024a9128da3ebfdaada67fd"},"id":"libvirt-402"} [root@lenovo-sr630-10 images]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-2-format","name":"ck3"}}' {"id":"libvirt-403","error":{"class":"GenericError","desc":"Dirty bitmap 'ck3' not found"}} ================================================================ Negative test: Abort the migration procedure and check if bitmaps in original image have impact 1. create a checkpoint 'ck1' for vm's vdb and vdc [root@dell-per740-01 ~]# virsh domblklist vm1 Target Source -------------------------------------------------------- vda /var/lib/libvirt/images/jeos-27-x86_64.qcow2 vdb /var/lib/libvirt/images/vdb.qcow2 vdc /var/lib/libvirt/images/vdc.qcow2 [root@dell-per740-01 ~]# virsh start vm1 Domain 'vm1' started [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test --name ck1 --diskspec vda,checkpoint=no vdb,checkpoint=bitmap vdc,checkpoint=bitmap Domain checkpoint ck1 created 2. generate some data to vdb and vdc in vm [root@localhost ~]# dd if=/dev/urandom of=/dev/vdb bs=1M count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174449 s, 60.1 MB/s [root@localhost ~]# dd if=/dev/urandom of=/dev/vdc bs=1M count=2 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0286931 s, 73.1 MB/s [root@localhost ~]# sync 3. record the hash value of 'ck1' [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"ck1"}}' {"return":{"sha256":"6c5c25c6989f62f371e512a110c82491b42cb01d0d19bad80ed8f9ea31976ef8"},"id":"libvirt-386"} [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-2-format","name":"ck1"}}' {"return":{"sha256":"34f3c20909ea13cc63353cd4614c43f9069a984ce024a9128da3ebfdaada67fd"},"id":"libvirt-387"} 4. do migration, but <ctrl-c> to cancel it during the process [root@dell-per740-01 ~]# virsh migrate vm1 --live qemu+ssh://lenovo-sr630-10.lab.eng.pek2.redhat.com/system --verbose --copy-storage-all root.eng.pek2.redhat.com's password: Migration: [ 3 %]^C^C^C^C^C^C^C^C^C^C^Cerror: operation aborted: migration out: canceled by client 5. check the hash value of the bitmaps, they should keep same as step 3 [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"ck1"}}' {"return":{"sha256":"6c5c25c6989f62f371e512a110c82491b42cb01d0d19bad80ed8f9ea31976ef8"},"id":"libvirt-428"} [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-2-format","name":"ck1"}}' {"return":{"sha256":"34f3c20909ea13cc63353cd4614c43f9069a984ce024a9128da3ebfdaada67fd"},"id":"libvirt-429"} 6. do migration again, this time, stop the libvirtd service on the dest host [root@dell-per740-01 ~]# virsh migrate vm1 --live qemu+ssh://lenovo-sr630-10.lab.eng.pek2.redhat.com/system --verbose --copy-storage-all root.eng.pek2.redhat.com's password: Migration: [ 23 %]error: operation failed: migration of disk vda failed: Input/output error [root@lenovo-sr630-10 images]# systemctl stop libvirtd Warning: Stopping libvirtd.service, but it can still be activated by: libvirtd-admin.socket libvirtd.socket libvirtd-ro.socket 7. when migration failed, check the bitmaps hash value on source vm, they should keep same as step 3 [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"ck1"}}' {"return":{"sha256":"6c5c25c6989f62f371e512a110c82491b42cb01d0d19bad80ed8f9ea31976ef8"},"id":"libvirt-477"} [root@dell-per740-01 ~]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-2-format","name":"ck1"}}' {"return":{"sha256":"34f3c20909ea13cc63353cd4614c43f9069a984ce024a9128da3ebfdaada67fd"},"id":"libvirt-478"}
Hi Peter, Here are 2 questions during the test, pls help to confirm if they worked as design Q1: If the dest host has a exisiting image, and it contains existing bitmaps, the migration won't erase it. This may cause inconsistence between source and dest host. 1. scp a image with dirty bitmap to dest host，this is to make sure iamge on dest host has dirty bitmap 'ck1' [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test --name ck1 --diskspec vda,checkpoint=bitmap Domain checkpoint ck1 created [root@dell-per740-01 ~]# virsh destroy vm1 Domain 'vm1' destroyed [root@dell-per740-01 ~]# scp /var/lib/libvirt/images/jeos-27-x86_64.qcow2 root.eng.pek2.redhat.com:/var/lib/libvirt/images/jeos-27-x86_64.qcow2 Warning: Permanently added 'lenovo-sr630-10.lab.eng.pek2.redhat.com' (ECDSA) to the list of known hosts. root.eng.pek2.redhat.com's password: jeos-27-x86_64.qcow2 100% 931MB 109.5MB/s 00:08 <=== now the image on dest host should have a bitmap named 'ck1' 2. delete the dirty bitmap of the image on source host [root@dell-per740-01 ~]# virsh start vm1 Domain 'vm1' started [root@dell-per740-01 ~]# virsh checkpoint-delete vm1 ck1 Domain checkpoint ck1 deleted <=== this is to make sure the image on source host has no bitmap now. 3. do migration [root@dell-per740-01 ~]# virsh migrate vm1 --live qemu+ssh://xxxxxx.redhat.com/system --verbose --copy-storage-all root.eng.pek2.redhat.com's password: Migration: [100 %] 4. on dest host, check the image info [root@lenovo-sr630-10 images]# virsh destroy vm1 Domain 'vm1' destroyed [root@lenovo-sr630-10 images]# qemu-img info /var/lib/libvirt/images/jeos-27-x86_64.qcow2 image: /var/lib/libvirt/images/jeos-27-x86_64.qcow2 file format: qcow2 virtual size: 10 GiB (10737418240 bytes) disk size: 1.75 GiB cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false bitmaps: [0]: flags: [0]: auto name: ck1 granularity: 65536 refcount bits: 16 corrupt: false extended l2: false <=== here we can see the image still has bitmap 'ck1', this is inconsistent with source host image's current status. Q2: When migration finished, the bitmap in the source image was removed. 1. start a vm with image has no bitmaps [root@dell-per740-01 ~]# virsh domblklist vm1 Target Source -------------------------------------------------------- vda /var/lib/libvirt/images/jeos-27-x86_64.qcow2 [root@dell-per740-01 ~]# qemu-img info /var/lib/libvirt/images/jeos-27-x86_64.qcow2 image: /var/lib/libvirt/images/jeos-27-x86_64.qcow2 file format: qcow2 virtual size: 10 GiB (10737418240 bytes) disk size: 1 GiB cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false extended l2: false [root@dell-per740-01 ~]# virsh start vm1 Domain 'vm1' started 2. create a checkpoint for vm's vda [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test --name ck1 --diskspec vda,checkpoint=bitmap Domain checkpoint ck1 created 3. do migration [root@dell-per740-01 ~]# virsh migrate vm1 --live qemu+ssh://lenovo-sr630-10.lab.eng.pek2.redhat.com/system --verbose --copy-storage-all root.eng.pek2.redhat.com's password: Migration: [100 %] 4. after migration, check the image's info on source host, the bitmap not saved in that image [root@dell-per740-01 ~]# qemu-img info /var/lib/libvirt/images/jeos-27-x86_64.qcow2 image: /var/lib/libvirt/images/jeos-27-x86_64.qcow2 file format: qcow2 virtual size: 10 GiB (10737418240 bytes) disk size: 1 GiB cluster_size: 65536 Format specific information: compat: 1.1 compression type: zlib lazy refcounts: false refcount bits: 16 corrupt: false extended l2: false 5. if we use this vm on source host again, the checkpoint metadata and bitmap in image will be inconsistent [root@dell-per740-01 ~]# virsh start vm1 Domain 'vm1' started [root@dell-per740-01 ~]# virsh checkpoint-list vm1 Name Creation Time ----------------------------------- ck1 2021-03-16 08:20:58 -0400 [root@dell-per740-01 ~]# virsh checkpoint-delete vm1 ck1 error: Failed to delete checkpoint ck1 error: internal error: bitmap 'ck1' not found in backing chain of 'vda'
(In reply to yisun from comment #11) > Hi Peter, > Here are 2 questions during the test, pls help to confirm if they worked as > design > > Q1: If the dest host has a exisiting image, and it contains existing > bitmaps, the migration won't erase it. This may cause inconsistence between > source and dest host. > 1. scp a image with dirty bitmap to dest host，this is to make sure iamge on > dest host has dirty bitmap 'ck1' > [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test > --name ck1 --diskspec vda,checkpoint=bitmap > Domain checkpoint ck1 created > > [root@dell-per740-01 ~]# virsh destroy vm1 > Domain 'vm1' destroyed > > [root@dell-per740-01 ~]# scp /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > root.eng.pek2.redhat.com:/var/lib/libvirt/images/jeos-27- > x86_64.qcow2 > Warning: Permanently added 'lenovo-sr630-10.lab.eng.pek2.redhat.com' (ECDSA) > to the list of known hosts. > root.eng.pek2.redhat.com's password: > jeos-27-x86_64.qcow2 > 100% 931MB 109.5MB/s 00:08 > <=== now the image on dest host should have a bitmap named 'ck1' > > 2. delete the dirty bitmap of the image on source host > [root@dell-per740-01 ~]# virsh start vm1 > Domain 'vm1' started > > [root@dell-per740-01 ~]# virsh checkpoint-delete vm1 ck1 > Domain checkpoint ck1 deleted > <=== this is to make sure the image on source host has no bitmap now. > > 3. do migration > [root@dell-per740-01 ~]# virsh migrate vm1 --live > qemu+ssh://xxxxxx.redhat.com/system --verbose --copy-storage-all > root.eng.pek2.redhat.com's password: > Migration: [100 %] > > 4. on dest host, check the image info > [root@lenovo-sr630-10 images]# virsh destroy vm1 > Domain 'vm1' destroyed > > [root@lenovo-sr630-10 images]# qemu-img info > /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > image: /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > file format: qcow2 > virtual size: 10 GiB (10737418240 bytes) > disk size: 1.75 GiB > cluster_size: 65536 > Format specific information: > compat: 1.1 > compression type: zlib > lazy refcounts: false > bitmaps: > [0]: > flags: > [0]: auto > name: ck1 > granularity: 65536 > refcount bits: 16 > corrupt: false > extended l2: false > <=== here we can see the image still has bitmap 'ck1', this is inconsistent > with source host image's current status. This is okay. Users must create the image correctly. Specifically, the bitmap migration is best-effort so that it can be enabled by default and doesn't break any other aspect of migration. > Q2: When migration finished, the bitmap in the source image was removed. > 1. start a vm with image has no bitmaps > [root@dell-per740-01 ~]# virsh domblklist vm1 > Target Source > -------------------------------------------------------- > vda /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > > [root@dell-per740-01 ~]# qemu-img info > /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > image: /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > file format: qcow2 > virtual size: 10 GiB (10737418240 bytes) > disk size: 1 GiB > cluster_size: 65536 > Format specific information: > compat: 1.1 > compression type: zlib > lazy refcounts: false > refcount bits: 16 > corrupt: false > extended l2: false > [root@dell-per740-01 ~]# virsh start vm1 > Domain 'vm1' started > > 2. create a checkpoint for vm's vda > [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test > --name ck1 --diskspec vda,checkpoint=bitmap > Domain checkpoint ck1 created > > 3. do migration > [root@dell-per740-01 ~]# virsh migrate vm1 --live > qemu+ssh://lenovo-sr630-10.lab.eng.pek2.redhat.com/system --verbose > --copy-storage-all > root.eng.pek2.redhat.com's password: > Migration: [100 %] > > 4. after migration, check the image's info on source host, the bitmap not > saved in that image > [root@dell-per740-01 ~]# qemu-img info > /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > image: /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > file format: qcow2 > virtual size: 10 GiB (10737418240 bytes) > disk size: 1 GiB > cluster_size: 65536 > Format specific information: > compat: 1.1 > compression type: zlib > lazy refcounts: false > refcount bits: 16 > corrupt: false > extended l2: false > > 5. if we use this vm on source host again, the checkpoint metadata and > bitmap in image will be inconsistent > [root@dell-per740-01 ~]# virsh start vm1 > Domain 'vm1' started > > [root@dell-per740-01 ~]# virsh checkpoint-list vm1 > Name Creation Time > ----------------------------------- > ck1 2021-03-16 08:20:58 -0400 > > [root@dell-per740-01 ~]# virsh checkpoint-delete vm1 ck1 > error: Failed to delete checkpoint ck1 > error: internal error: bitmap 'ck1' not found in backing chain of 'vda' This is weird. Please file a new bug for this case.
(In reply to Peter Krempa from comment #12) > (In reply to yisun from comment #11) > > Hi Peter, > > Here are 2 questions during the test, pls help to confirm if they worked as > > design > > > > Q1: If the dest host has a exisiting image, and it contains existing > > bitmaps, the migration won't erase it. This may cause inconsistence between > > source and dest host. > > 1. scp a image with dirty bitmap to dest host，this is to make sure iamge on > > dest host has dirty bitmap 'ck1' > > [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test > > --name ck1 --diskspec vda,checkpoint=bitmap > > Domain checkpoint ck1 created > > > > [root@dell-per740-01 ~]# virsh destroy vm1 > > Domain 'vm1' destroyed > > > > [root@dell-per740-01 ~]# scp /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > > root.eng.pek2.redhat.com:/var/lib/libvirt/images/jeos-27- > > x86_64.qcow2 > > Warning: Permanently added 'lenovo-sr630-10.lab.eng.pek2.redhat.com' (ECDSA) > > to the list of known hosts. > > root.eng.pek2.redhat.com's password: > > jeos-27-x86_64.qcow2 > > 100% 931MB 109.5MB/s 00:08 > > <=== now the image on dest host should have a bitmap named 'ck1' > > > > 2. delete the dirty bitmap of the image on source host > > [root@dell-per740-01 ~]# virsh start vm1 > > Domain 'vm1' started > > > > [root@dell-per740-01 ~]# virsh checkpoint-delete vm1 ck1 > > Domain checkpoint ck1 deleted > > <=== this is to make sure the image on source host has no bitmap now. > > > > 3. do migration > > [root@dell-per740-01 ~]# virsh migrate vm1 --live > > qemu+ssh://xxxxxx.redhat.com/system --verbose --copy-storage-all > > root.eng.pek2.redhat.com's password: > > Migration: [100 %] > > > > 4. on dest host, check the image info > > [root@lenovo-sr630-10 images]# virsh destroy vm1 > > Domain 'vm1' destroyed > > > > [root@lenovo-sr630-10 images]# qemu-img info > > /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > > image: /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > > file format: qcow2 > > virtual size: 10 GiB (10737418240 bytes) > > disk size: 1.75 GiB > > cluster_size: 65536 > > Format specific information: > > compat: 1.1 > > compression type: zlib > > lazy refcounts: false > > bitmaps: > > [0]: > > flags: > > [0]: auto > > name: ck1 > > granularity: 65536 > > refcount bits: 16 > > corrupt: false > > extended l2: false > > <=== here we can see the image still has bitmap 'ck1', this is inconsistent > > with source host image's current status. > > This is okay. Users must create the image correctly. > > Specifically, the bitmap migration is best-effort so that it can be enabled > by default and doesn't break any other aspect of migration. > > > > Q2: When migration finished, the bitmap in the source image was removed. > > 1. start a vm with image has no bitmaps > > [root@dell-per740-01 ~]# virsh domblklist vm1 > > Target Source > > -------------------------------------------------------- > > vda /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > > > > [root@dell-per740-01 ~]# qemu-img info > > /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > > image: /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > > file format: qcow2 > > virtual size: 10 GiB (10737418240 bytes) > > disk size: 1 GiB > > cluster_size: 65536 > > Format specific information: > > compat: 1.1 > > compression type: zlib > > lazy refcounts: false > > refcount bits: 16 > > corrupt: false > > extended l2: false > > [root@dell-per740-01 ~]# virsh start vm1 > > Domain 'vm1' started > > > > 2. create a checkpoint for vm's vda > > [root@dell-per740-01 ~]# virsh checkpoint-create-as vm1 --description test > > --name ck1 --diskspec vda,checkpoint=bitmap > > Domain checkpoint ck1 created > > > > 3. do migration > > [root@dell-per740-01 ~]# virsh migrate vm1 --live > > qemu+ssh://lenovo-sr630-10.lab.eng.pek2.redhat.com/system --verbose > > --copy-storage-all > > root.eng.pek2.redhat.com's password: > > Migration: [100 %] > > > > 4. after migration, check the image's info on source host, the bitmap not > > saved in that image > > [root@dell-per740-01 ~]# qemu-img info > > /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > > image: /var/lib/libvirt/images/jeos-27-x86_64.qcow2 > > file format: qcow2 > > virtual size: 10 GiB (10737418240 bytes) > > disk size: 1 GiB > > cluster_size: 65536 > > Format specific information: > > compat: 1.1 > > compression type: zlib > > lazy refcounts: false > > refcount bits: 16 > > corrupt: false > > extended l2: false > > > > 5. if we use this vm on source host again, the checkpoint metadata and > > bitmap in image will be inconsistent > > [root@dell-per740-01 ~]# virsh start vm1 > > Domain 'vm1' started > > > > [root@dell-per740-01 ~]# virsh checkpoint-list vm1 > > Name Creation Time > > ----------------------------------- > > ck1 2021-03-16 08:20:58 -0400 > > > > [root@dell-per740-01 ~]# virsh checkpoint-delete vm1 ck1 > > error: Failed to delete checkpoint ck1 > > error: internal error: bitmap 'ck1' not found in backing chain of 'vda' > > This is weird. Please file a new bug for this case. Thanks for the info, open a new Bug 1940036 to track this, set current bz to VERIFIED then.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt:av bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:2098