Description of problem: Based on discussion on qemu-devel list it's required that the 'dirty-bitmaps' migration capability is made configurable or at least does not use node-names for matching the source and destination image to copy the bitmaps to. The rationale (described in the linked thread) is that bitmaps track changes in guest visible data so they don't really map to node-names well, in cases when the backing chain is copied or modified at migration while keeping guest-visible state identical. Additionally node-names are not-stable accross migration as used by libvirt. Most of the points for this were raised in the following thread: https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg00990.html https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg00646.html https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg00601.html
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks
Test on qemu-kvm-5.1.0-4.module+el8.3.0+7846+ae9b566f, bug has been fixed. Test Env: kernel version:4.18.0-233.el8.x86_64 qemu-kvm version:qemu-kvm-5.1.0-4.module+el8.3.0+7846+ae9b566f Scenario 1: mapping node-name and bitmap in src. Test Steps: 1.In src, start guest with qemu cmds: ... -blockdev node-name=file_image1,driver=file,aio=threads,filename=/mnt/rhel830-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \ -device virtio-blk-pci,id=image1,drive=drive_image1,write-cache=on,bus=pcie-root-port-2,iothread=iothread0 \ ... 2.In dst. start guest with qemu cmds: ... -blockdev node-name=file_img1,driver=file,aio=threads,filename=/home/mirror.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_img1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_img1 \ -device virtio-blk-pci,id=img1,drive=drive_img1,write-cache=on,bus=pcie-root-port-2,iothread=iothread0 \ -incoming tcp:0:5000 \ ... 3.Expose dst image { "execute": "nbd-server-start", "arguments": { "addr": { "type": "inet","data": { "host": "10.66.144.75", "port": "3333" } } } } { "execute": "nbd-server-add", "arguments":{ "device": "drive_img1", "writable": true } } 4.Do mirror from src to dst {"execute":"blockdev-add","arguments":{"driver":"nbd","node-name":"mirror","server":{"type":"inet","host":"10.66.144.75","port":"3333"},"export":"drive_img1"}} {"execute": "blockdev-mirror", "arguments": { "device": "drive_image1","target": "mirror", "sync": "full", "job-id":"j1"}} {"timestamp": {"seconds": 1599725553, "microseconds": 761010}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "j1"}} {"timestamp": {"seconds": 1599725553, "microseconds": 761175}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "j1"}} {"return": {}} {"timestamp": {"seconds": 1599725630, "microseconds": 313251}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "j1"}} {"timestamp": {"seconds": 1599725630, "microseconds": 313352}, "event": "BLOCK_JOB_READY", "data": {"device": "j1", "len": 21475033088, "offset": 21475033088, "speed": 0, "type": "mirror"}} 5.Set migration capabilities in both src and dst. {"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"events","state":true},{"capability":"dirty-bitmaps","state":true}]}} 6.Add bitmap to src image and write file in src guest and check bitmap info { "execute": "block-dirty-bitmap-add", "arguments": {"node": "drive_image1", "name": "bitmap0"}} (src guest)#dd if=/dev/urandom of=test bs=1M count=1000 {"execute":"query-block"} {"return": [{"io-status": "ok", "device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/mnt/rhel830-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 6699487232, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "compression-type": "zlib", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 0}, {"recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 0}], "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/mnt/rhel830-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/image1/virtio-backend", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 0}, {"recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 0}], "type": "unknown"}]} 7.Set pre-switchover in src and do migration from src to dst {"execute":"migrate-continue","arguments":{"state":"pre-switchover"}} {"execute":"migrate-set-parameters","arguments":{"block-bitmap-mapping":[{"node-name":"drive_image1","alias":"drive_img1","bitmaps":[{"name":"bitmap0","alias":"bitmapa"}]}]}} {"execute": "migrate","arguments":{"uri": "tcp:10.66.144.75:5000"}} {"timestamp": {"seconds": 1599726335, "microseconds": 403995}, "event": "MIGRATION", "data": {"status": "setup"}} {"return": {}} {"timestamp": {"seconds": 1599726335, "microseconds": 413324}, "event": "MIGRATION_PASS", "data": {"pass": 1}} {"timestamp": {"seconds": 1599726335, "microseconds": 413582}, "event": "MIGRATION", "data": {"status": "active"}} {"timestamp": {"seconds": 1599726405, "microseconds": 878064}, "event": "MIGRATION_PASS", "data": {"pass": 2}} {"timestamp": {"seconds": 1599726407, "microseconds": 180523}, "event": "MIGRATION_PASS", "data": {"pass": 3}} {"timestamp": {"seconds": 1599726407, "microseconds": 480314}, "event": "MIGRATION_PASS", "data": {"pass": 4}} {"timestamp": {"seconds": 1599726407, "microseconds": 486676}, "event": "STOP"} {"timestamp": {"seconds": 1599726407, "microseconds": 488527}, "event": "MIGRATION", "data": {"status": "pre-switchover"}} 8.Migrate continue {"execute":"migrate-continue","arguments":{"state":"pre-switchover"}} {"return": {}} {"timestamp": {"seconds": 1599726428, "microseconds": 819726}, "event": "MIGRATION", "data": {"status": "device"}} {"timestamp": {"seconds": 1599726428, "microseconds": 820726}, "event": "MIGRATION_PASS", "data": {"pass": 5}} {"timestamp": {"seconds": 1599726428, "microseconds": 830600}, "event": "MIGRATION", "data": {"status": "completed"}} 9.Check status in dst (qemu)info status VM status: running 10.Check bitmap status in dst {"execute":"query-block"} {"return": [{"io-status": "ok", "device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/mirror.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 6701453312, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "compression-type": "zlib", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_img1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "dirty-bitmaps": [{"name": "bitmapa", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 319488000}], "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/home/mirror.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/img1/virtio-backend", "dirty-bitmaps": [{"name": "bitmapa", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 319488000}], "type": "unknown"}]} Scenario 2: mapping node-name and bitmap in dst. Test Steps： 1.In src, start guest with qemu cmds: ... -blockdev node-name=file_image1,driver=file,aio=threads,filename=/mnt/rhel830-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \ -device virtio-blk-pci,id=image1,drive=drive_image1,write-cache=on,bus=pcie-root-port-2,iothread=iothread0 \ ... 2.In dst. start guest with qemu cmds: ... -blockdev node-name=file_img1,driver=file,aio=threads,filename=/home/mirror.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_img1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_img1 \ -device virtio-blk-pci,id=img1,drive=drive_img1,write-cache=on,bus=pcie-root-port-2,iothread=iothread0 \ -incoming tcp:0:5000 \ ... 3.Expose dst image { "execute": "nbd-server-start", "arguments": { "addr": { "type": "inet","data": { "host": "10.66.144.75", "port": "3333" } } } } { "execute": "nbd-server-add", "arguments":{ "device": "drive_img1", "writable": true } } 4.Do mirror from src to dst {"execute":"blockdev-add","arguments":{"driver":"nbd","node-name":"mirror","server":{"type":"inet","host":"10.66.144.75","port":"3333"},"export":"drive_img1"}} {"execute": "blockdev-mirror", "arguments": { "device": "drive_image1","target": "mirror", "sync": "full", "job-id":"j1"}} {"timestamp": {"seconds": 1599725553, "microseconds": 761010}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "j1"}} {"timestamp": {"seconds": 1599725553, "microseconds": 761175}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "j1"}} {"return": {}} {"timestamp": {"seconds": 1599725630, "microseconds": 313251}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "j1"}} {"timestamp": {"seconds": 1599725630, "microseconds": 313352}, "event": "BLOCK_JOB_READY", "data": {"device": "j1", "len": 21475033088, "offset": 21475033088, "speed": 0, "type": "mirror"}} 5.Set migration capabilities in both src and dst. {"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"events","state":true},{"capability":"dirty-bitmaps","state":true}]}} 6.Add bitmap to src image and write file in src guest and check bitmap info { "execute": "block-dirty-bitmap-add", "arguments": {"node": "drive_image1", "name": "bitmap0"}} (src guest)#dd if=/dev/urandom of=test bs=1M count=1000 {"execute":"query-block"} {"return": [{"io-status": "ok", "device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/mnt/rhel830-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 6699487232, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "compression-type": "zlib", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 0}, {"recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 0}], "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/mnt/rhel830-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/image1/virtio-backend", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 0}, {"recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 0}], "type": "unknown"}]} 7.In dst, set bitmap mapping {"execute":"migrate-set-parameters","arguments":{"block-bitmap-mapping":[{"node-name":"drive_image1","alias":"drive_img1","bitmaps":[{"name":"bitmap0","alias":"bitmapa"}]}]}} 8.Set pre-switchover in src and do migration from src to dst {"execute":"migrate-continue","arguments":{"state":"pre-switchover"}} {"execute": "migrate","arguments":{"uri": "tcp:10.66.144.75:5000"}} {"timestamp": {"seconds": 1599726335, "microseconds": 403995}, "event": "MIGRATION", "data": {"status": "setup"}} {"return": {}} {"timestamp": {"seconds": 1599726335, "microseconds": 413324}, "event": "MIGRATION_PASS", "data": {"pass": 1}} {"timestamp": {"seconds": 1599726335, "microseconds": 413582}, "event": "MIGRATION", "data": {"status": "active"}} {"timestamp": {"seconds": 1599726405, "microseconds": 878064}, "event": "MIGRATION_PASS", "data": {"pass": 2}} {"timestamp": {"seconds": 1599726407, "microseconds": 180523}, "event": "MIGRATION_PASS", "data": {"pass": 3}} {"timestamp": {"seconds": 1599726407, "microseconds": 480314}, "event": "MIGRATION_PASS", "data": {"pass": 4}} {"timestamp": {"seconds": 1599726407, "microseconds": 486676}, "event": "STOP"} {"timestamp": {"seconds": 1599726407, "microseconds": 488527}, "event": "MIGRATION", "data": {"status": "pre-switchover"}} 9.Migrate continue {"execute":"migrate-continue","arguments":{"state":"pre-switchover"}} {"return": {}} {"timestamp": {"seconds": 1599726428, "microseconds": 819726}, "event": "MIGRATION", "data": {"status": "device"}} {"timestamp": {"seconds": 1599726428, "microseconds": 820726}, "event": "MIGRATION_PASS", "data": {"pass": 5}} {"timestamp": {"seconds": 1599726428, "microseconds": 830600}, "event": "MIGRATION", "data": {"status": "completed"}} 10.Check status in dst (qemu)info status VM status: running 11.Check bitmap status in dst {"execute":"query-block"} {"return": [{"io-status": "ok", "device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/mirror.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 6701453312, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "compression-type": "zlib", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_img1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "dirty-bitmaps": [{"name": "bitmapa", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 319488000}], "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/home/mirror.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/img1/virtio-backend", "dirty-bitmaps": [{"name": "bitmapa", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 319488000}], "type": "unknown"}]}
The "block-bitmap-mapping" works ok except an coredump issue, has tracked it by a new bug: bz1877724, so set this bug to "Verified".
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5137