Bug 1756413 - backport support for transactionable block-dirty-bitmap-remove for incremental backup support
Summary: backport support for transactionable block-dirty-bitmap-remove for incrementa...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.1
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: rc
: ---
Assignee: John Snow
QA Contact: aihua liang
URL:
Whiteboard:
: 1662370 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-09-27 15:03 UTC by Peter Krempa
Modified: 2020-02-04 18:29 UTC (History)
5 users (show)

Fixed In Version: qemu-kvm-4.1.0-14.module+el8.1.1+4632+a8269660
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-02-04 18:28:50 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:0404 0 None None None 2020-02-04 18:29:46 UTC

Description Peter Krempa 2019-09-27 15:03:15 UTC
Description of problem:
libvirt intends to use block-dirty-bitmap-remove via the 'transaction' command to simplify the error paths in the implementation of incremental backups

block-dirty-bitmap-remove is supported in transaction only starting qemu 4.2

qemu commit c4e4b0fa598

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 4 aihua liang 2019-11-13 06:38:55 UTC
Test on qemu-kvm-4.1.0-14.module+el8.1.1+4632+a8269660, block-dirty-bitmap-remove in transaction mode can work normally, set bug's status to "Verified".

Test Steps:
  1.Start guest with qemu cmds:
   /usr/local/bin/qemu-system-x86_64 \
    -name 'avocado-vt-vm1' \
    -machine q35  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x1  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190602-221944-MrlxVzia,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190602-221944-MrlxVzia,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idn20piu  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190602-221944-MrlxVzia,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20190602-221944-MrlxVzia,path=/var/tmp/seabios-20190602-221944-MrlxVzia,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20190602-221944-MrlxVzia,iobase=0x402 \
    -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
    -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \
    -object iothread,id=iothread0 \
    -object iothread,id=iothread1 \
    -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -drive driver=qcow2,id=drive_image1,file=/home/kvm_autotest_root/images/rhel810-64-virtio-scsi.qcow2,if=none,cache=none \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pcie.0,iothread=iothread0,bus=pcie.0-root-port-3,addr=0x0 \
    -device pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
    -blockdev driver=file,filename=/mnt/data.qcow2,node-name=file_node \
    -blockdev driver=qcow2,file=file_node,node-name=drive_data1 \
    -device virtio-blk-pci,id=data1,drive=drive_data1,bus=pcie.0-root-port-5,addr=0x0,iothread=iothread0 \
    -device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
    -drive format=qcow2,file=/mnt/data2.qcow2,id=drive_data2,cache=none,if=none \
    -device virtio-blk-pci,id=data2,drive=drive_data2,bus=pcie.0-root-port-6,addr=0x0 \
    -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
    -device virtio-net-pci,mac=9a:33:34:35:36:37,id=idj01pFr,vectors=4,netdev=idMgbx8B,bus=pcie.0-root-port-4,addr=0x0  \
    -netdev tap,id=idMgbx8B,vhost=on \
    -m 4096  \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2  \
    -cpu 'Skylake-Client',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \
    -qmp tcp:0:3000,server,nowait \

 2. Add persistent bitmaps to disks
    { "execute": "transaction", "arguments": { "actions": [ {"type":"block-dirty-bitmap-add", "data": { "node": "drive_image1", "name": "bitmap0","persistent":true}},{"type":"block-dirty-bitmap-add", "data": { "node": "drive_data1", "name": "bitmap0","persistent":true}},{"type":"block-dirty-bitmap-add", "data": { "node": "drive_data2", "name": "bitmap0","persistent":true}}]}}
{"return": {}}

 3. Query block info
    {"execute":"query-block"}
{"return": [{"io-status": "ok", "device": "drive_image1", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel810-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4904472576, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "#block104", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel810-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/image1/virtio-backend", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": true, "busy": false, "status": "active", "granularity": 65536, "count": 131072}], "type": "unknown"}, {"io-status": "ok", "device": "drive_data2", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 2147483648, "filename": "/home/data2.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 200704, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "#block354", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/home/data2.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/data2/virtio-backend", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": true, "busy": false, "status": "active", "granularity": 65536, "count": 0}], "type": "unknown"}, {"io-status": "ok", "device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 2147483648, "filename": "/home/data.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 200704, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_data1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/data.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/data1/virtio-backend", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": true, "busy": false, "status": "active", "granularity": 65536, "count": 0}], "type": "unknown"}]}

  4. Shutdown vm
     (qemu)system_powerdown

  5. Start vm, check bitmap info:
     {"execute":"query-block"}
{"return": [{"io-status": "ok", "device": "drive_image1", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel810-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4904595456, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "bitmaps": [{"flags": ["in-use", "auto"], "name": "bitmap0", "granularity": 65536}], "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "#block159", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel810-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/image1/virtio-backend", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": true, "busy": false, "status": "active", "granularity": 65536, "count": 35586048}], "type": "unknown"}, {"io-status": "ok", "device": "drive_data2", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 2147483648, "filename": "/home/data2.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 208896, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "bitmaps": [{"flags": ["in-use", "auto"], "name": "bitmap0", "granularity": 65536}], "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "#block385", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/home/data2.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/data2/virtio-backend", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": true, "busy": false, "status": "active", "granularity": 65536, "count": 0}], "type": "unknown"}, {"io-status": "ok", "device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 2147483648, "filename": "/home/data.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 270336, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "bitmaps": [{"flags": ["in-use", "auto"], "name": "bitmap0", "granularity": 65536}], "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_data1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/data.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/data1/virtio-backend", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": true, "busy": false, "status": "active", "granularity": 65536, "count": 0}], "type": "unknown"}]}

  6. Remove bitmaps from disks
    { "execute": "transaction", "arguments": { "actions": [ {"type":"block-dirty-bitmap-remove", "data": { "node": "drive_image1", "name": "bitmap0" }},{"type":"block-dirty-bitmap-remove", "data": { "node": "drive_data1", "name": "bitmap0" }},{"type":"block-dirty-bitmap-remove", "data": { "node": "drive_data2", "name": "bitmap0" }}]}}
{"return": {}}

  7. Check bitmap info:
    {"execute":"query-block"}
{"return": [{"io-status": "ok", "device": "drive_image1", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel810-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4904529920, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "#block159", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel810-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/image1/virtio-backend", "type": "unknown"}, {"io-status": "ok", "device": "drive_data2", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 2147483648, "filename": "/home/data2.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 208896, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "#block385", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": true, "writeback": true}, "file": "/home/data2.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/data2/virtio-backend", "type": "unknown"}, {"io-status": "ok", "device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 2147483648, "filename": "/home/data.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 270336, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_data1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/data.qcow2", "encryption_key_missing": false}, "qdev": "/machine/peripheral/data1/virtio-backend", "type": "unknown"}]}


Test Result:
  After step7, all bitmaps are removed under transaction mode.

Comment 5 aihua liang 2019-11-13 06:50:45 UTC
*** Bug 1662370 has been marked as a duplicate of this bug. ***

Comment 9 errata-xmlrpc 2020-02-04 18:28:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0404


Note You need to log in before you can comment on or make changes to this bug.