RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1631877 - qemu dirty bitmap merge fails to update count
Summary: qemu dirty bitmap merge fails to update count
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.7
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: John Snow
QA Contact: aihua liang
URL:
Whiteboard:
Depends On: 1658343
Blocks: 1636224 1651787
TreeView+ depends on / blocked
 
Reported: 2018-09-21 19:12 UTC by Eric Blake
Modified: 2019-08-22 09:19 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-08-22 09:18:53 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2019:2553 0 None None None 2019-08-22 09:19:46 UTC

Description Eric Blake 2018-09-21 19:12:15 UTC 
Description of problem:
When dirty bitmap bits are set or cleared, they update a count of affected sectors visible to the user via QMP 'query-block'. But when dirty bitmaps are merged with x-block-dirty-bitmap-merge, this count is not updated, leaving the destination with the same count it had prior to the merge even when it was otherwise updated.

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.12.0-17.el7

How reproducible:
100%

Steps to Reproduce:
1. Attach bitmaps to a block device visible through 'query-bitmap', and perform various x-block-dirty-bitmap-merge and x-debug-block-dirty-bitmap-sha256 to track that contents were merged but count was not.
2.
3.

Actual results:
https://lists.gnu.org/archive/html/qemu-devel/2018-09/msg02550.html

Also, some places in the code short-circuit when count is zero. If you are trying to merge C into B, then B into A, but B started life empty, then bitmap A does NOT end up receiving the contents of C (because even though those contents were copied into B, the inaccurate size of 0 made the copy into A return early)

Expected results:
count should always be accurate, especially since libvirt wants to take query-block count and expose it to the end user as a way to estimate the resulting size of an incremental backup image.

Additional info:
Comment 1 Eric Blake 2018-09-21 19:12:58 UTC 
related to bug 1631052
Comment 3 John Snow 2018-10-09 20:34:35 UTC 
upstream, pending merge:
https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg00641.html
Comment 4 John Snow 2018-12-11 23:23:08 UTC 
This fix is included as part of the patches for BZ #1658343 now in POST.
Setting dependency and marking TestOnly.
Comment 6 aihua liang 2019-02-15 07:19:56 UTC 
Test on qemu-kvm-rhev-2.12.0-23.el7.x86_64, don't hit this issue.

Test steps:
  1. start guest with qemu cmds:
      /usr/libexec/qemu-kvm \
    -S \
    -name 'avocado-vt-vm1' \
    -machine pc  \
    -nodefaults \
    -device VGA,bus=pci.0,addr=0x2  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190123-032240-rOoB4cgD,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190123-032240-rOoB4cgD,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=id8Ec4Bn  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190123-032240-rOoB4cgD,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20190123-032240-rOoB4cgD,path=/var/tmp/seabios-20190123-032240-rOoB4cgD,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20190123-032240-rOoB4cgD,iobase=0x402 \
    -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 \
    -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 \
    -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 \
    -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x3 \
    -blockdev driver=file,node-name=file_base,filename=/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2,auto-read-only=on \
    -blockdev driver=qcow2,file=file_base,node-name=drive_image1,auto-read-only=on \
    -device scsi-hd,id=image1,drive=drive_image1 \
    -device virtio-net-pci,mac=9a:39:3a:3b:3c:3d,id=id1JNQsL,vectors=4,netdev=idVpZZ6A,bus=pci.0,addr=0x4  \
    -netdev tap,id=idVpZZ6A,vhost=on \
    -m 4096  \
    -smp 2,maxcpus=2,cores=1,threads=1,sockets=2  \
    -cpu 'Penryn',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \
    -qmp tcp:0:3000,server,nowait \

 2. Create bitmap0
    { "execute": "block-dirty-bitmap-add", "arguments": {"node": "drive_image1", "name":"bitmap0"}}

 3. Cont vm, wait for guest boot up, then check bitmap info
    (qemu)vm
    {"execute":"query-block"}
{"return": [{"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4498259968, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity": 65536, "count": 62193664}], "type": "unknown"}]}

  4. Disable bitmap0
     { "execute":"x-block-dirty-bitmap-disable", "arguments": { "node": "drive_image1", "name": "bitmap0" } }

  5. Add a disabled bitmap:bitmap1, check bitmap info:
     { "execute": "block-dirty-bitmap-add", "arguments": { "node": "drive_image1", "name": "bitmap1", "x-disabled": true } }
     {"execute":"query-block"}
{"return": [{"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4498259968, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap1", "status": "disabled", "granularity": 65536, "count": 0}, {"name": "bitmap0", "status": "disabled", "granularity": 65536, "count": 62259200}], "type": "unknown"}]}

  6. In guest, create a file a, then check bitmap info:
     (guest)# dd if=/dev/urandom of=/home/a bs=1M count=2 oflag=dsync
     {"execute":"query-block"}
{"return": [{"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4498259968, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap1", "status": "disabled", "granularity": 65536, "count": 0}, {"name": "bitmap0", "status": "disabled", "granularity": 65536, "count": 62259200}], "type": "unknown"}]}

  7. Add bitmap bitmap_tmp, check bitmap info:
     { "execute": "block-dirty-bitmap-add", "arguments": {"node": "drive_image1", "name":"bitmap_tmp"}}
     {"execute":"query-block"}
{"return": [{"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4498259968, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap_tmp", "status": "active", "granularity": 65536, "count": 0}, {"name": "bitmap1", "status": "disabled", "granularity": 65536, "count": 0}, {"name": "bitmap0", "status": "disabled", "granularity": 65536, "count": 62259200}], "type": "unknown"}]}

  8. Create file b in guest, then check bitmap info:
     (guest)# dd if=/dev/urandom of=/home/b bs=1M count=2 oflag=dsync
     {"execute":"query-block"}
{"return": [{"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4498259968, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap_tmp", "status": "active", "granularity": 65536, "count": 2293760}, {"name": "bitmap1", "status": "disabled", "granularity": 65536, "count": 0}, {"name": "bitmap0", "status": "disabled", "granularity": 65536, "count": 62259200}], "type": "unknown"}]}

   9. Merge bitmap1 to bitmap_tmp, then check bitmap info:
      {"execute": "x-block-dirty-bitmap-merge","arguments":{"node": "drive_image1", "src_name": "bitmap1", "dst_name":"bitmap_tmp" } }
{"return": {}}
{"execute":"query-block"}
{"return": [{"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4498259968, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap_tmp", "status": "active", "granularity": 65536, "count": 3604480}, {"name": "bitmap1", "status": "disabled", "granularity": 65536, "count": 0}, {"name": "bitmap0", "status": "disabled", "granularity": 65536, "count": 62259200}], "type": "unknown"}]}

   10. Disable bitmap_tmp, check bitmap info:
       { "execute":"x-block-dirty-bitmap-disable", "arguments": { "node": "drive_image1", "name": "bitmap_tmp"}}
{"return": {}}
{"execute":"query-block"}
{"return": [{"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4498259968, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap_tmp", "status": "disabled", "granularity": 65536, "count": 3801088}, {"name": "bitmap1", "status": "disabled", "granularity": 65536, "count": 0}, {"name": "bitmap0", "status": "disabled", "granularity": 65536, "count": 62259200}], "type": "unknown"}]}

   11. Merge bitmap0 to bitmap_tmp, then check bitmap info:
       {"execute": "x-block-dirty-bitmap-merge","arguments":{"node": "drive_image1", "src_name": "bitmap0", "dst_name":"bitmap_tmp" } }
{"return": {}}
{"execute":"query-block"}
{"return": [{"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "image": {"virtual-size": 21474836480, "filename": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "cluster-size": 65536, "format": "qcow2", "actual-size": 4498259968, "format-specific": {"type": "qcow2", "data": {"compat": "1.1", "lazy-refcounts": false, "refcount-bits": 16, "corrupt": false}}, "dirty-flag": false}, "iops_wr": 0, "ro": false, "node-name": "drive_image1", "backing_file_depth": 0, "drv": "qcow2", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/home/kvm_autotest_root/images/rhel80-64-virtio-scsi.qcow2", "encryption_key_missing": false}, "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap_tmp", "status": "disabled", "granularity": 65536, "count": 64487424}, {"name": "bitmap1", "status": "disabled", "granularity": 65536, "count": 0}, {"name": "bitmap0", "status": "disabled", "granularity": 65536, "count": 62259200}], "type": "unknown"}]}

 Test result:
  Count for active bitmap is updated for all operations.

 Will set bug's status to "Verified" after its status changed to "ON_QA".
Comment 7 aihua liang 2019-03-19 06:36:59 UTC 
Set bug's status to "Verified", for it has keywords "TestOnly", thanks.
Comment 9 errata-xmlrpc 2019-08-22 09:18:53 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:2553
Note You need to log in before you can comment on or make changes to this bug.