Bug 1441460

Summary: 'query-block' dirty bitmap count is shown in sectors but documented in bytes
Product: Red Hat Enterprise Linux 7 Reporter: Eric Blake <eblake>
Component: qemu-kvm-rhevAssignee: Eric Blake <eblake>
Status: CLOSED ERRATA QA Contact: Qianqian Zhu <qizhu>
Severity: low Docs Contact:
Priority: low    
Version: 7.4CC: chayang, coli, eblake, juzhang, michen, mrezanin, rbalakri, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: All   
Fixed In Version: qemu-kvm-rhev-2.10.0-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-11 00:16:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Eric Blake 2017-04-12 02:35:08 UTC
Description of problem:
In its introduction in qemu 1.3, BlockDirtyInfo reported 'count' in bytes.  But in 2.0 (commits e4654d2 and 21b5683), we accidentally broke it to start reporting 'count' in sectors.  I don't know if any higher level software cares (for example, libvirt does NOT scrape 'query-block' for the 'dirty-bitmaps' array containing BlockDirtyInfo), but it's a simple fix if we want it.

Version-Release number of selected component (if applicable):

How reproducible:
found by code inspection, but presumably 100%

Steps to Reproduce:
1. create a block job with a dirty bitmap, then cause the bitmap to get dirtier [at this point, I haven't experimented with a command line and subsequent guest step that would reliably repeat this, but am going off the code]
2. scrape the output of 'query-block' looking for 'dirty-bitmaps' then 'count', to see if the number resembles bytes or sectors

Actual results:

Expected results:

Additional info:
since libvirt doesn't scrape this information (yet), RHEL might not care, so I'm okay if this bug gets closed WONTFIX.

Comment 2 Eric Blake 2017-05-10 18:23:08 UTC
Proposed upstream patch

Comment 3 Eric Blake 2017-09-20 18:55:54 UTC
Moving to post; will be picked up by rebase to upstream 2.10 which includes

commit 6c98c57af3f4fab85bdf5f01616c91322bd4312a
Author: Eric Blake <eblake@redhat.com>
Date:   Fri Jul 21 13:32:42 2017 -0500

    dirty-bitmap: Report BlockDirtyInfo.count in bytes, as documented
    We've been documenting the value in bytes since its introduction
    in commit b9a9b3a4 (v1.3), where it was actually reported in bytes.
    Commit e4654d2 (v2.0) then removed things from block/qapi.c, in
    preparation for a rewrite to a list of dirty sectors in the next
    commit 21b5683 in block.c, but the new code mistakenly started
    reporting in sectors.
    Fixes: https://bugzilla.redhat.com/1441460
    CC: qemu-stable@nongnu.org
    Signed-off-by: Eric Blake <eblake@redhat.com>
    Reviewed-by: John Snow <jsnow@redhat.com>
    Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
    Signed-off-by: Kevin Wolf <kwolf@redhat.com>

Comment 5 Qianqian Zhu 2017-12-07 09:49:37 UTC
Tested with below steps for qemu 2.9 and latest qemu version:

1. Launch guest with a image installed os:
/usr/libexec/qemu-kvm  \
-cpu host \
-m 1024 \
-S \
-drive file=/home/win2012-64r2-virtio-scsi.raw,if=none,cache=writeback,id=drive0,format=raw \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive0,id=virtio-disk0 \
-qmp  tcp:0:5555,server,nowait \
-monitor stdio

2. Add dirty bitmap and perform live backup:
{ "execute": "transaction", "arguments": { "actions": [ {"type": "block-dirty-bitmap-add", "data": {"node": "drive0", "name": "bitmap0"} }, {"type": "drive-backup", "data": {"device": "drive0", "target": "full_backup.img", "sync": "full", "format": "qcow2"} } ] } }

3. Write to image and "query-block" to check the dirty-bitmaps count:
(qemu) qemu-io drive0 "write -P 0x9a 0 512"

(qemu) qemu-io drive0 "write -P 0x55 8M 352k"

(qemu) qemu-io drive0 "write -P 0x78 15872k 1M"


"dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity": 65536, "count": 128}]
"dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity": 65536, "count": 896}]
"dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity": 65536, "count": 2944}]

"dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity": 65536, "count": 65536}]
"dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity": 65536, "count": 458752}]
"dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity": 65536, "count": 1507328}]

Looks like it meets the expectation.
Eric, would you help double confirm? Thanks.

Comment 6 Eric Blake 2017-12-07 21:21:14 UTC
(In reply to Qianqian Zhu from comment #5)

> qemu-kvm-rhev-2.9.0-14.el7.x86_64:
> "dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity":
> 65536, "count": 128}]
> "dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity":
> 65536, "count": 896}]
> "dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity":
> 65536, "count": 2944}]
> qemu-kvm-rhev-2.10.0-11.el7.x86_64:
> "dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity":
> 65536, "count": 65536}]
> "dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity":
> 65536, "count": 458752}]
> "dirty-bitmaps": [{"name": "bitmap0", "status": "active", "granularity":
> 65536, "count": 1507328}]

Yep, the larger 'count' in the fixed version is what you want to see.

> Looks like it meets the expectation.

Comment 7 Qianqian Zhu 2017-12-08 02:04:21 UTC
Moving to VERIFIED per comment 5 and comment 6.

Comment 9 errata-xmlrpc 2018-04-11 00:16:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.