Bug 1691048

Summary: Add qemu-img info support for querying bitmaps offline
Product: Red Hat Enterprise Linux 7 Reporter: John Snow <jsnow>
Component: qemu-kvm-rhevAssignee: John Snow <jsnow>
Status: CLOSED ERRATA QA Contact: aihua liang <aliang>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.7CC: eblake, jinzhao, juzhang, ngu, qzhang, virt-maint
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.12.0-27.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-08-22 09:20:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description John Snow 2019-03-20 17:43:41 UTC
This BZ is to track the ability for qemu-img to see persistent dirty bitmaps stored in qcow2 files ***while the VM is offline***.

This functionality is expected to be useful for debugging in the libvirt and RHV layers, as well as providing functionality to those management tools while the VM is not running.

Comment 3 Miroslav Rezanina 2019-04-25 03:52:57 UTC
Fix included in qemu-kvm-rhev-2.12.0-27.el7

Comment 4 aihua liang 2019-04-25 07:01:35 UTC
Verified it in qemu-kvm-rhev-2.12.0-27.el7, and flag info of bitmap can't be displayed correctly.

1. Add persistent bitmap, shutdown vm, check image info offline.
#qemu-img info rhel77-64-virtio-scsi.qcow2
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
                [0]: auto       ---> bit 0 has value "in_use", and bit 1 has value "auto"
            name: bitmap10
            granularity: 65536

2.Re-start vm, check image info online
#qemu-img info rhel77-64-virtio-scsi.qcow2 -U
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
                [0]: in-use
                [1]: auto        ---> Can display correctly when check image info online.
            name: bitmap10
            granularity: 65536

Reference: qemu-4.0.0-rc4/docs/interop/qcow2.txt
  Structure of a bitmap directory entry:

    Byte 0 -  7:    bitmap_table_offset
                    Offset into the image file at which the bitmap table
                    (described below) for the bitmap starts. Must be aligned to
                    a cluster boundary.

         8 - 11:    bitmap_table_size
                    Number of entries in the bitmap table of the bitmap.

        12 - 15:    flags
                    Bit
                      0: in_use
                         The bitmap was not saved correctly and may be
                         inconsistent. Although the bitmap metadata is still
                         well-formed from a qcow2 perspective, the metadata
                         (such as the auto flag or bitmap size) or data
                         contents may be outdated.

                      1: auto
                         The bitmap must reflect all changes of the virtual
                         disk by any application that would write to this qcow2
                         file (including writes, snapshot switching, etc.). The
                         type of this bitmap must be 'dirty tracking bitmap'.
                      2: extra_data_compatible
                         This flags is meaningful when the extra data is
                         unknown to the software (currently any extra data is
                         unknown to Qemu).
                         If it is set, the bitmap may be used as expected, extra
                         data must be left as is.
                         If it is not set, the bitmap must not be used, but
                         both it and its extra data be left as is.

                    Bits 3 - 31 are reserved and must be 0.

Hi, John

  Please help to check, thanks.

Comment 6 John Snow 2019-04-26 20:11:25 UTC
Sorry, that output is confusing. The number it prints is not meaningful... (ouch)

It's just an ordered list of the flags found to be set ... it doesn't actually correlate to the bit numbers. So, sadly, "working as intended" but I agree that it's confusing.

Better would be just:

Flags:
  +in_use
  +auto

(etc) without the numbers. It's a symptom simply of the "info" structure returning a list of flags, and the info printer prints all lists as numbered.

Comment 7 aihua liang 2019-04-28 02:19:09 UTC
(In reply to John Snow from comment #6)
> Sorry, that output is confusing. The number it prints is not meaningful...
> (ouch)
> 
> It's just an ordered list of the flags found to be set ... it doesn't
> actually correlate to the bit numbers. So, sadly, "working as intended" but
> I agree that it's confusing.
> 
> Better would be just:
> 
> Flags:
>   +in_use
>   +auto
> 
> (etc) without the numbers. It's a symptom simply of the "info" structure
> returning a list of flags, and the info printer prints all lists as numbered.

Got it. 

  Flags:
     +in_use
     +auto

Maybe better, i think.

Comment 8 aihua liang 2019-04-28 02:21:23 UTC
According to #c4 and #c6, the bug has been resolved, set bug's status to "Verified".

And if there's an optimization for flag display in later version, it'll be better.

Comment 9 Eric Blake 2019-04-28 02:28:07 UTC
(In reply to John Snow from comment #6)
> Sorry, that output is confusing. The number it prints is not meaningful...
> (ouch)
> 
> It's just an ordered list of the flags found to be set ... it doesn't
> actually correlate to the bit numbers. So, sadly, "working as intended" but
> I agree that it's confusing.

Confusing in this context, but generic and consistent (it is the same code used to print any QAPI list out to human-readable form)

> 
> Better would be just:
> 
> Flags:
>   +in_use
>   +auto

Except that figuring out how to wire up code to treat this output differently from the generic output will probably be a lot of difficult work, and be counter to having consistent output for all QAPI arrays.

> 
> (etc) without the numbers. It's a symptom simply of the "info" structure
> returning a list of flags, and the info printer prints all lists as numbered.

Comment 10 John Snow 2019-04-29 17:27:08 UTC
Unfortunately, Eric is right that due to the existing code infrastructure it's probably a lot of effort to change this, so it's not likely to change soon. I will try to document this quirk.

Thank you for your work in verifying these BZs so far.

Comment 12 errata-xmlrpc 2019-08-22 09:20:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:2553