1666884 – persistent bitmaps prevent qcow2 image resize

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1666884 - persistent bitmaps prevent qcow2 image resize

Summary: persistent bitmaps prevent qcow2 image resize

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	qemu-kvm-rhev
Sub Component:
Version:	7.7
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	John Snow
QA Contact:	aihua liang
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	1683037 1717161
TreeView+	depends on / blocked

Reported:	2019-01-16 20:49 UTC by Eric Blake
Modified:	2019-08-22 09:20 UTC (History)
CC List:	8 users (show)
Fixed In Version:	qemu-kvm-rhev-2.12.0-27.el7
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Clones:	1683037 1717161 (view as bug list)
Environment:
Last Closed:	2019-08-22 09:19:59 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHSA-2019:2553	0	None	None	None	2019-08-22 09:20:49 UTC

Description Eric Blake 2019-01-16 20:49:10 UTC

Description of problem:
The existence of a persistent qcow2 bitmap prevents 'qemu-img resize' and the matching QMP commands from running.  Ideally, we should allow at least growth (but do we treat the expanded tail as dirty because it is new content not previously visible to guest, or as clean since the guest hasn't touched it yet?), but even nicer would be also worrying about interactions with shrink (prevent shrink without a force request if the tail contains any dirty clusters, or if the tail is clean go ahead and do the shrink).

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. add a persistent bitmap to a qcow2 image (no easy qemu-img command for this yet; must use qemu QMP commands)
2. try to resize the image
3.

Actual results:
qemu gives error

Expected results:
resize should be supported

Additional info:

Comment 2 aihua liang 2019-02-26 06:57:12 UTC

For incremental_live_backup, test on qemu-kvm-rhev-2.12.0-23.el7.x86_64, only in this 
testing scenarios, we can hit this issue.

 1.Start guest with a qcow2 image.
    -object iothread,id=iothread0 \
    -blockdev driver=file,node-name=file_base,filename=/home/kvm_autotest_root/images/rhel77-64-virtio.qcow2,auto-read-only=on \
    -blockdev driver=qcow2,file=file_base,node-name=drive_image1,auto-read-only=on \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,iothread=iothread0 \
 
 2.Create a persistent bitmap.
     { "execute": "block-dirty-bitmap-add", "arguments": {"node": "drive_image1", "name": "bitmap0","persistent":true}}

 3.Shutdown vm.
   (qemu)quit
 
 4.Re-start it with the qcow2 image.
    -object iothread,id=iothread0 \
    -blockdev driver=file,node-name=file_base,filename=/home/kvm_autotest_root/images/rhel77-64-virtio.qcow2,auto-read-only=on \
    -blockdev driver=qcow2,file=file_base,node-name=drive_image1,auto-read-only=on \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,iothread=iothread0 \

 5.Do block resize
    { "execute": "block_resize", "arguments": { "node-name":"drive_image1", "size":32212254720}}

Test Result:
 Block_resize in step5 failed with error info:
   {"error": {"class": "GenericError", "desc": "Can't resize an image which has bitmaps"}}


 When we do block_resize after step2, don't hit this issue.

Comment 3 John Snow 2019-03-06 21:20:57 UTC

Preliminary fixes for this have been posted upstream, setting to assigned.

Ref: https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg01319.html

Comment 4 Tingting Mao 2019-03-26 03:06:41 UTC

Reproduce this bug offline as below.

Tested with:
qemu-kvm-rhev-2.12.0-25.el7
kernel-3.10.0-1014.el7


Steps:
1. Boot a guest with a data disk of qcow2 format.
# /usr/libexec/qemu-kvm \
        ......
        -device virtio-blk-pci,id=virtio_blk_pci1,drive=my_file1,bus=pcie.0,addr=06 \
        -blockdev driver=qcow2,cache.direct=off,cache.no-flush=on,node-name=my_file1,file.driver=file,file.filename=data.qcow2 \
        ......
2. Check blocks info with HMP
(qemu) info block
my_file: sn.qcow2 (qcow2)
    Attached to:      /machine/peripheral/virtio_blk_pci0/virtio-backend
    Cache mode:       writethrough, ignore flushes
    Backing file:     json:{"driver": "raw", "file": {"pool": "rbd", "image": "new_base.img", "driver": "rbd"}} (chain depth: 1)

my_file1: data.qcow2 (qcow2)
    Attached to:      /machine/peripheral/virtio_blk_pci1/virtio-backend
    Cache mode:       writethrough, ignore flushes
(qemu) 

3. Create a persistent bitmap by QMP
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 12, "major": 2}, "package": "qemu-kvm-rhev-2.12.0-25.el7"}, "capabilities": []}}

{"execute": "qmp_capabilities"}
{"return": {}}

{ "execute": "block-dirty-bitmap-add", "arguments": {"node": "my_file1", "name": "bitmap0","persistent":true}}
{"return": {}}

4. Shutdown the guest

5. Resize the data disk
# qemu-img resize data.qcow2 +1G
qemu-img: Can't resize an image which has bitmaps

Comment 6 Miroslav Rezanina 2019-04-25 03:51:53 UTC

Fix included in qemu-kvm-rhev-2.12.0-27.el7

Comment 7 aihua liang 2019-04-25 08:52:34 UTC

Verified in qemu-kvm-rhev-2.12.0-27.el7, it has been fixed. Will set bug's status to "Verified" after "ON_QA".

Test steps:
 1.Start guest with qemu cmds:
    /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1' \
    -machine pc  \
    -nodefaults \
    -device VGA,bus=pci.0,addr=0x2  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190412-011839-sZJY2UtD,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190412-011839-sZJY2UtD,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idbSDNv4  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190412-011839-sZJY2UtD,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20190412-011839-sZJY2UtD,path=/var/tmp/seabios-20190412-011839-sZJY2UtD,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20190412-011839-sZJY2UtD,iobase=0x402 \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 \
    -object iothread,id=iothread0 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x4,iothread=iothread0 \
    -blockdev driver=file,node-name=file_node,filename=/home/kvm_autotest_root/images/rhel77-64-virtio-scsi.qcow2 \
    -blockdev driver=qcow2,node-name=drive_image1,file=file_node \
    -device scsi-hd,id=image1,drive=drive_image1 \
    -device virtio-net-pci,mac=9a:dc:dd:de:df:e0,id=idb7yKIW,vectors=4,netdev=idpOQ1y9,bus=pci.0,addr=0x5  \
    -netdev tap,id=idpOQ1y9,vhost=on \
    -m 30720  \
    -smp 16,maxcpus=16,cores=8,threads=1,sockets=2  \
    -cpu 'IvyBridge',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot menu=off,strict=off,order=cdn,once=c \
    -enable-kvm \
    -monitor stdio \
    -qmp tcp:0:3000,server,nowait \

  2.Add a persistent bitmap
    { "execute": "block-dirty-bitmap-add", "arguments": {"node": "drive_image1", "name": "bitmap1","persistent":true,"disabled":true,"granularity":4096}}

  3.Resize disk size to 30G
    { "execute": "block_resize", "arguments": { "node-name":"drive_image1", "size":32212254720}}
  
  4.Check image info.
    # qemu-img info rhel77-64-virtio-scsi.qcow2 -U
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 30G (32212254720 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
  
  5.Shutdown vm, resize disk size to 31G
     qemu-img resize /home/kvm_autotest_root/images/rhel77-64-virtio-scsi.qcow2 +1G
 
  6.Check image info:
     # qemu-img info rhel77-64-virtio-scsi.qcow2
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 31G (33285996544 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
            name: bitmap1
            granularity: 4096
    refcount bits: 16
    corrupt: false

  7.Re-start vm, shrink its size to 30G
     { "execute": "block_resize", "arguments": { "node-name":"drive_image1", "size":32212254720}}
  
  8.Check image info:
     qemu-img info rhel77-64-virtio-scsi.qcow2 -U
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 30G (32212254720 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
                [0]: in-use
            name: bitmap1
            granularity: 4096
    refcount bits: 16
    corrupt: false

  9.Shutdown vm, shrink its size to 29G.
    qemu-img resize --shrink /home/kvm_autotest_root/images/rhel77-64-virtio-scsi.qcow2 -1G

  10.Check image info:
     qemu-img info rhel77-64-virtio-scsi.qcow2
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 29G (31138512896 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
            name: bitmap1
            granularity: 4096
    refcount bits: 16
    corrupt: false

 11.Re-start vm, then remove bitmap and resize image size to 30G:
     { 'execute': 'block-dirty-bitmap-remove', 'arguments': {'node':'drive_image1', 'name':'bitmap1'}}
     { "execute": "block_resize", "arguments": { "node-name":"drive_image1", "size":32212254720}}

 12.Check image info:
     qemu-img info rhel77-64-virtio-scsi.qcow2 -U
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 30G (32212254720 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Comment 9 aihua liang 2019-04-28 02:29:37 UTC

As comment 7, set bug's status to "Verified".

Comment 11 errata-xmlrpc 2019-08-22 09:19:59 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:2553

Note You need to log in before you can comment on or make changes to this bug.