Bug 1666884

Summary: persistent bitmaps prevent qcow2 image resize
Product: Red Hat Enterprise Linux 7 Reporter: Eric Blake <eblake>
Component: qemu-kvm-rhevAssignee: John Snow <jsnow>
Status: CLOSED ERRATA QA Contact: aihua liang <aliang>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.7CC: aliang, chayang, coli, juzhang, ngu, qzhang, timao, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.12.0-27.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1683037 1717161 (view as bug list) Environment:
Last Closed: 2019-08-22 09:19:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1683037, 1717161    

Description Eric Blake 2019-01-16 20:49:10 UTC
Description of problem:
The existence of a persistent qcow2 bitmap prevents 'qemu-img resize' and the matching QMP commands from running.  Ideally, we should allow at least growth (but do we treat the expanded tail as dirty because it is new content not previously visible to guest, or as clean since the guest hasn't touched it yet?), but even nicer would be also worrying about interactions with shrink (prevent shrink without a force request if the tail contains any dirty clusters, or if the tail is clean go ahead and do the shrink).

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. add a persistent bitmap to a qcow2 image (no easy qemu-img command for this yet; must use qemu QMP commands)
2. try to resize the image
3.

Actual results:
qemu gives error

Expected results:
resize should be supported

Additional info:

Comment 2 aihua liang 2019-02-26 06:57:12 UTC
For incremental_live_backup, test on qemu-kvm-rhev-2.12.0-23.el7.x86_64, only in this 
testing scenarios, we can hit this issue.

 1.Start guest with a qcow2 image.
    -object iothread,id=iothread0 \
    -blockdev driver=file,node-name=file_base,filename=/home/kvm_autotest_root/images/rhel77-64-virtio.qcow2,auto-read-only=on \
    -blockdev driver=qcow2,file=file_base,node-name=drive_image1,auto-read-only=on \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,iothread=iothread0 \
 
 2.Create a persistent bitmap.
     { "execute": "block-dirty-bitmap-add", "arguments": {"node": "drive_image1", "name": "bitmap0","persistent":true}}

 3.Shutdown vm.
   (qemu)quit
 
 4.Re-start it with the qcow2 image.
    -object iothread,id=iothread0 \
    -blockdev driver=file,node-name=file_base,filename=/home/kvm_autotest_root/images/rhel77-64-virtio.qcow2,auto-read-only=on \
    -blockdev driver=qcow2,file=file_base,node-name=drive_image1,auto-read-only=on \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,iothread=iothread0 \

 5.Do block resize
    { "execute": "block_resize", "arguments": { "node-name":"drive_image1", "size":32212254720}}

Test Result:
 Block_resize in step5 failed with error info:
   {"error": {"class": "GenericError", "desc": "Can't resize an image which has bitmaps"}}


 When we do block_resize after step2, don't hit this issue.

Comment 3 John Snow 2019-03-06 21:20:57 UTC
Preliminary fixes for this have been posted upstream, setting to assigned.

Ref: https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg01319.html

Comment 4 Tingting Mao 2019-03-26 03:06:41 UTC
Reproduce this bug offline as below.

Tested with:
qemu-kvm-rhev-2.12.0-25.el7
kernel-3.10.0-1014.el7


Steps:
1. Boot a guest with a data disk of qcow2 format.
# /usr/libexec/qemu-kvm \
        ......
        -device virtio-blk-pci,id=virtio_blk_pci1,drive=my_file1,bus=pcie.0,addr=06 \
        -blockdev driver=qcow2,cache.direct=off,cache.no-flush=on,node-name=my_file1,file.driver=file,file.filename=data.qcow2 \
        ......
2. Check blocks info with HMP
(qemu) info block
my_file: sn.qcow2 (qcow2)
    Attached to:      /machine/peripheral/virtio_blk_pci0/virtio-backend
    Cache mode:       writethrough, ignore flushes
    Backing file:     json:{"driver": "raw", "file": {"pool": "rbd", "image": "new_base.img", "driver": "rbd"}} (chain depth: 1)

my_file1: data.qcow2 (qcow2)
    Attached to:      /machine/peripheral/virtio_blk_pci1/virtio-backend
    Cache mode:       writethrough, ignore flushes
(qemu) 

3. Create a persistent bitmap by QMP
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 12, "major": 2}, "package": "qemu-kvm-rhev-2.12.0-25.el7"}, "capabilities": []}}

{"execute": "qmp_capabilities"}
{"return": {}}

{ "execute": "block-dirty-bitmap-add", "arguments": {"node": "my_file1", "name": "bitmap0","persistent":true}}
{"return": {}}

4. Shutdown the guest

5. Resize the data disk
# qemu-img resize data.qcow2 +1G
qemu-img: Can't resize an image which has bitmaps

Comment 6 Miroslav Rezanina 2019-04-25 03:51:53 UTC
Fix included in qemu-kvm-rhev-2.12.0-27.el7

Comment 7 aihua liang 2019-04-25 08:52:34 UTC
Verified in qemu-kvm-rhev-2.12.0-27.el7, it has been fixed. Will set bug's status to "Verified" after "ON_QA".

Test steps:
 1.Start guest with qemu cmds:
    /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1' \
    -machine pc  \
    -nodefaults \
    -device VGA,bus=pci.0,addr=0x2  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190412-011839-sZJY2UtD,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190412-011839-sZJY2UtD,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idbSDNv4  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190412-011839-sZJY2UtD,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20190412-011839-sZJY2UtD,path=/var/tmp/seabios-20190412-011839-sZJY2UtD,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20190412-011839-sZJY2UtD,iobase=0x402 \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 \
    -object iothread,id=iothread0 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x4,iothread=iothread0 \
    -blockdev driver=file,node-name=file_node,filename=/home/kvm_autotest_root/images/rhel77-64-virtio-scsi.qcow2 \
    -blockdev driver=qcow2,node-name=drive_image1,file=file_node \
    -device scsi-hd,id=image1,drive=drive_image1 \
    -device virtio-net-pci,mac=9a:dc:dd:de:df:e0,id=idb7yKIW,vectors=4,netdev=idpOQ1y9,bus=pci.0,addr=0x5  \
    -netdev tap,id=idpOQ1y9,vhost=on \
    -m 30720  \
    -smp 16,maxcpus=16,cores=8,threads=1,sockets=2  \
    -cpu 'IvyBridge',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot menu=off,strict=off,order=cdn,once=c \
    -enable-kvm \
    -monitor stdio \
    -qmp tcp:0:3000,server,nowait \

  2.Add a persistent bitmap
    { "execute": "block-dirty-bitmap-add", "arguments": {"node": "drive_image1", "name": "bitmap1","persistent":true,"disabled":true,"granularity":4096}}

  3.Resize disk size to 30G
    { "execute": "block_resize", "arguments": { "node-name":"drive_image1", "size":32212254720}}
  
  4.Check image info.
    # qemu-img info rhel77-64-virtio-scsi.qcow2 -U
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 30G (32212254720 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
  
  5.Shutdown vm, resize disk size to 31G
     qemu-img resize /home/kvm_autotest_root/images/rhel77-64-virtio-scsi.qcow2 +1G
 
  6.Check image info:
     # qemu-img info rhel77-64-virtio-scsi.qcow2
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 31G (33285996544 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
            name: bitmap1
            granularity: 4096
    refcount bits: 16
    corrupt: false

  7.Re-start vm, shrink its size to 30G
     { "execute": "block_resize", "arguments": { "node-name":"drive_image1", "size":32212254720}}
  
  8.Check image info:
     qemu-img info rhel77-64-virtio-scsi.qcow2 -U
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 30G (32212254720 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
                [0]: in-use
            name: bitmap1
            granularity: 4096
    refcount bits: 16
    corrupt: false

  9.Shutdown vm, shrink its size to 29G.
    qemu-img resize --shrink /home/kvm_autotest_root/images/rhel77-64-virtio-scsi.qcow2 -1G

  10.Check image info:
     qemu-img info rhel77-64-virtio-scsi.qcow2
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 29G (31138512896 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
            name: bitmap1
            granularity: 4096
    refcount bits: 16
    corrupt: false

 11.Re-start vm, then remove bitmap and resize image size to 30G:
     { 'execute': 'block-dirty-bitmap-remove', 'arguments': {'node':'drive_image1', 'name':'bitmap1'}}
     { "execute": "block_resize", "arguments": { "node-name":"drive_image1", "size":32212254720}}

 12.Check image info:
     qemu-img info rhel77-64-virtio-scsi.qcow2 -U
image: rhel77-64-virtio-scsi.qcow2
file format: qcow2
virtual size: 30G (32212254720 bytes)
disk size: 4.3G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Comment 9 aihua liang 2019-04-28 02:29:37 UTC
As comment 7, set bug's status to "Verified".

Comment 11 errata-xmlrpc 2019-08-22 09:19:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:2553