RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2032410 - RFE: Propagate subcluster allocation settings of QCOW2 images to new overlay images formated by libvirt
Summary: RFE: Propagate subcluster allocation settings of QCOW2 images to new overlay ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: libvirt
Version: 9.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Peter Krempa
QA Contact: Meina Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-14 13:02 UTC by Peter Krempa
Modified: 2022-05-17 13:06 UTC (History)
7 users (show)

Fixed In Version: libvirt-8.0.0-0rc1.1.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-17 12:45:52 UTC
Type: Feature Request
Target Upstream Version: 8.0.0
Embargoed:
pm-rhel: mirror+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github autotest tp-libvirt pull 4062 0 None Merged add case for blockcopy with extended_l2 on disk 2022-03-09 10:19:57 UTC
Github autotest tp-libvirt pull 4081 0 None Merged add case for encrypted disk wth extended l2 on status 2022-04-21 02:42:04 UTC
Red Hat Issue Tracker RHELPLAN-105745 0 None None None 2021-12-14 13:05:22 UTC
Red Hat Product Errata RHBA-2022:2390 0 None None None 2022-05-17 12:46:20 UTC

Description Peter Krempa 2021-12-14 13:02:21 UTC
Description of problem:
Subcluster allocation settings should be propagated into new overlay images created by libvirt for external snapshots and block copy.

Comment 3 Peter Krempa 2021-12-22 14:52:15 UTC
I've used the following steps to test the feature when I was developing it:

1) format qcow2 image with subclusters (extended_l2)
 # qemu-img create -f qcow2 -o extended_l2=on /tmp/subcluster.qcow2 10M
 Formatting '/tmp/subcluster.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=on compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16
 # qemu-img info /tmp/subcluster.qcow2 
 image: /tmp/subcluster.qcow2
 file format: qcow2
 virtual size: 10 MiB (10485760 bytes)
 disk size: 196 KiB
 cluster_size: 65536
 Format specific information:
     compat: 1.1
     compression type: zlib
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
     extended l2: true <<<< Here

2) start a VM with the formatted image as disk:
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/tmp/subcluster.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </disk>

3) create an external snapshot:
 # virsh snapshot-create-as --disk-only --no-metadata cd
 Domain snapshot 1640184344 created

4) do a block-copy
 # virsh blockcopy cd vda /tmp/copy.qcow2 --transient-job --pivot --wait
 
 Successfully pivoted

5) destroy the vm 

6) check whether images have extended l2 enabled

6a) the snapshot:
 # qemu-img info /tmp/subcluster.1640184344 
 image: /tmp/subcluster.1640184344
 file format: qcow2
 virtual size: 10 MiB (10485760 bytes)
 disk size: 196 KiB
 cluster_size: 65536
 backing file: /tmp/subcluster.qcow2
 backing file format: qcow2
 Format specific information:
     compat: 1.1
     compression type: zlib
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
     extended l2: true <<<<

6b) the copy target:
 # qemu-img info /tmp/copy.qcow2 
 image: /tmp/copy.qcow2
 file format: qcow2
 virtual size: 10 MiB (10485760 bytes)
 disk size: 260 KiB
 cluster_size: 65536
 Format specific information:
     compat: 1.1
     compression type: zlib
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
     extended l2: true <<<

Comment 4 Peter Krempa 2021-12-22 14:54:19 UTC
Added upstream by:

commit 4273b74b82003c729babe62792c8e71ab57b62ce
Author: Peter Krempa <pkrempa>
Date:   Thu Dec 16 18:35:34 2021 +0100

    qemuBlockStorageSourceCreateDetectSize: Propagate 'extended_l2' feature to new overlays
    
    In cases where the qcow2 image is using subclusters/extended_l2 entries
    we should propagate them to the new images which are based on such
    images.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

commit 3275be238c2896dd6a5ba8c66f2f047bba7bb1e5
Author: Peter Krempa <pkrempa>
Date:   Thu Dec 16 11:00:55 2021 +0100

    qemu: monitor: Extract whether qcow2 image uses extended L2 allocation data
    
    In order to be able to propagate image configuration to newly formatted
    images we need to be able to query it.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

commit df0e867447e7f50d4639b735af6dc6782d28084a
Author: Peter Krempa <pkrempa>
Date:   Thu Dec 16 12:46:51 2021 +0100

    qemuBlockStorageSourceCreateGetFormatPropsQcow2: Add support for 'extended-l2' feature
    
    Allow creating the qcow2 with the new subcluster format if required.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

v7.10.0-371-g4273b74b82

Comment 6 Meina Li 2021-12-28 08:50:42 UTC
Pre-verified Version:
libvirt-8.0.0-1.fc35.x86_64
qemu-kvm-6.1.0-13.fc35.x86_64

Verified Steps:
S1: Do blockcopy after creating external snapshot with extended_l2 in disk image
1. Create a disk image with extended_l2=on.
# qemu-img create -f qcow2 -o cluster_size=2M,extended_l2=on /var/lib/libvirt/images/subcluster.qcow2 500M
Formatting '/var/lib/libvirt/images/subcluster.qcow2', fmt=qcow2 cluster_size=2097152 extended_l2=on compression_type=zlib size=524288000 lazy_refcounts=off refcount_bits=16
# qemu-img info /var/lib/libvirt/images/subcluster.qcow2 
image: /var/lib/libvirt/images/subcluster.qcow2
file format: qcow2
virtual size: 500 MiB (524288000 bytes)
disk size: 6 MiB
cluster_size: 2097152
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: true
2. Start a guest with the created disk image.
# virsh start lmn
Domain 'lmn' started
# virsh dumpxml lmn |grep /disk -B8
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/subcluster.qcow2' index='1'/>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>
3. Create a snapshot and check the snapshot info.
# virsh snapshot-create-as lmn s1 --disk-only
Domain snapshot s1 created
# qemu-img info /var/lib/libvirt/images/subcluster.s1 -U
image: /var/lib/libvirt/images/subcluster.s1
file format: qcow2
virtual size: 500 MiB (524288000 bytes)
disk size: 6 MiB
cluster_size: 2097152
backing file: /var/lib/libvirt/images/subcluster.qcow2
backing file format: qcow2
qemu-img create --object secret,id=sec0,data=redhat -f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0 /var/lib/libvirt/images/luks/luks_1.img 1GFormat specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: true
4. Do blockcopy and check the copied info.
# virsh blockcopy lmn vdb /tmp/copy.qcow2 --wait --verbose --transient-job --pivot
Block Copy: [100 %]
Successfully pivoted
# virsh dumpxml lmn | grep /disk -B8
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/tmp/copy.qcow2' index='6'/>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>
# qemu-img info /tmp/copy.qcow2  -U
image: /tmp/copy.qcow2
file format: qcow2
virtual size: 500 MiB (524288000 bytes)
disk size: 184 MiB
cluster_size: 2097152
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: true
5. Write data to the disk in guest.
[in guest] # mkfs.ext4 /dev/vdb;mount /dev/vdb /mnt/; dd if=/dev/urandom of=/mnt/file bs=1M count=100

S2: Do blockcopy after creating external snapshot with extended_l2 in encrypted disk image
1. Prepare a luks secret.
# cat luks-secret.xml 
 <secret ephemeral='no' private='yes'>
         <description>LUKS Sample Secret</description>
         <uuid>f981dd17-143f-45bc-88e6-ed1fe20ce9da</uuid>
         <usage type='volume'>
            <volume>/var/lib/libvirt/images/luks/luks_1.img</volume>
         </usage>
      </secret>
# virsh secret-define luks-secret.xml 
Secret f981dd17-143f-45bc-88e6-ed1fe20ce9da created
# MYSECRET=`printf %s "redhat" | base64`
#virsh secret-set-value f981dd17-143f-45bc-88e6-ed1fe20ce9da $MYSECRET
Secret value set
2. Create a luks image with extended_l2=on.
# qemu-img create --object secret,id=sec0,data=redhat -f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0,cluster_size=2M,extended_l2=on /var/lib/libvirt/images/luks_1.img 1G
Formatting '/var/lib/libvirt/images/luks_1.img', fmt=qcow2 encrypt.format=luks encrypt.key-secret=sec0 cluster_size=2097152 extended_l2=on compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
# qemu-img info /var/lib/libvirt/images/luks_1.img 
image: /var/lib/libvirt/images/luks_1.img
file format: qcow2
virtual size: 1 GiB (1073741824 bytes)
disk size: 8 MiB
encrypted: yes
cluster_size: 2097152
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    encrypt:
        ......
        format: luks
......
    corrupt: false
    extended l2: true
3. Start a guest with the encrypted disk.
# virsh start lmn
Domain 'lmn' started
# virsh dumpxml lmn | grep /disk -B12
......
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/luks_1.img' index='1'>
        <encryption format='luks' engine='qemu'>
          <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/>
        </encryption>
      </source>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>
4. Create a snapshot and check snapshot info.
# virsh snapshot-create-as lmn s1 --disk-only
Domain snapshot s1 created
# qemu-img info /var/lib/libvirt/images/luks_1.s1 -U
image: /var/lib/libvirt/images/luks_1.s1
file format: qcow2
virtual size: 1 GiB (1073741824 bytes)
disk size: 6 MiB
cluster_size: 2097152
backing file: /var/lib/libvirt/images/luks_1.img
backing file format: qcow2
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: true
5. Do blockcopy and check the copied info.
# virsh blockcopy lmn vdb /tmp/copy.qcow2 --wait --verbose --transient-job --pivot
Block Copy: [100 %]
Successfully pivoted
# qemu-img info /tmp/copy.qcow2 -U
image: /tmp/copy.qcow2
file format: qcow2
virtual size: 1 GiB (1073741824 bytes)
disk size: 8 MiB
cluster_size: 2097152
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: true
6. Write data to the disk in guest.
[in guest] # mkfs.ext4 /dev/vdb;mount /dev/vdb /mnt/; dd if=/dev/urandom of=/mnt/file bs=1M count=100

Comment 9 Meina Li 2022-01-24 08:16:26 UTC
Verified Version:
libvirt-8.0.0-1.el9.x86_64
qemu-kvm-6.2.0-3.el9.x86_64

Verified Steps in comment 6 and the test results are expected.
Also test with qemu-img convert.
1. Prepare an image with extended_l2=on
# qemu-img info /var/lib/libvirt/images/subcluster.qcow2 
image: /var/lib/libvirt/images/subcluster.qcow2
file format: qcow2
virtual size: 500 MiB (524288000 bytes)
disk size: 6 MiB
cluster_size: 2097152
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: true
2. Convert subcluster.qcow2 to another image.
# qemu-img convert -O qcow2 -o cluster_size=2M,extended_l2=on /var/lib/libvirt/images/subcluster.qcow2 /var/lib/libvirt/images/test.qcow2
3. Check the converted image.
# qemu-img info /var/lib/libvirt/images/test.qcow2 
image: /var/lib/libvirt/images/test.qcow2
file format: qcow2
virtual size: 500 MiB (524288000 bytes)
disk size: 6 MiB
cluster_size: 2097152
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: true

Comment 11 errata-xmlrpc 2022-05-17 12:45:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: libvirt), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:2390


Note You need to log in before you can comment on or make changes to this bug.