Bug 1995865

Summary: Wrong backing-fmt in QMP command when do push mode incremental backup for luks encrypted qcow2 disk
Product: Red Hat Enterprise Linux 8 Reporter: Fangge Jin <fjin>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED ERRATA QA Contact: yisun
Severity: low Docs Contact:
Priority: low    
Version: 8.5CC: jdenemar, lmen, pkrempa, virt-maint, xuzhang, yalzhang, yisun
Target Milestone: rcKeywords: Triaged
Target Release: 8.6   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-7.8.0-1.module+el8.6.0+12978+7d7a0321 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2001327 (view as bug list) Environment:
Last Closed: 2022-05-10 13:20:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version: 7.7.0
Embargoed:
Bug Depends On:    
Bug Blocks: 2001327    

Description Fangge Jin 2021-08-20 01:08:36 UTC
Description of problem:
Wrong backing-fmt in QMP command when do push mode incremental backup for luks encrypted qcow2 disk

Version-Release number of selected component:
libvirt-7.6.0-2.module+el8.5.0+12219+a5ea13d2.x86_64
qemu-kvm-6.0.0-27.module+el8.5.0+12121+c40c8708.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Start a vm with luks encrypted qcow2 image.
vm xml snippet:
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/luks.full.backup.qcow2' index='1'>
        <encryption format='luks'>
          <secret type='passphrase' uuid='bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb'/>
        </encryption>
      </source>
      <backingStore/>
      <target dev='vdd' bus='virtio'/>
      <alias name='virtio-disk3'/>
      <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
    </disk>

# qemu-img info -U /var/lib/libvirt/images/luks.full.backup.qcow2
image: /var/lib/libvirt/images/luks.full.backup.qcow2
file format: qcow2
virtual size: 100 MiB (104857600 bytes)
disk size: 123 MiB
encrypted: yes
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    encrypt:
        ivgen alg: plain64
        hash alg: sha256
        cipher alg: aes-256
        uuid: 259a3bae-82fc-47cd-bf64-a9edd571f854
        format: luks
        cipher mode: xts
        slots:
            [0]:
                active: true
                iters: 1305498
                key offset: 4096
                stripes: 4000
            [1]:
                active: false
                key offset: 262144
            [2]:
                active: false
                key offset: 520192
            [3]:
                active: false
                key offset: 778240
            [4]:
                active: false
                key offset: 1036288
            [5]:
                active: false
                key offset: 1294336
            [6]:
                active: false
                key offset: 1552384
            [7]:
                active: false
                key offset: 1810432
        payload offset: 2068480
        master key iters: 324727
    corrupt: false
    extended l2: false

2. Create full backup and checkpoint for disk vdd
# cat backup-push-full-blk.xml 
<domainbackup mode='push'>
  <disks>
    <disk name='vda' backup='no' />
    <disk name='vdc' backup='no' />
    <disk name='vdd' backup='yes' type='block' backupmode='full'>
      <driver type='qcow2'/>
      <target dev='/dev/vdb2'>
        <encryption format='luks'>
          <secret type='passphrase' usage='/var/lib/libvirt/images/luks.backup.qcow2'/>
        </encryption>
      </target>
    </disk>
  </disks>
</domainbackup>

# cat checkpoint.xml 
<domaincheckpoint>
  <name>check10</name>
  <disks>
    <disk checkpoint='no' name='vda'/>
    <disk checkpoint='no' name='vdc'/>
    <disk checkpoint='bitmap' name='vdd'/>
  </disks>
</domaincheckpoint>

# virsh backup-begin backup-push-full-blk.xml

3. Do incremental backup for disk vdd
# cat backup.xml
<domainbackup mode='push'>
  <disks>
    <disk name='vda' backup='no' />
    <disk name='vdc' backup='no' />
    <disk name='vdd' backup='yes' type='block' backupmode='incremental' incremental='check10'>
     <driver type='qcow2'/>
      <target dev='/dev/vdb3'>
        <encryption format='luks'>
          <secret type='passphrase' usage='/var/lib/libvirt/images/luks.backup.qcow2'/>
        </encryption>
      </target>
    </disk>
  </disks>
</domainbackup>

# virsh backup-begin vm1 backup.xml
Backup started

# virsh domjobinfo vm1 --completed
Job type:         Completed  
Operation:        Backup      
Time elapsed:     6343         ms
File processed:   128.000 KiB
File remaining:   0.000 B
File total:       128.000 KiB

4. Check backup image info:
# qemu-img info --backing-chain /dev/vdb3
qemu-img: Could not open '/var/lib/libvirt/images/luks.full.backup.qcow2': Volume is not in LUKS format

# qemu-img info /dev/vdb3
image: /dev/vdb3
file format: qcow2
virtual size: 100 MiB (104857600 bytes)
disk size: 0 B
encrypted: yes
cluster_size: 65536
backing file: /var/lib/libvirt/images/luks.full.backup.qcow2
backing file format: luks
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    encrypt:
        ivgen alg: plain64
        hash alg: sha256
        cipher alg: aes-256
        uuid: 42edd449-2e13-4f9f-b241-03001c2a7b88
        format: luks
        cipher mode: xts
        slots:
            [0]:
                active: true
                iters: 1110778
                key offset: 4096
                stripes: 4000
            [1]:
                active: false
                key offset: 262144
            [2]:
                active: false
                key offset: 520192
            [3]:
                active: false
                key offset: 778240
            [4]:
                active: false
                key offset: 1036288
            [5]:
                active: false
                key offset: 1294336
            [6]:
                active: false
                key offset: 1552384
            [7]:
                active: false
                key offset: 1810432
        payload offset: 2068480
        master key iters: 323408
    corrupt: false
    extended l2: false

5. Try to rebase the backup image:
# qemu-img rebase --object secret,id=sec0,data=redhat1 -b 'json:{"encrypt.key-secret": "sec0", "driver": "qcow2", "file": {"driver": "host_device", "filename": "/dev/vdb2"}}'  -f qcow2 -F qcow2 'json:{"encrypt.key-secret": "sec0", "driver": "qcow2", "file": {"driver": "host_device", "filename": "/dev/vdb3"}}'
qemu-img: Could not open 'json:{"encrypt.key-secret": "sec0", "driver": "qcow2", "file": {"driver": "host_device", "filename": "/dev/vdb3"}}': Could not open backing file: Parameter 'key-secret' is required for cipher


6. Check QMP commands during incremental backup:
{"execute":"query-named-block-nodes","arguments":{"flat":true},"id":"libvirt-482"}
{"execute":"blockdev-add","arguments":{"driver":"host_device","filename":"/dev/vdb3","node-name":"libvirt-11-storage","auto-read-only":true,"discard":"unmap"},"id":"libvirt-483"}
{"execute":"object-add","arguments":{"qom-type":"secret","id":"libvirt-11-format-encryption-secret0","data":"WNgF+Sx+HVyEIu7YfAC8jQ==","keyid":"masterKey0","iv":"oC66OwWrXfe3jyP1kXbVHw==","format":"base64"},"id":"libvirt-484"}
{"execute":"blockdev-create","arguments":{"job-id":"create-libvirt-11-format","options":{"driver":"qcow2","file":"libvirt-11-storage","size":104857600,"cluster-size":65536,"backing-file":"/var/lib/libvirt/images/luks.full.backup.qcow2","backing-fmt":"luks","encrypt":{"key-secret":"libvirt-11-format-encryption-secret0","format":"luks"}}},"id":"libvirt-485"}
{"execute":"query-jobs","id":"libvirt-486"}
{"execute":"job-dismiss","arguments":{"id":"create-libvirt-11-format"},"id":"libvirt-487"}
{"execute":"blockdev-add","arguments":{"node-name":"libvirt-11-format","read-only":false,"driver":"qcow2","encrypt":{"format":"luks","key-secret":"libvirt-11-format-encryption-secret0"},"file":"libvirt-11-storage","backing":"libvirt-1-format"},"id":"libvirt-488"}
{"execute":"transaction","arguments":{"actions":[{"type":"block-dirty-bitmap-add","data":{"node":"libvirt-1-format","name":"backup-vdd","persistent":false,"disabled":true,"granularity":65536}},{"type":"block-dirty-bitmap-merge","data":{"node":"libvirt-1-format","target":"backup-vdd","bitmaps":[{"node":"libvirt-1-format","name":"check10"}]}},{"type":"blockdev-backup","data":{"device":"libvirt-1-format","job-id":"backup-vdd-libvirt-1-format","target":"libvirt-11-format","sync":"incremental","bitmap":"backup
{"execute":"query-jobs","id":"libvirt-490"}
{"execute":"job-dismiss","arguments":{"id":"backup-vdd-libvirt-1-format"},"id":"libvirt-491"}
{"execute":"blockdev-del","arguments":{"node-name":"libvirt-11-format"},"id":"libvirt-492"}
{"execute":"blockdev-del","arguments":{"node-name":"libvirt-11-storage"},"id":"libvirt-493"}
{"execute":"object-del","arguments":{"id":"libvirt-11-format-encryption-secret0"},"id":"libvirt-494"}
{"execute":"block-dirty-bitmap-remove","arguments":{"node":"libvirt-1-format","name":"backup-vdd"},"id":"libvirt-495"}


Actual results:
backing-fmt is wrong for incremental backup image, it leads to failure of image rebase.

Expected results:
backing-fmt should be qcow2


Additional info:

Comment 1 Peter Krempa 2021-08-20 13:45:50 UTC
Fixed upstream:

commit ca444a2eb27996b8940217d045c31a98882092e1
Author: Peter Krempa <pkrempa>
Date:   Fri Aug 20 13:25:19 2021 +0200

    qemublocktest: Add test for creating a qcow2 on top of an luks-encrypted qcow2
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

commit 15ab65583588702eb15b9f323fa57b8b5441a8ea
Author: Peter Krempa <pkrempa>
Date:   Fri Aug 20 13:26:13 2021 +0200

    qemu: block: Use correct format name when formatting overlay of qcow2+luks
    
    A logic bug in the code creating overlays on existing images resulted
    into wrongly using "luks" instead of "qcow2" for the backing format if
    the backing image is an luks-encrypted qcow2. The special format munging
    is needed only for raw luks images.
    
    In practice the impact is not as critical as to use encrypted images in
    the backing chain the user must fully describe the backing chain
    including backing images to provide encryption keys, which overrides the
    metadata recorded in the qcow2 header.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

Comment 2 John Ferlan 2021-09-05 14:25:35 UTC
Move to RHEL since RHEL-AV will only be a rebuild of RHEL starting w/ 8.6.0

Comment 5 yisun 2021-10-28 08:30:08 UTC
Verified with: libvirt-7.8.0-1.module+el8.6.0+12978+7d7a0321.x86_64
Same steps as https://bugzilla.redhat.com/show_bug.cgi?id=2001327#c1

Comment 7 errata-xmlrpc 2022-05-10 13:20:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: virt:rhel and virt-devel:rhel security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1759