Description: virtual disk's backing file encryption info lost when start/restart vm Versions: qemu-kvm-4.2.0-4.module+el8.2.0+5220+e82621dc.x86_64 libvirt-5.10.0-1.module+el8.2.0+5135+ed3b2489.x86_64 How reproducible: 100% Steps: 1. making a disk chain: top.qcow2 ---> base.qcow2, in which base.qcow2 is luks encrypted qcow2 file # qemu-img create --object secret,id=sec0,data=123456 -f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0 base.qcow2 1G Formatting 'base.qcow2', fmt=qcow2 size=1073741824 encrypt.format=luks encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off refcount_bits=16 # qemu-img create -f qcow2 top.qcow2 100M Formatting 'top.qcow2', fmt=qcow2 size=104857600 cluster_size=65536 lazy_refcounts=off refcount_bits=16 # qemu-img rebase --object secret,id=sec0,data=123456 --image-opts driver=qcow2,file.filename=top.qcow2 -b 'json:{"encrypt.key-secret": "sec0", "driver": "qcow2", "file": {"driver": "file", "filename": "/home/images/base.qcow2"}}' 2. add the disk chain into a vm's xml # virsh secret-list UUID Usage -------------------------------------------------------------------------- 0a5fa242-64a8-48dd-932e-340f56b5ad48 volume /home/images/base.qcow2 # MYSECRET=`printf %s "123456" | base64` # virsh secret-set-value 8d17b267-7d6d-475d-8ba0-2549acc0d4ce $MYSECRET Secret value set # virsh edit vm1 ... <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/home/images/top.qcow2'/> <backingStore type='file'> <format type='qcow2'/> <source file='/home/images/base.qcow2'> <encryption format='luks'> <secret type='passphrase' uuid='0a5fa242-64a8-48dd-932e-340f56b5ad48'/> </encryption> </source> <backingStore/> </backingStore> <target dev='vdb' bus='virtio'/> <alias name='virtio-disk1'/> </disk> 3. start the vm, the vm can be started normally # virsh start vm1 Domain vm1 started qemu process has correct info about base.qcow2's encryption # ps -ef | grep vm1 qemu 23898 1 99 05:22 ? 00:00:37 /usr/libexec/qemu-kvm -name guest=vm1... -device virtio-blk-pci,scsi=off,bus=pci.4,addr=0x0,drive=libvirt-3-format,id=virtio-disk0,bootindex=1 -object secret,id=libvirt-2-format-luks-secret0,data=6OiOyd7roAIby5YbQ2QSrQ==,keyid=masterKey0,iv=/gWthTSNaeNBP0KHY4m8Gg==,format=base64 -blockdev {"driver":"file","filename":"/home/images/base.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":true,"driver":"qcow2","encrypt":{"format":"luks","key-secret":"libvirt-2-format-luks-secret0"},"file":"libvirt-2-storage","backing":null} ... 4. check the vm's xml now, <encryption> part gone # virsh dumpxml vm1| awk '/<disk/,/<\/disk/' setlocale: No such file or directory ... <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/home/images/top.qcow2' index='1'/> <backingStore type='file' index='2'> <format type='qcow2'/> <source file='/home/images/base.qcow2'/> <backingStore/> </backingStore> <target dev='vdb' bus='virtio'/> <alias name='virtio-disk1'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </disk> 5. since the encryption part gone, vm cannot be restarted due to missing cipher # virsh destroy vm1; virsh start vm1 Domain vm1 destroyed error: Failed to start domain vm1 error: internal error: process exited while connecting to monitor: 2020-01-08T10:25:59.327051Z qemu-kvm: -blockdev {"node-name":"libvirt-2-format","read-only":true,"driver":"qcow2","file":"libvirt-2-storage","backing":null}: Parameter 'encrypt.key-secret' is required for cipher Actual result: In step4, the encryption part lost Expected resutl: The backing file's encryption info should be kept so vm can be resatrted successfully
Fixed upstream: commit 3f2d167d9c733f588e693d44d7aa9b21dcb415c7 (HEAD -> master, origin/master, origin/HEAD) Author: Peter Krempa <pkrempa> Date: Fri Jan 10 17:25:16 2020 +0100 conf: Always format storage source auth and encryption under <source> for backing files Historically there are two places where we format authentication and encryption for a disk. The logich which formats it for backing files was flawed though and didn't format it at all. This worked if the image became a backing file through the means of a snapshot but not directly. Force formatting of the source and encryption for any non-disk case to fix the issue. This caused problems in many places as we use the formatter to copy the definition. Effectively any copy lost the secret definition.
Test result: PASS [root@hp-dl320eg8-05 bz1788898]# rpm -qa | egrep "^libvirt-6|^qemu-kvm-4" qemu-kvm-4.2.0-10.module+el8.2.0+5740+c3dff59e.x86_64 libvirt-6.0.0-5.module+el8.2.0+5765+64816f89.x86_64 1. prepare a secret for luks image [root@hp-dl320eg8-05 bz1788898]# virsh secret-define sec.xml Secret f981dd17-143f-45bc-88e6-ed1fe20ce9da created [root@hp-dl320eg8-05 bz1788898]# MYSECRET=`printf %s "123456" | base64` [root@hp-dl320eg8-05 bz1788898]# [root@hp-dl320eg8-05 bz1788898]# virsh secret-set-value f981dd17-143f-45bc-88e6-ed1fe20ce9da $MYSECRET Secret value set 2. prepare a luks image 'luks.qcow2' [root@hp-dl320eg8-05 bz1788898]# qemu-img create --object secret,id=sec0,data=123456 -f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0 /var/lib/libvirt/images/luks.qcow2 1G Formatting '/var/lib/libvirt/images/luks.qcow2', fmt=qcow2 size=1073741824 encrypt.format=luks encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off refcount_bits=16 3. create a new image 'top.qcow2' based on luks.qcow2 [root@hp-dl320eg8-05 bz1788898]# qemu-img create -f qcow2 /var/lib/libvirt/images/top.qcow2 100M Formatting '/var/lib/libvirt/images/top.qcow2', fmt=qcow2 size=104857600 cluster_size=65536 lazy_refcounts=off refcount_bits=16 [root@hp-dl320eg8-05 bz1788898]# qemu-img rebase --object secret,id=sec0,data=123456 --image-opts driver=qcow2,file.filename=/var/lib/libvirt/images/top.qcow2 -b 'json:{"encrypt.key-secret": "sec0", "driver": "qcow2", "file": {"driver": "file", "filename": "/var/lib/libvirt/images/luks.qcow2"}}' 4. prepare correct disk xml in vm as follow [root@hp-dl320eg8-05 bz1788898]# virsh dumpxml vm1 --inactive| awk '/<disk/,/<\/disk/' ... <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/top.qcow2'/> <backingStore type='file'> <format type='qcow2'/> <source file='/var/lib/libvirt/images/luks.qcow2'> <encryption format='luks'> <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/> </encryption> </source> <backingStore/> </backingStore> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </disk> ... 5. start vm [root@hp-dl320eg8-05 bz1788898]# virsh start vm1 Domain vm1 started 6. restart vm, now everything is ok: vm can be started; disk xml is still correct; qemu process has correct secret info [root@hp-dl320eg8-05 bz1788898]# virsh destroy vm1; sleep 2; virsh start vm1 Domain vm1 destroyed Domain vm1 started [root@hp-dl320eg8-05 bz1788898]# virsh dumpxml vm1 | awk '/<disk/,/<\/disk/' ... <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/top.qcow2' index='1'/> <backingStore type='file' index='2'> <format type='qcow2'/> <source file='/var/lib/libvirt/images/luks.qcow2'> <encryption format='luks'> <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-ed1fe20ce9da'/> </encryption> </source> <backingStore/> </backingStore> <target dev='vdb' bus='virtio'/> <alias name='virtio-disk1'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </disk> [root@hp-dl320eg8-05 bz1788898]# ps -ef | grep vm1 | grep luks ... -object secret,id=libvirt-2-format-luks-secret0,data=WQeoTGCWco2A3XA0LvgmiQ==,keyid=masterKey0,iv=xpit3ZeGEYjnf7DUn0ewkg==,format=base64 -blockdev {"driver":"file","filename":"/var/lib/libvirt/images/luks.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":true,"driver":"qcow2","encrypt":{"format":"luks","key-secret":"libvirt-2-format-luks-secret0"},"file":"libvirt-2-storage","backing":null} (ps: auth part is found and covered by bz1789310)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2017