Bug 1375855

Summary: LUKS encryption not supported for snapshots' backing files.
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: yisun
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED ERRATA QA Contact: yisun
Severity: medium Docs Contact:
Priority: medium    
Version: 8.0CC: d8514, dyuan, jdenemar, kchamart, lmen, pkrempa, xuzhang
Target Milestone: rc   
Target Release: 8.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-5.10.0-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-05 09:43:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 760547, 1788898    
Bug Blocks: 1821539, 1301026    

Description yisun 2016-09-14 06:39:11 UTC 
decription:
luks encryption not supported for snapshots' backing files. 


versions:
libvirt-2.0.0-8.el7.x86_64

how reproducible: 100%

This is from bug https://bugzilla.redhat.com/show_bug.cgi?id=1301021#c20



steps:
# virsh dumpxml virtlab_test_copy
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/luks/hei.img'/>
      <target dev='vdd' bus='virtio'/>
      <encryption format='luks'>
        <secret type='passphrase' uuid='4ff78f8c-6ee6-4a8d-b638-2b59d5d49279'/>
      </encryption>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </disk>
...



# virsh start virtlab_test_copy
Domain virtlab_test_copy started


# virsh domblklist virtlab_test_copy
Target     Source
------------------------------------------------
...
vdd        /var/lib/libvirt/images/luks/hei.img


# virsh snapshot-create-as virtlab_test_copy snap1 --disk-only --diskspec vdd,file=/tmp/hei.img.snap1
Domain snapshot snap1 created


# virsh dumpxml virtlab_test_copy
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/tmp/hei.img.snap1'/>
      <backingStore type='file' index='1'>
        <format type='raw'/>
        <source file='/var/lib/libvirt/images/luks/hei.img'/> 
        <backingStore/>
      </backingStore>
      <target dev='vdd' bus='virtio'/>
      <alias name='virtio-disk3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </disk>
...




# virsh destroy virtlab_test_copy
Domain virtlab_test_copy destroyed



# virsh start virtlab_test_copy
error: Failed to start domain virtlab_test_copy
error: internal error: process exited while connecting to monitor: 2016-09-05T11:31:39.269057Z qemu-kvm: -drive file=/tmp/hei.img.snap1,format=qcow2,if=none,id=drive-virtio-disk3: Could not open backing file: No secret with id 'virtio-disk3-luks-secret0'


//now the vm is still in shutdown status
# virsh dumpxml virtlab_test_copy   
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/tmp/hei.img.snap1'/>
      <target dev='vdd' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </disk>
...

Expected result:
luks encryption should be supported for snapshots' backing file
Comment 3 Jaroslav Suchanek 2019-04-24 12:26:37 UTC 
This bug is going to be addressed in next major release.
Comment 4 yisun 2019-11-18 08:40:10 UTC 
*** Bug 1773399 has been marked as a duplicate of this bug. ***
Comment 5 Peter Krempa 2019-11-27 09:45:23 UTC 
To start the VM properly after the snapshot libvirt must be able to specify the full backing chain on the command line which allows to provide the secret for encryption of the disk. This is possible now when -blockdev is used to instantiate the disks on the command line.

The blockdev feature was enabled since:

commit c6a9e54ce3252196f1fc6aa9e57537a659646d18
Author: Peter Krempa <pkrempa>
Date:   Mon Jan 7 11:45:19 2019 +0100

    qemu: enable blockdev support

    Now that all pieces are in place (hopefully) let's enable -blockdev.

    We base the capability on presence of the fix for 'auto-read-only' on
    files so that blockdev works properly, mandate that qemu supports
    explicit SCSI id strings to avoid ABI regression and that the fix for
    'savevm' is present so that internal snapshots work.

v5.9.0-390-gc6a9e54ce3

and requires upstream qemu-4.2 or appropriate downstream.
Comment 7 yisun 2020-01-07 08:28:42 UTC 
Hi Peter,
Pls help to check if following issue should be fixed within this bz
(libvirt version: libvirt-5.10.0-1.module+el8.2.0+5135+ed3b2489.x86_64)

Problem: The encryption info lost when vm destroyed
1.  Have a running vm with a luks encrypted disk=vdb
[root@dell-per730-58 bug]#  virsh dumpxml vm1 | awk '/<disk/,/<\/disk/'
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/home/images/raw.luks' index='1'>
        <encryption format='luks'>
          <secret type='passphrase' uuid='0a5fa242-64a8-48dd-932e-340f56b5ad48'/>
        </encryption>
      </source>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>
[root@dell-per730-58 bug]# virsh domstate vm1
running

2. make a diskonly snapshot
[root@dell-per730-58 bug]# virsh snapshot-create-as vm1 snap1 --disk-only
Domain snapshot snap1 created

3. Check the vm's xml, everything is ok now, disk has encryption info:
[root@dell-per730-58 bug]# virsh dumpxml vm1 --inactive| awk '/<disk/,/<\/disk/'
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/images/raw.snap1'/>
      <backingStore type='file'>
        <format type='raw'/>
        <source file='/home/images/raw.luks'>
          <encryption format='luks'>
            <secret type='passphrase' uuid='0a5fa242-64a8-48dd-932e-340f56b5ad48'/>
          </encryption>
        </source>
      </backingStore>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>

4. Destroy vm, the encryption info gone, and vm cannot be started unless we manually edit its xml
[root@dell-per730-58 bug]# virsh destroy vm1
Domain vm1 destroyed

[root@dell-per730-58 bug]# virsh dumpxml vm1 --inactive| awk '/<disk/,/<\/disk/'
   ....
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/images/raw.snap1'/>
      <backingStore type='file'>
        <format type='raw'/>
        <source file='/home/images/raw.luks'/>
<====== encryption info gone here, and will lead to a failure of vm starting
      </backingStore>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>

[root@dell-per730-58 bug]# virsh start vm1
error: Failed to start domain vm1
error: internal error: process exited while connecting to monitor: 2020-01-07T08:17:37.221533Z qemu-kvm: -blockdev {"node-name":"libvirt-4-format","read-only":false,"driver":"qcow2","file":"libvirt-4-storage","backing":null}: Could not reopen file: Permission denied
Comment 8 Peter Krempa 2020-01-13 14:07:14 UTC 
The above problem was correctly extracted as bug https://bugzilla.redhat.com/show_bug.cgi?id=1788898 and fixed there.
Comment 9 yisun 2020-02-26 09:08:32 UTC 
Test result: PASS
[root@hp-dl320eg8-05 bz1375855]# rpm -qa | egrep "^libvirt-6|^qemu-kvm-4"
qemu-kvm-4.2.0-10.module+el8.2.0+5740+c3dff59e.x86_64
libvirt-6.0.0-5.module+el8.2.0+5765+64816f89.x86_64


1. Prepare 2 luks encrypted image, one is qcow2 native luks, the other is raw luks
[root@hp-dl320eg8-05 ~]# qemu-img create --object secret,id=sec0,data=123456 -f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0 /var/lib/libvirt/images/vda.qcow2 1G
Formatting '/var/lib/libvirt/images/vda.qcow2', fmt=qcow2 size=1073741824 encrypt.format=luks encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off refcount_bits=16

[root@hp-dl320eg8-05 ~]# qemu-img create -f luks --object secret,id=sec0,data=`printf %s "123456" | base64`,format=base64 -o key-secret=sec0 /var/lib/libvirt/images/vdb.luks 500M
Formatting '/var/lib/libvirt/images/vdb.luks', fmt=luks size=524288000 key-secret=sec0

2. Prepare libvirt secrets
[root@hp-dl320eg8-05 bz1375855]# virsh secret-define sec1.xml
Secret f981dd17-143f-45bc-88e6-111111111111 created

[root@hp-dl320eg8-05 bz1375855]# virsh secret-define sec2.xml
Secret f981dd17-143f-45bc-88e6-222222222222 created

[root@hp-dl320eg8-05 bz1375855]# virsh secret-dumpxml f981dd17-143f-45bc-88e6-111111111111
<secret ephemeral='no' private='yes'>
  <uuid>f981dd17-143f-45bc-88e6-111111111111</uuid>
  <description>LUKS Secret</description>
  <usage type='volume'>
    <volume>/var/lib/libvirt/images/vda.qcow2</volume>
  </usage>
</secret>


[root@hp-dl320eg8-05 bz1375855]# virsh secret-dumpxml f981dd17-143f-45bc-88e6-222222222222
<secret ephemeral='no' private='yes'>
  <uuid>f981dd17-143f-45bc-88e6-222222222222</uuid>
  <description>LUKS Secret</description>
  <usage type='volume'>
    <volume>/var/lib/libvirt/images/vdb.luks</volume>
  </usage>
</secret>

[root@hp-dl320eg8-05 bz1375855]# MYSECRET=`printf %s "123456" | base64`

[root@hp-dl320eg8-05 bz1375855]# virsh secret-set-value f981dd17-143f-45bc-88e6-111111111111 $MYSECRET
Secret value set

[root@hp-dl320eg8-05 bz1375855]# virsh secret-set-value f981dd17-143f-45bc-88e6-222222222222 $MYSECRET
Secret value set

3. Start a vm with the 2 luks images. Vda has <encryption> part outside <source>. Vdb has <encryption> part inside <source>
[root@hp-dl320eg8-05 bz1375855]# virsh start vm1
Domain vm1 started

[root@hp-dl320eg8-05 bz1375855]# virsh dumpxml vm1 | awk '/<disk/,/<\/disk/'
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/vda.qcow2' index='2'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <encryption format='luks'>
        <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-111111111111'/>
      </encryption>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/vdb.luks' index='1'>
        <encryption format='luks'>
          <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-222222222222'/>
        </encryption>
      </source>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>

4. Create 4 disk snapshots for the vm
[root@hp-dl320eg8-05 bz1375855]# for i in {1..4}; do virsh snapshot-create-as vm1 snap$i --disk-only; done
Domain snapshot snap1 created
Domain snapshot snap2 created
Domain snapshot snap3 created
Domain snapshot snap4 created

5. Check the <encryption> part still existing in right place in the backing chain
[root@hp-dl320eg8-05 bz1375855]# virsh dumpxml vm1 | awk '/<disk/,/<\/disk/'
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/vda.snap4' index='9'/>
      <backingStore type='file' index='7'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vda.snap3'/>
        <backingStore type='file' index='5'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/vda.snap2'/>
          <backingStore type='file' index='3'>
            <format type='qcow2'/>
            <source file='/var/lib/libvirt/images/vda.snap1'/>
            <backingStore type='file' index='2'>
              <format type='qcow2'/>
              <source file='/var/lib/libvirt/images/vda.qcow2'>
                <encryption format='luks'>
                  <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-111111111111'/>
                </encryption>
              </source>
              <backingStore/>
            </backingStore>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/vdb.snap4' index='10'/>
      <backingStore type='file' index='8'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vdb.snap3'/>
        <backingStore type='file' index='6'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/vdb.snap2'/>
          <backingStore type='file' index='4'>
            <format type='qcow2'/>
            <source file='/var/lib/libvirt/images/vdb.snap1'/>
            <backingStore type='file' index='1'>
              <format type='raw'/>
              <source file='/var/lib/libvirt/images/vdb.luks'>
                <encryption format='luks'>
                  <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-222222222222'/>
                </encryption>
              </source>
              <backingStore/>
            </backingStore>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>


6. Destroy and start the vm again, make sure vm can be started, no encryption error.
[root@hp-dl320eg8-05 bz1375855]# virsh destroy vm1; virsh start vm1; virsh domblklist vm1
Domain vm1 destroyed
Domain vm1 started

 Target   Source
---------------------------------------------
 vda      /var/lib/libvirt/images/vda.snap4
 vdb      /var/lib/libvirt/images/vdb.snap4

[root@hp-dl320eg8-05 bz1375855]# virsh dumpxml vm1 | awk '/<disk/,/<\/disk/'
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/vda.snap4' index='6'/>
      <backingStore type='file' index='7'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vda.snap3'/>
        <backingStore type='file' index='8'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/vda.snap2'/>
          <backingStore type='file' index='9'>
            <format type='qcow2'/>
            <source file='/var/lib/libvirt/images/vda.snap1'/>
            <backingStore type='file' index='10'>
              <format type='qcow2'/>
              <source file='/var/lib/libvirt/images/vda.qcow2'>
                <encryption format='luks'>
                  <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-111111111111'/>
                </encryption>
              </source>
              <backingStore/>
            </backingStore>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/vdb.snap4' index='1'/>
      <backingStore type='file' index='2'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vdb.snap3'/>
        <backingStore type='file' index='3'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/vdb.snap2'/>
          <backingStore type='file' index='4'>
            <format type='qcow2'/>
            <source file='/var/lib/libvirt/images/vdb.snap1'/>
            <backingStore type='file' index='5'>
              <format type='raw'/>
              <source file='/var/lib/libvirt/images/vdb.luks'>
                <encryption format='luks'>
                  <secret type='passphrase' uuid='f981dd17-143f-45bc-88e6-222222222222'/>
                </encryption>
              </source>
              <backingStore/>
            </backingStore>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>
Comment 11 errata-xmlrpc 2020-05-05 09:43:16 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2017