Bug 1526382 - Cannot vol-create qcow2 encrypted volume due to qemu-img params changed
Summary: Cannot vol-create qcow2 encrypted volume due to qemu-img params changed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.5
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: John Ferlan
QA Contact: Meina Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-12-15 10:22 UTC by yisun
Modified: 2018-10-30 09:52 UTC (History)
8 users (show)

Fixed In Version: libvirt-4.5.0-1.el7
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2018-10-30 09:52:20 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:3113 None None None 2018-10-30 09:52:59 UTC

Description yisun 2017-12-15 10:22:29 UTC
Description of problem:
Cannot vol-create qcow2 encrypted volume due to **qemu-img create** params changed

Version-Release number of selected component (if applicable):
libvirt-3.9.0-6.el7.x86_64
qemu-kvm-rhev-2.10.0-12.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. create a encrypted vol with encryption format=default
## cat vol.xml
<volume>
<name>encrypt.img</name>
<allocation>0</allocation>
<capacity unit="G">1</capacity>
<source/>
<target>
<path>/var/lib/libvirt/images/encrypt.img</path>
<format type='qcow2'/>
<encryption format="default"/>
<permissions>
<label>virt_image_t</label>
</permissions>
<compat>1.1</compat>
<features/>
</target>
</volume>

## virsh vol-create default vol.xml
error: Failed to create vol from vol.xml
error: internal error: Child process (/usr/bin/qemu-img create -f qcow2 -o encryption=on,compat=1.1 /var/lib/libvirt/images/encrypt.img 1048576K) unexpected exit status 1: qemu-img: /var/lib/libvirt/images/encrypt.img: Parameter 'encrypt.key-secret' is required for cipher


2. create a encrypted vol with encryption format=qcow
## virsh secret-list
 UUID                                  Usage
--------------------------------------------------------------------------------
 33406385-1633-425a-a7e8-a02243603283  volume /var/lib/libvirt/images/encrypt1.img


## cat vol1.xml 
<volume>
<name>encrypt1.img</name>
<allocation>0</allocation>
<capacity unit="G">1</capacity>
<source/>
<target>
<path>/var/lib/libvirt/images/encrypt1.img</path>
<format type='qcow2'/>
<encryption format='qcow'>
       <secret type='passphrase' uuid='33406385-1633-425a-a7e8-a02243603283' />
</encryption>
<permissions>
<label>virt_image_t</label>
</permissions>
<compat>1.1</compat>
<features/>
</target>
</volume>


## virsh vol-create default vol1.xml
error: Failed to create vol from vol1.xml
error: internal error: Child process (/usr/bin/qemu-img create -f qcow2 -o encryption=on,compat=1.1 /var/lib/libvirt/images/encrypt1.img 1048576K) unexpected exit status 1: 2017-12-15 10:17:49.686+0000: 11786: debug : virFileClose:110 : Closed fd 28
2017-12-15 10:17:49.686+0000: 11786: debug : virFileClose:110 : Closed fd 30
2017-12-15 10:17:49.686+0000: 11786: debug : virFileClose:110 : Closed fd 26
qemu-img: /var/lib/libvirt/images/encrypt1.img: Parameter 'encrypt.key-secret' is required for cipher

Actual results:
vol creation failed.

Expected results:
vol creation works.

Comment 2 Daniel Berrangé 2017-12-15 10:25:51 UTC
Historically qemu would do an interactive prompt for passwords, but QEMU made an intentionally compatibility breaking change to how it gets passwords. It now mandates use of the '--object secret' framework.

QEMU decided back compat was not important, because qcow2 traditional encryption is terminally broken by design & deprecated for many releases. So no one should create new qcow2 encryopted images anymore.   LUKS is preferred instead, which is tracked in a separate libvirt bug,

Comment 3 yisun 2017-12-15 10:35:02 UTC
hmm, should provide a qemu-img version
qemu-img-rhev-2.10.0-12.el7.x86_64

yes, it's recommended to use LUKS encryption, and IIRC, latest qemu-kvm-rhev will not let you start a vm with qcow2 encrypted image by default. 
but anyway, we may keep the backward compatibility, or explicitly provide a suitable message to let the user avoid using it anymore in libvirt.

Comment 5 Jaroslav Suchanek 2017-12-15 16:09:32 UTC
Maybe it is for good that it is broken. Moving for further reconsideration in
rhel-7.6.

Comment 6 John Ferlan 2018-04-17 19:24:26 UTC
Patches posted upstream:

https://www.redhat.com/archives/libvir-list/2018-April/msg01578.html

Comment 7 John Ferlan 2018-05-16 12:42:19 UTC
Posted v2 of patches:

https://www.redhat.com/archives/libvir-list/2018-May/msg01268.html

This time rather than disallowing, alter the code to allow and use secrets for qcow[2] encrypted images.

This will leave a "gap" where installations using QEMU 2.9 or earlier will not work for qcow[2] encrypted image handling.

In addition to creation found that image conversion and resize needed adjustment and the series of patches above manages that.

Setting up a qcow[2] secret and a luks secret and using /home/vm-images as my target location for my default pool, I used the following XML for input to a:

virsh vol-create default $file.xml

$ cat create-encrypt1.xml
<volume>
  <name>encrypt1.img</name>
  <allocation>0</allocation>
  <capacity unit="M">100</capacity>
  <source/>
  <target>
    <path>/home/vm-images/encrypt1.img</path>
    <format type='qcow2'/>
    <encryption format='qcow'>
      <secret type='passphrase' uuid='f52a81b2-424e-490c-823d-6bd4235bc588'/>
    </encryption>
    <permissions>
      <label>virt_image_t</label>
    </permissions>
    <compat>1.1</compat>
    <features/>
  </target>
</volume>

$ cat create-luks.xml
<volume>
  <name>luks.img</name>
  <allocation>0</allocation>
  <capacity unit="M">100</capacity>
  <source/>
  <target>
    <path>/home/vm-images/luks.img</path>
    <format type='raw'/>
    <encryption format='luks'>
      <secret type='passphrase' uuid='5e94e1d5-a8da-4a77-bc72-0b19614db485'/>
    </encryption>
    <permissions>
      <label>virt_image_t</label>
    </permissions>
  </target>
</volume>


Similarly for conversion using a 500M "vol1" input image from the default pool and the same virsh vol-create default $file.xml command:

$cat convert-encrypt1.xml
<volume>
  <name>encrypt1.img</name>
  <allocation>0</allocation>
  <capacity unit="M">500</capacity>
  <source>
    <path>/home/vm-images/vol1</path>
  </source>
  <target>
    <path>/home/vm-images/encrypt1.img</path>
    <format type='qcow2'/>
    <encryption format='qcow'>
      <secret type='passphrase' uuid='f52a81b2-424e-490c-823d-6bd4235bc588'/>
    </encryption>
    <permissions>
      <label>virt_image_t</label>
    </permissions>
    <compat>1.1</compat>
    <features/>
  </target>
</volume>

$ cat convert-luks.xml
<volume>
  <name>luks.img</name>
  <allocation>0</allocation>
  <capacity unit="M">500</capacity>
  <source>
    <path>/home/vm-images/vol1</path>
  </source>
  <target>
    <path>/home/vm-images/luks.img</path>
    <format type='raw'/>
    <encryption format='luks'>
      <secret type='passphrase' uuid='5e94e1d5-a8da-4a77-bc72-0b19614db485'/>
    </encryption>
    <permissions>
      <label>virt_image_t</label>
    </permissions>
  </target>
</volume>

Finally for resize it's:

$ virsh vol-resize encrypt1.img 600M default

or 

$ virsh vol-resize luks.img 550M default

Comment 8 John Ferlan 2018-05-24 17:12:17 UTC
NB: Recent patches posted to remove qcow2 encrypted volume support for domains:

https://www.redhat.com/archives/libvir-list/2018-May/msg01694.html

if accepted would effectively mean qcow2 encryption support for storage and secrets doesn't really matter.

Comment 9 John Ferlan 2018-06-21 11:10:41 UTC
Based on the patches from comment 8 being accepted/pushed upstream, I posted a new series which will essentially follow the same for storage to remove support for creating or using qcow/default encrypted volumes, see:

https://www.redhat.com/archives/libvir-list/2018-June/msg01475.html

In particular, patch 5 removes the support.

Comment 10 John Ferlan 2018-06-26 18:09:36 UTC
This change is now pushed upstream:

commit 8f83af6823f1ecdb1ff09728e3482b1e4d09029f
Author: John Ferlan <jferlan@redhat.com>
Date:   Wed Jun 20 16:21:50 2018 -0400

    storage: Disallow create/resize of qcow2 encrypted images
    
...
    
    Since commit c4eedd793 disallowed qcow2 encrypted images to be
    used for domains, it no longer makes sense to allow a qcow2
    encrypted volume to be created or resized.
    
    Add a test that will exhibit the failure of creation as well
    as the xml2xml validation of the format still being correct.
    
    Update the documentation to note the removal of the capability
    to create and use qcow/default encrypted volumes.
...


$ git describe 8f83af6823f1ecdb1ff09728e3482b1e4d09029f
v4.4.0-354-g8f83af6823
$

Comment 12 Meina Li 2018-08-03 03:29:25 UTC
Hi John,

For the storage volume test of luks encryption, I have encountered some questions on it, can you help review it? Thanks in advance.

(1) For "Add support to use an raw input volume for encryption", I think it can also use an raw input volume with encryption to create an new encryption volume, but it has an error even though output file is not smaller than input file.
# virsh vol-dumpxml test1.img default
<volume type='file'>
  <name>test1.img</name>
  <key>/var/lib/libvirt/images/test1.img</key>
  <source>
  </source>
  <capacity unit='bytes'>209715200</capacity>
  <allocation unit='bytes'>262144</allocation>
  <physical unit='bytes'>209715200</physical>
  <target>
    <path>/var/lib/libvirt/images/test1.img</path>
    <format type='raw'/>
    <permissions>
      <mode>0600</mode>
      <owner>0</owner>
      <group>0</group>
      <label>system_u:object_r:virt_image_t:s0</label>
    </permissions>
    <timestamps>
      <atime>1533263232.996317301</atime>
      <mtime>1533263232.988317514</mtime>
      <ctime>1533263232.988317514</ctime>
    </timestamps>
  </target>
</volume>
# virsh vol-dumpxml encrypt1.img default
<volume type='file'>
  <name>encrypt1.img</name>
  <key>/var/lib/libvirt/images/encrypt1.img</key>
  <source>
  </source>
  <capacity unit='bytes'>209715200</capacity>
  <allocation unit='bytes'>262144</allocation>
  <physical unit='bytes'>211783680</physical>
  <target>
    <path>/var/lib/libvirt/images/encrypt1.img</path>
    <format type='raw'/>
    <permissions>
      <mode>0600</mode>
      <owner>0</owner>
      <group>0</group>
      <label>system_u:object_r:virt_image_t:s0</label>
    </permissions>
    <timestamps>
      <atime>1533261677.171485324</atime>
      <mtime>1533260873.922614416</mtime>
      <ctime>1533260873.922614416</ctime>
    </timestamps>
    <encryption format='luks'>
      <secret type='passphrase' uuid='33406385-1633-425a-a7e8-a02243603283'/>
    </encryption>
  </target>
</volume>
# cat luks-vol.xml 
<volume type='file'>
  <name>encrypt2.img</name>
  <key>/var/lib/libvirt/images/encrypt2.img</key>
  <source>
  </source>
  <capacity unit='bytes'>209715200</capacity>
  <allocation unit='bytes'>262144</allocation>
  <physical unit='bytes'>211783680</physical>
  <target>
    <path>/var/lib/libvirt/images/encrypt2.img</path>
    <format type='raw'/>
    <permissions>
      <mode>0600</mode>
      <owner>0</owner>
      <group>0</group>
      <label>system_u:object_r:virt_image_t:s0</label>
    </permissions>
    <timestamps>
      <atime>1533261677.171485324</atime>
      <mtime>1533260873.922614416</mtime>
      <ctime>1533260873.922614416</ctime>
    </timestamps>
    <encryption format='luks'>
      <secret type='passphrase' uuid='33406385-1633-425a-a7e8-a02243603283'/>
    </encryption>
  </target>
</volume>
# virsh vol-create-from default luks-vol.xml test1.img --inputpool defaultVol encrypt2.img created from input vol test1.img
# virsh vol-create-from default luks-vol.xml encrypt1.img --inputpool defaulterror: Failed to create vol from luks-vol.xml
error: internal error: Child process (/usr/bin/qemu-img convert --image-opts -n --target-image-opts --object secret,id=encrypt2.img_encrypt0,file=/var/run/libvirt/storage/default.encrypt2.img.secret.dkh1GL driver=raw,file.filename=/var/lib/libvirt/images/encrypt1.img driver=luks,file.filename=/var/lib/libvirt/images/encrypt2.img,key-secret=encrypt2.img_encrypt0) unexpected exit status 1: 2018-08-03 02:41:42.213+0000: 4077: debug : virFileClose:111 : Closed fd 28
2018-08-03 02:41:42.213+0000: 4077: debug : virFileClose:111 : Closed fd 30
2018-08-03 02:41:42.213+0000: 4077: debug : virFileClose:111 : Closed fd 25
qemu-img: output file is smaller than input file

(2) For vol-resize, it failed with --shrink.
# qemu-img info /var/lib/libvirt/images/encrypt1.img 
image: /var/lib/libvirt/images/encrypt1.img
file format: luks
virtual size:  200M (209715200 bytes)
disk size: 256K
encrypted: yes
Format specific information:
    ivgen alg: plain64
    hash alg: sha256
    cipher alg: aes-256
# virsh vol-resize /var/lib/libvirt/images/encrypt1.img 100M --shrink
error: Failed to change size of volume 'encrypt1.img' to 100M
error: internal error: Child process (/usr/bin/qemu-img resize --object secret,id=encrypt1.img_encrypt0,file=/var/run/libvirt/storage/default.encrypt1.img.secret.gVYVsx --image-opts driver=luks,key-secret=encrypt1.img_encrypt0,file.filename=/var/lib/libvirt/images/encrypt1.img 104857600) unexpected exit status 1: 2018-08-03 03:04:32.825+0000: 4388: debug : virFileClose:111 : Closed fd 28
2018-08-03 03:04:32.825+0000: 4388: debug : virFileClose:111 : Closed fd 30
2018-08-03 03:04:32.825+0000: 4388: debug : virFileClose:111 : Closed fd 25
qemu-img: warning: Shrinking an image will delete all data beyond the shrunken image's end. Before performing such an operation, make sure there is no important data there.
qemu-img: Use the --shrink option to perform a shrink operation.

(3) For "Don't allow encryption	secretPath to be NULL", how can I apply/test secretPath from the perspective of the storage volume in libvirt? I just know it will give an error if I don't use file or data in qemu-img:
# qemu-img create -f luks --object secret,id=LuksDemo.img_encrypt0 -o key-secret=LuksDemo.img_encrypt0,cipher-alg=serpent-256,cipher-mode=cbc,hash-alg=sha256,ivgen-alg=plain64,ivgen-hash-alg=sha256 /var/lib/libvirt/images/LuksDemo.img 5242880K
qemu-img: Either 'file' or 'data' must be provided

Comment 13 John Ferlan 2018-08-03 20:53:57 UTC
Been a few weeks since I thought about this and next week I'll be away, but I wanted to at least provide some feedback. 

The instructions and examples are a bit confusing since qcow[2] support has been removed as an encrypted volume type/style. So anything that was "encrypt" where format type is "default" or "qcow" - try to ignore it.

For (1) as long as you can have "some" RAW input file and generate a LUKS encrypted output file, then I think you accomplish the task.

That "vol" type='file' looks like the input and "encrypt1.img" looks a bit like an output file from/for "some" LUKS encrypted volume.  The luks-vol.xml is the file used to create encrypt2.img from test.img using the "virsh vol-create-from default luks-vol.xml encrypt1.img --inputpool default" command and it looks like it succeeds.

The second command ("virsh vol-create-from default luks-vol.xml encrypt1.img --inputpool defaul") looks like you're trying to take a LUKS encrypted encrypt1.img volume and use it as input for creation encrypt2.img...  That won't work and fails spectacularly as I would expect. In order for that to work one would have to decrypt and download encrypt1.img into some file and then create the encrypt2 from that.  Not something useful to do IMO.  There's no "mechanism" available to create an encrypted volume from another encrypted volume and a lot of that has to do with the way the qemu-img command works.

For (2), don't think I tested that type of conversion. I definitely tested the growing conversion, but not the shrinking one.  And like the comment from qemu.img points out - it's a really dangerous thing to do. In a way, perhaps good thing that qemu-img prohibits us from doing that without supplying a --shrink qualifier. We do allow shrinking non-encrypted volumes and I suppose we should allow that for encrypted volumes.  Perhaps outside the realm of the original problem report (Cannot vol-create qcow2 encrypted volume due to **qemu-img create** params changed). If you feel like creating a 7.7 related bz requesting support for resize/shrink to work, then that's fine. Not something for 7.6 at this point.

For (3) - I'm not sure it's possible (anymore), but the test and error still remains in the code as a safety valve of sorts to anyone calling the API incorrectly.

Comment 14 Meina Li 2018-08-08 10:26:55 UTC
Filed new bugs to track the above questions:bug1613737 and bug1613746.

Verified on libvirt-4.5.0-6.el7.x86_64.

Test steps:
Scenario 1: qcow2/default encryption (negative)
1. Prepare an volume xml with qcow2 encryption.
# virsh secret-list 
 UUID                                  Usage
--------------------------------------------------------------------------------
 33406385-1633-425a-a7e8-a02243603283  volume /var/lib/libvirt/images/encrypt1.img
# cat qcow-vol.xml 
<volume>
<name>encrypt1.img</name>
<allocation>0</allocation>
<capacity unit="G">1</capacity>
<source/>
<target>
<path>/var/lib/libvirt/images/encrypt1.img</path>
<format type='qcow2'/>
<encryption format='qcow'>
       <secret type='passphrase' uuid='33406385-1633-425a-a7e8-a02243603283' />
</encryption>
<permissions>
<label>virt_image_t</label>
</permissions>
<compat>1.1</compat>
<features/>
</target>
</volume>

2. Create encrypt1.img volume.
# virsh vol-create default qcow-vol.xml 
error: Failed to create vol from qcow-vol.xml
error: unsupported configuration: creation of qcow2 encrypted image is not supported

3. Prepare an guest xml with qcow2 encryption disk.
# cat demo-guest.xml
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/encrypt1.img'/>
<target dev='hda' bus='ide'/>
<encryption format='qcow'>
<secret type='passphrase' uuid='33406385-1633-425a-a7e8-a02243603283'/>
</encryption>
</disk>

4. Define guest.
# virsh define demo-guest.xml
error: Failed to define domain from demo-guest.xml
error: unsupported configuration: old qcow/qcow2 encryption is not supported

Scenario 2: luks encryption (positive)
1. Prepare an volume xml with luks encryption.
# virsh secret-list 
 UUID                                  Usage
--------------------------------------------------------------------------------
 33406385-1633-425a-a7e8-a02243603283  volume /var/lib/libvirt/images/encrypt1.img
# cat encrypt1-vol.xml
<volume>
  <name>encrypt1.img</name>
  <allocation>0</allocation>
  <capacity unit="M">100</capacity>
  <source/>
  <target>
    <path>/var/lib/libvirt/images/encrypt1.img</path>
    <format type='raw'/>
    <encryption format='luks'>
      <secret type='passphrase' uuid='33406385-1633-425a-a7e8-a02243603283'/>
    </encryption>
    <permissions>
      <label>virt_image_t</label>
    </permissions>
  </target>
</volume>

2. Create encrypt1.img volume.
# virsh vol-create default luks-vol.xml 
Vol encrypt1.img created from luks-vol.xml
# virsh vol-list default
 Name                 Path                                    
------------------------------------------------------------------------------
 encrypt1.img         /var/lib/libvirt/images/encrypt1.img    
 lmn.qcow2            /var/lib/libvirt/images/lmn.qcow2 
 test1.img            /var/lib/libvirt/images/test1.img

3. Create an encrypted volume encrypt2.img from non-encrypted volume test1.img.
# cat encrypt2-vol.xml
<volume type='file'>
  <name>encrypt2.img</name>
  <key>/var/lib/libvirt/images/encrypt2.img</key>
  <source>
  </source>
  <capacity unit='bytes'>209715200</capacity>
  <allocation unit='bytes'>262144</allocation>
  <physical unit='bytes'>211783680</physical>
  <target>
    <path>/var/lib/libvirt/images/encrypt2.img</path>
    <format type='raw'/>
    <permissions>
      <mode>0600</mode>
      <owner>0</owner>
      <group>0</group>
      <label>system_u:object_r:virt_image_t:s0</label>
    </permissions>
    <timestamps>
      <atime>1533261677.171485324</atime>
      <mtime>1533260873.922614416</mtime>
      <ctime>1533260873.922614416</ctime>
    </timestamps>
    <encryption format='luks'>
      <secret type='passphrase' uuid='44306385-1633-425a-a7e8-a02243603283'/>
    </encryption>
  </target>
</volume>

# virsh vol-create-from default encrypt2-vol.xml test1.img --inputpool default
Vol encrypt2.img created from input vol test1.img

# virsh vol-list default
 Name                 Path                                    
------------------------------------------------------------------------------
 encrypt1.img         /var/lib/libvirt/images/encrypt1.img   
 encrypt2.img         /var/lib/libvirt/images/encrypt2.img 
 lmn.qcow2            /var/lib/libvirt/images/lmn.qcow2 
 test1.img            /var/lib/libvirt/images/test1.img

4. Resize luks encrypted volume.
# virsh vol-resize /var/lib/libvirt/images/encrypt1.img 200M
Size of volume 'encrypt1.img' successfully changed to 200M
# qemu-img info /var/lib/libvirt/images/encrypt1.img 
image: /var/lib/libvirt/images/encrypt1.img
file format: luks
virtual size: 200M (209715200 bytes)
disk size: 256K
encrypted: yes
Format specific information:
    ivgen alg: plain64
    hash alg: sha256
...

5. Prepare an guest xml with luks encryption disk.
# cat demo-guest.xml
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/var/lib/libvirt/images/encrypt1.img'/>
<target dev='hda' bus='scsi'/>
<encryption format='luks'>
<secret type='passphrase' uuid='33406385-1633-425a-a7e8-a02243603283'/>
</encryption>
</disk>

6. Define guest.
# virsh define demo-guest.xml
Domain demo-guest defined from demo-guest.xml.
# virsh start demo-guest
Domain demo-guest started.
# ps aux | grep qemu | sed 's/-device/\n-device/g'
…
-device virtio-blk-pci,scsi=off,bus=pci.5,addr=0x0,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -object secret,id=scsi0-0-0-0-luks-secret0,data=e6hhHm1FwNl9lEMPxXqzbg==,keyid=masterKey0,iv=vKIher/8mJNL5ezU8+Zh8g==,format=base64 -drive file=/var/lib/libvirt/images/encrypt1.img,key-secret=scsi0-0-0-0-luks-secret0,format=luks,if=none,id=drive-scsi0-0-0-0 
...

The results are expected and move this bug to verified.

Comment 16 errata-xmlrpc 2018-10-30 09:52:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113


Note You need to log in before you can comment on or make changes to this bug.