See also: BZ 1625604

Basically the problem here is that qemu-img doesn't have the flexibility
to create all the backing chain, but rather only supports chain of length of 2
(that is leaf image and its immediate backing file)

The problem is that backing chain length is not bound, and for each node,
the user can supply various parameters, including which encryption key to use.
the qmp blockdev-add/blockdev-create solves this in the right way.

Another thought is that if block devices in the backing chain all have different passwords,
the external application managing it would have hard time tracking all these passwords.

Also note that qemu-img sort of can open a qcow2 image encrypted with one password,
while its backing file encrypted with another password, if the backing file
is stored in json form in the snapshot qcow2. This is ugly as hell, since in this case,
the backing file contains the 'secret id', which might already be in use, and short of
modifying the backing file in the actual qcow2 image, you have to supply secret with that id.

Example:

qemu-img info ./sn1.qcow2 
image: ./sn1.qcow2
file format: qcow2
virtual size: 500M (524288000 bytes)
disk size: 258M
encrypted: yes
cluster_size: 65536
backing file: json:{
        "encrypt.key-secret": "sec1", 
        "driver": "qcow2", 
        "file": { "driver": "file", "filename": "base2.qcow2"}
    }
backing file format: qcow2
Format specific information:


The whole thing really needs discussion on the mailing list,
and I would say, it might be a good idea to start writing
a replacement for qemu-img which would be much more aligned with
blockdev-* interfaces.

Created attachment 1621043 [details]
demo script

Another thing worth noting is that you can right now specify the whole backing chain if you use the 'json' syntax,
thus I think that this bug should actually be closed.

I attach a quick and dirty shell script which demonstrates this feature.

One thing that we should improve I think, but not related to this bug,
is that when you specify backing file as a json, it encodes the whole
of it into the image which might not be desired

Hi, Maxim
  
   Thanks for your script, i tried it and it works ok for qemu-img + luks/luks-inside-qcow2 operations with different/same encryption key on snapshots chain.
   
   As you described in comment4, the key point to open an encrypted backing file is that the snapshot node should contain the backing encryption info in json format, like:
#qemu-img info ./sn1.qcow2
image: ./sn1.qcow2
....
backing file: json:{
        "encrypt.key-secret": "sec1", 
        "driver": "qcow2", 
        "file": { "driver": "file", "filename": "base2.qcow2"}
    }

So the question is:
   How can i add this info to my snapshot node by blockdev-create? 
   At present, only params 'backing-file'and 'backing-fmt' supported in blockdev-create.
   {'execute':'blockdev-create','arguments':{'options': {'driver': 'qcow2','file':'drive_sn2','size':1073741824,'encrypt': {'key-secret': 'sec0', 'format': 'luks'},'backing-file':'/root/sn1','backing-fmt':'qcow2'},'job-id':'job2'}}

When switch to -blockdev, 'blockdev-create' not 'qemu-img' is suggested to use for nodes creation during block jobs operations, so i think a solution for this issue is needed.


BR,
aliang

Hi!

For blockdev-create, the backing-file is a string which is written as is to the qcow2 file. I think it can be a json blob as well, just like in case of qemu-img create.

Or, it is also possible to not specify a backing file at all, but specify it explicitly when opening the block devices (that needs some external means of tracking the backing chain.
This is possible because blockdev-add qmp command accepts an already opened blockdev for both backing file, thus you can open the whatever qcow2 image you know is a backing file and use
a whatever password that is needed to do so and then open the image itsel and specify that opened block device as the backing file.

(In reply to Maxim Levitsky from comment #9)
> Hi!
Sorry for the late reply, just back from Shanghai Open Stack submit meeting.
> 
> For blockdev-create, the backing-file is a string which is written as is to
> the qcow2 file. I think it can be a json blob as well, just like in case of
> qemu-img create.

Maxim, I tried with json format, but still failed:
 
{'execute':'blockdev-create','arguments':{'options': {'driver': 'qcow2','file':'drive_sn1','size':21474836480,'encrypt': {'key-secret': 'sec0', 'format': 'luks'},'backing':{'driver':'qcow2','file':'/home/kvm_autotest_root/images/rhel8.luks','encrypt.key-secret':'sec0'}},'job-id':'job2'}}
{"error": {"class": "GenericError", "desc": "Parameter 'options.backing' is unexpected"}}

Any error in my command?
 
> 
> Or, it is also possible to not specify a backing file at all, but specify it
> explicitly when opening the block devices (that needs some external means of
> tracking the backing chain.
How to? Can you give me an example?

Work flow for my block jobs always be:
    create image online--> add this new node to node graph by blockdev-add --> do block jobs on this node(like snapshot)

It seems opening the node will be the final step...:)

> This is possible because blockdev-add qmp command accepts an already opened
> blockdev for both backing file, thus you can open the whatever qcow2 image
> you know is a backing file and use
> a whatever password that is needed to do so and then open the image itsel
> and specify that opened block device as the backing file.


BR,
aliang

Yes. The backing file should be  a string, but in json format.

It should be something like

'backing':"json://{'driver':'qcow2','file':'/home/kvm_autotest_root/images/rhel8.luks','encrypt.key-secret':'sec0'}"

or something like that

Best regards,
    Maxim Levitsky

It's an usage error for image creation with encrypted backing file, update its steps bellow, and close this bug as "NOTABUG".


Test Steps:
  1.Start guest with imageformat "luks-inside-qcow2":
   /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1' \
    -machine pc  \
    -nodefaults \
    -device VGA,bus=pci.0 \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20181225-045358-gDqDsfCu,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20181225-045358-gDqDsfCu,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idHtZPbY  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20181225-045358-gDqDsfCu,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20181225-045358-gDqDsfCu,path=/var/tmp/seabios-20181225-045358-gDqDsfCu,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20181225-045358-gDqDsfCu,iobase=0x402 \
    -device qemu-xhci,id=usb1,bus=pci.0 \
    -object secret,id=sec0,data=backing \
    -blockdev driver=file,node-name=driveimage1,filename=/home/kvm_autotest_root/images/rhel8.qcow2 \
    -blockdev node-name=drive_image1,file=driveimage1,driver=qcow2,encrypt.key-secret=sec0,encrypt.format=luks \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0 \
    -device virtio-net-pci,mac=9a:14:15:16:17:18,id=idQJqHNz,vectors=4,netdev=idXGWoum,bus=pci.0  \
    -netdev tap,id=idXGWoum,vhost=on \
    -m 14336  \
    -smp 10,maxcpus=10,cores=5,threads=1,sockets=2  \
    -cpu 'Broadwell',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \
    -qmp tcp:0:3000,server,nowait \
    -blockdev driver=file,node-name=drivedata1,filename=/home/data1.qcow2 \
    -blockdev node-name=drive_data1,file=drivedata1,driver=qcow2,encrypt.key-secret=sec0,encrypt.format=luks \
    -device virtio-blk-pci,id=data1,drive=drive_data1,bus=pci.0 \
    -blockdev driver=file,node-name=drivedata2,filename=/home/data2.qcow2 \
    -blockdev node-name=drive_data2,file=drivedata2,driver=qcow2 \
    -device virtio-blk-pci,id=data2,drive=drive_data2,bus=pci.0 \

  2. Create an image in previous for full backup.
   {'execute':'blockdev-create','arguments':{'options': {'driver':'file','filename':'/root/sn1','size':1073741824},'job-id':'job1'}}
   {'execute':'blockdev-add','arguments':{'driver':'file','node-name':'drive_sn1','filename':'/root/sn1'}}
   {'execute':'blockdev-create','arguments':{'options': {'driver': 'qcow2','file':'drive_sn1','size':1073741824,'encrypt':{'key-secret':'sec0', 'format': 'luks'}},'job-id':'job2'}}
   {'execute':'blockdev-add','arguments':{'driver':'qcow2','node-name':'sn1','file':'drive_sn1','encrypt': {'key-secret':'sec0','format': 'luks'}}}
   {'execute':'job-dismiss','arguments':{'id':'job1'}}
   {'execute':'job-dismiss','arguments':{'id':'job2'}}

  3. Do full backup and add bitmap "bitmap1"
   {"execute": "transaction", "arguments": { "actions": [ {"type": "blockdev-backup", "data": { "device": "drive_image1", "target": "sn1", "sync": "full", "job-id":"j1" } }, {"type": "block-dirty-bitmap-add", "data": { "node": "drive_image1", "name": "bitmap1" } } ] } }

  4. Do dd in guest.
  (guest)#dd if=/dev/urandom of=a bs=1M count=1 oflag=dsync

  5. Create image for incremental live backup.
     {'execute':'blockdev-create','arguments':{'options': {'driver':'file','filename':'/root/sn2','size':1073741824},'job-id':'job1'}}" >&3
     {'execute':'blockdev-add','arguments':{'driver':'file','node-name':'drive_sn2','filename':'/root/sn2'}}
     {'execute':'blockdev-create','arguments':{'options': {'driver': 'qcow2','file':'drive_sn2','size':1073741824,'encrypt': {'key-secret': 'sec0', 'format': 'luks'},'backing-file':"json:{'driver':'qcow2','file':{'driver':'file','filename':'/root/sn1'},'encrypt.key-secret':'sec0'}",'backing-fmt':'qcow2'},'job-id':'job2'}}
     {'execute':'blockdev-add','arguments':{'driver':'qcow2','node-name':'sn2','file':'drive_sn2','encrypt': {'key-secret': 'sec0', 'format': 'luks'},'backing':'sn1'}}
     {'execute':'job-dismiss','arguments':{'id':'job1'}}
     {'execute':'job-dismiss','arguments':{'id':'job2'}}

  6. Do incremental live backup.
    {"execute":"blockdev-backup","arguments":{"device":"drive_image1","target":"sn2","sync":"incremental","bitmap":"bitmap1","job-id":"j1"}}

  7. Check sn2 image info:
  #qemu-img info /root/sn2
   image: /root/sn2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 8.5M
encrypted: yes
cluster_size: 65536
backing file: json:{'driver':'qcow2','file':{'driver':'file','filename':'/root/sn1'},'encrypt.key-secret':'sec0'}
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    encrypt:
        ivgen alg: plain64
        hash alg: sha256
        cipher alg: aes-256
        uuid: 8ff66f21-2c95-4326-a868-19da40d6a826
        format: luks
        cipher mode: xts
        slots:
            [0]:
                active: true
                iters: 100206
                key offset: 4096
                stripes: 4000
            [1]:
                active: false
                key offset: 262144
            [2]:
                active: false
                key offset: 520192
            [3]:
                active: false
                key offset: 778240
            [4]:
                active: false
                key offset: 1036288
            [5]:
                active: false
                key offset: 1294336
            [6]:
                active: false
                key offset: 1552384
            [7]:
                active: false
                key offset: 1810432
        payload offset: 2068480
        master key iters: 25634
    corrupt: false

  8. After backup finished,start guest with sn2.
    ...
    -blockdev driver=file,node-name=drivedata1,filename=/root/sn2 \
    -blockdev node-name=drive_data1,file=drivedata1,driver=qcow2,encrypt.key-secret=sec0,encrypt.format=luks \
    -device virtio-blk-pci,id=data1,drive=drive_data1,bus=pci.0 \
    ...

Test Result:
  After step8, guest can start up successfully.