Bug 1804617

Summary: Fail to parse json backing without file object
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: Han Han <hhan>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED ERRATA QA Contact: Han Han <hhan>
Severity: medium Docs Contact:
Priority: high    
Version: 8.2CC: eblake, jdenemar, jferlan, jinzhao, jsuchane, juzhang, lmen, mlevitsk, mprivozn, mtessun, pkrempa, virt-maint, xuzhang
Target Milestone: rcKeywords: Triaged
Target Release: 8.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-6.0.0-15.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-05 09:57:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Han Han 2020-02-19 09:47:04 UTC
Description of problem:
As subject

Version-Release number of selected component (if applicable):
libvirt-6.0.0-5.module+el8.2.0+5765+64816f89.x86_64
qemu-kvm-4.2.0-10.module+el8.2.0+5740+c3dff59e.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare an VM with nvme disk:
# virsh dumpxml q35|xmllint --xpath //disk -
<disk type="nvme" device="disk">
      <driver name="qemu" type="qcow2" copy_on_read="on"/>
      <source type="pci" managed="yes" namespace="1" index="1">
        <address domain="0x0000" bus="0x85" slot="0x00" function="0x0"/>
      </source>
      <target dev="sda" bus="usb"/>
      <alias name="ua-usb"/>
      <address type="usb" bus="0" port="4"/>
    </disk>

2. Create external snapshot
# virsh snapshot-create-as q35 ss --no-metadata --disk-only --diskspec sda,file=/tmp/nvme.ss
Domain snapshot ss created

3. Check the backing chain of snapshot

# qemu-img info /tmp/nvme.ss -U                
image: /tmp/nvme.ss
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 200 KiB
cluster_size: 65536
backing file: json:{"driver":"nvme","device":"0000:85:00.0","namespace":1}
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

# qemu-img info /tmp/nvme.ss -U --backing-chain                                    
qemu-img: Could not open 'json:{"driver":"nvme","device":"0000:85:00.0","namespace":1}': A block device must be specified for "file"


Actual results:
As above

Expected results:
Show the full backing chain info.

Additional info:

Comment 11 Han Han 2020-03-17 01:48:29 UTC
Hi Michel,
Could you help to see if it is a libvirt issue?

Comment 16 Peter Krempa 2020-03-20 10:39:02 UTC
Oh, I understand now! The issue is really in libvirt's new function that produces the strings for the backing file specification which I added for -blockdev usage.

That one skips the wrapper object which is filled by the format properties by qemu automatically. The older instance of the function which was used for pre-blockdev usage actually formats the strings correctly. Luckily with blockdev usage the strings are interpreted by libvirt so VM usage will not be broken and additionally we can fix the generator.

Re-assigning to libvirt.

Comment 19 Peter Krempa 2020-03-24 15:11:13 UTC
Fixed upstream:

1b84dd190c storage: Parse 'nvme' disk source properties from json:{} pseudo-uri
3b06103e69 qemuBlockGetBackingStoreString: Properly handle 'http/s' with cookies and others
bafd2e94fa qemu: block: Extract formatting of cookie string
7ba2208add qemuBlockGetBackingStoreString: Add extra wrapping object to JSON strings
5a70f1048f storage: Implement backing store support for "fat:" prefix
c60fe3106e qemuBlockGetBackingStoreString: Remove 'ret' variable
5f3b4a3727 qemublocktest: Test backing store strings
93171b63c3 testQemuDiskXMLToProps: Store all per-image data in one structure
a83c1dc70e qemuBlockGetBackingStoreString: Add 'pretty' argument
fed97cb435 testQemuDiskXMLToPropsValidateFileSrcOnly: Move together with rest of xml->json code
4a9f355535 qemublocktest: xml->json: Refactor cleanup in test case functions
fac9a8b4c4 virDomainDiskSourceNVMeFormat: Format only valid 'managed' values
2337dbfdd1 qemublocktest: xml->json: Add test for NVMe
5793b8baa7 conf: rename 'namespace' property of struct _virStorageSourceNVMeDef

Specifically by:

7ba2208add qemuBlockGetBackingStoreString: Add extra wrapping object to JSON strings

The rest of the patches refactors/adds new tests and few others fix some issues found along.

Comment 24 Han Han 2020-04-09 01:59:43 UTC
Verified at libvirt-6.0.0-17.module+el8.2.0+6257+0d066c28.x86_64 qemu-kvm-4.2.0-17.module+el8.2.0+6141+0f540f16.x86_64
1. Start an VM with nvme disk:
<disk type="nvme" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source type="pci" managed="yes" namespace="1" index="1">
        <address domain="0x0000" bus="0x44" slot="0x00" function="0x0"/>
        <slices>
          <slice type="storage" offset="0" size="10737418240"/>
        </slices>
      </source>
      <target dev="sda" bus="sata"/>
      <alias name="sata1-0-0"/>
      <address type="drive" controller="1" bus="0" target="0" unit="0"/>
    </disk>

2. Create external snapshot:
# virsh snapshot-create-as nvme s1 --disk-only --no-metadata --diskspec sda,file=/tmp/nvme.ss                 
Domain snapshot s1 created

3. Destroy VM and attach nvme to vfio
➜  ~ virsh destroy nvme                                                                         
Domain nvme destroyed

➜  ~ ./vfio_adm --attach --device 0000:44:00.0                                                  
Device is already VFIO/UIO bound

Check the snapshot backing chain:
➜  ~ qemu-img info /tmp/nvme.ss -U --backing-chain
image: /tmp/nvme.ss
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 768 KiB
cluster_size: 65536
backing file: json:{"file":{"driver":"raw","offset":0,"size":10737418240,"file":{"driver":"nvme","device":"0000:44:00.0","namespace":1}}}
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

image: json:{"driver": "qcow2", "file": {"offset": 0, "driver": "raw", "size": 10737418240, "file": {"device": "0000:44:00.0", "driver": "nvme", "namespace": 1}}}
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: unavailable
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false


For the test coverage, I think it is better to cover it in the block xml2json test.

Comment 25 Han Han 2020-04-22 01:58:05 UTC
Covered by unit test: qemublocktest

Comment 27 errata-xmlrpc 2020-05-05 09:57:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2017