Bug 1466177 - libvirt cannot parse back file json format of multi-servers with index
Summary: libvirt cannot parse back file json format of multi-servers with index
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.1
Assignee: Peter Krempa
QA Contact: Meina Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-29 08:23 UTC by Han Han
Modified: 2020-11-17 17:45 UTC (History)
8 users (show)

Fixed In Version: libvirt-6.2.0-1.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-11-17 17:44:45 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Han Han 2017-06-29 08:23:24 UTC
Description of problem:
As subject

Version-Release number of selected component (if applicable):
libvirt-3.2.0-14..el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create a back file json format of multi-servers with index to dir pool
# virsh pool-dumpxml default
<pool type='dir'>
  <name>default</name>
  <uuid>8b10594a-bd07-4cd4-b237-2621735f00e5</uuid>
  <capacity unit='bytes'>535632543744</capacity>
  <allocation unit='bytes'>62641999872</allocation>
  <available unit='bytes'>472990543872</available>
  <source>
  </source>
  <target>
    <path>/var/lib/libvirt/images</path>
    <permissions>
      <mode>0711</mode>
      <owner>0</owner>
      <group>0</group>
      <label>system_u:object_r:virt_image_t:s0</label>
    </permissions>
  </target>
</pool>

# qemu-img create -f qcow2 -b 'json:{"file": {"server.0":{"host": "B.B.B.B", "type": "tcp", "port":"24007"}, "server.1":{"host": "A.A.A.A", "type": "tcp", "port":"24007"}, "server.2":{"host": "C.C.C.C", "type": "tcp","port":"24007"}, "driver": "gluster", "path": "/raw", "volume": "vol1"}}' -o backing_fmt=raw /var/lib/libvirt/images/gls.s3
Formatting '/var/lib/libvirt/images/gls.s3', fmt=qcow2 size=10737418240 backing_file=json:{"file": {"server.0":{"host": "B.B.B.B",, "type": "tcp",, "port":"24007"},, "server.1":{"host": "A.A.A.A",, "type": "tcp",, "port":"24007"},, "server.2":{"host": "C.C.C.C",, "type": "tcp",,"port":"24007"},, "driver": "gluster",, "path": "/raw",, "volume": "vol1"}} backing_fmt=raw encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16

2. Do pool-refresh
# virsh pool-refresh default
error: Failed to refresh pool default
error: invalid argument: missing 'volume', 'path' or 'server' attribute in JSON backing definition for gluster volume

Actual results:
As above

Expected results:
Libvirt could parse the back file json format of multi-servers with index


Additional info:
Rbd may be also affected because it has similar json structure with gluster.
You should be aware of BZ1464821 in case server key with index with unflattened "server" structure cannot be parsed. It's better to fix them together. Fix all the issues of protocols using 'server' in json meta-data.

It can also be reproduced on RHEL7.3

Comment 4 Han Han 2017-06-30 00:33:38 UTC
For the impact, it will block blockcommit from libvirt generated external snapshot to gluster connection with multi-servers:
Reproduce it on libvirt-2.0.0-10.el7_3.9.x86_64:
1. Start a VM with gluster multi-server
# virsh dumpxml gls| awk '/<disk/,/<\/disk/'
    <disk type='network' device='disk' snapshot='no'>
      <driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>
      <source protocol='gluster' name='vol1/raw'>
        <host name='A.A.A.A'/>
        <host name='B.B.B.B'/>
        <host name='C.C.C.C'/>
      </source>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <serial>8a96ac00-96fc-452d-ae42-eb2a8bd27c43</serial>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>

2. Create a external snapshot by libvirt:
# virsh snapshot-create-as gls s1 --diskspec sda,file=/var/lib/libvirt/images/gls.s1 --disk-only --no-metadata
Domain snapshot s1 created


3. Do blockcommit
# virsh blockcommit gls sda --active --pivot                                                                  
error: invalid argument: missing 'volume', 'path' or 'server' attribute in JSON backing definition for gluster volume

The qemu-img info:
# qemu-img info /var/lib/libvirt/images/gls.s1 
image: /var/lib/libvirt/images/gls.s1
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 772K
cluster_size: 65536
backing file: json:{"driver": "raw", "file": {"server.0.host": "A.A.A.A", "server.1.host": "B.B.B.B", "server.2.host": "C.C.C.C", "driver": "gluster", "path": "/raw", "server.0.type": "tcp", "server.1.type": "tcp", "server.2.type": "tcp", "server.0.port": "24007", "server.1.port": "24007", "server.2.port": "24007", "volume": "vol1"}}
backing file format: raw
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

So I suggest to fix it in 7.3.z. If have no time to fix it on 7.4. Please write it as known issue in release note.

Comment 6 Jaroslav Suchanek 2019-04-24 12:26:45 UTC
This bug is going to be addressed in next major release.

Comment 7 Peter Krempa 2020-03-20 08:57:04 UTC
Fixed upstream:

commit 096e94cef6edf453bb6196d05e9eedbe16771ba9
Author: Peter Krempa <pkrempa>
Date:   Thu Dec 14 18:01:05 2017 +0100

    tests: virstoragetest: validate that array deflattening works for gluster
    
    Validate that we are able to parse back the dotted syntax arrays we were
    generating in the pre-blockdev era.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Michal Privoznik <mprivozn>

commit cdf82b60fd6db5e959cd4cfb0a742dcdd53d8bfe
Author: Peter Krempa <pkrempa>
Date:   Wed Mar 18 17:02:42 2020 +0100

    jsontest: Add test cases for deflattening of arrays
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Michal Privoznik <mprivozn>

commit 070310a680b0ea16af98efb592a1f4771022e791
Author: Peter Krempa <pkrempa>
Date:   Thu Dec 14 18:03:04 2017 +0100

    virjson: Deflatten arrays generated by the json->commandline generator
    
    For the few instances where we'd generate an array in dotted syntax we
    should be able to parse it back. Add another step in deflattening of the
    dotted syntax which reconstructs the arrays so that the backing store
    parser can parse it.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1466177
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Michal Privoznik <mprivozn>

commit df304575878e0d5b8b27c73d554beec5882ef6b8
Author: Peter Krempa <pkrempa>
Date:   Thu Mar 19 18:11:48 2020 +0100

    util: json: Extract deflattening of keys into a separate function
    
    Extract the code so that there's a clean separation once we'll want do
    do other steps.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Michal Privoznik <mprivozn>

commit fca6f593c9408c53d7fc52c483069ccba1cd7215
Author: Peter Krempa <pkrempa>
Date:   Wed Mar 18 16:38:16 2020 +0100

    virJSONValueObjectDeflattenWorker: Refactor cleanup
    
    Use automatic memory handling to remove the cleanup section.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Michal Privoznik <mprivozn>

commit 3ceb6951bdb11f303f3d9f62fcdf363306fcd873
Author: Peter Krempa <pkrempa>
Date:   Wed Mar 18 15:51:14 2020 +0100

    virBitmapNewEmpty: Use g_new0 to allocate and remove error checking
    
    virBitmapNewEmpty can't fail now so we can make it obvious and fix all
    callers.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Michal Privoznik <mprivozn>

Comment 10 Meina Li 2020-07-29 07:00:11 UTC
Verified Version:
libvirt-6.5.0-1.module+el8.3.0+7323+d54bb644.x86_64
qemu-kvm-5.0.0-2.module+el8.3.0+7379+0505d6ca.x86_64

Verified Steps:
SC1: Do blockcommit for gluster network disk with multi-servers
1. Start a VM with gluster multi-server
# virsh dumpxml lmn | awk '/<disk/,/<\/disk/'
<disk type='network' device='disk' snapshot='no'>
      <driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>
      <source protocol='gluster' name='gv0/test.img' index='1'>
        <host name='10.0.137.154' port='24007'/>
        <host name='10.0.139.68' port='24007'/>
        <host name='10.0.139.5' port='24007'/>
      </source>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <serial>8a96ac00-96fc-452d-ae42-eb2a8bd27c43</serial>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
2. Create a external snapshot.
# virsh snapshot-create-as lmn s1 --diskspec sda,file=/var/lib/libvirt/images/gfs.s1 --disk-only
Domain snapshot s1 created
3. Do blockcommit.
# virsh blockcommit lmn sda --active --wait --pivot
Successfully pivoted

SC2: Do pool-refresh after creating gluster image with multi-server in dir pool 
1. Create a back file json format of multi-servers image to default dir pool.
 # qemu-img create -f qcow2 -b 'json:{"file": {"server.0":{"host": "10.0.137.154", "type": "tcp", "port":"24007"}, "server.1":{"host": "10.0.139.68", "type": "tcp", "port":"24007"}, "server.2":{"host": "10.0.139.5", "type": "tcp","port":"24007"}, "driver": "gluster", "path": "/test.img", "volume": "gv0"}}' -o backing_fmt=raw /var/lib/libvirt/images/gls.s2
Formatting '/var/lib/libvirt/images/gls.s2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 backing_file=json:{"file": {"server.0":{"host": "10.0.137.154",, "type": "tcp",, "port":"24007"},, "server.1":{"host": "10.0.139.68",, "type": "tcp",, "port":"24007"},, "server.2":{"host": "10.0.139.5",, "type": "tcp",,"port":"24007"},, "driver": "gluster",, "path": "/test.img",, "volume": "gv0"}} backing_fmt=raw lazy_refcounts=off refcount_bits=16
2. Do pool-refresh the dir pool.
# virsh pool-refresh default
Pool default refreshed
# virsh vol-list default
 Name        Path
------------------------------------------------
 gls.s2      /var/lib/libvirt/images/gls.s2
 lmn.qcow2   /var/lib/libvirt/images/lmn.qcow2

SC1: Do blockcommit for rbd network disk with multi-servers
1. Start a VM with rbd multi-server
# virsh dumpxml lmn | awk '/<disk/,/<\/disk/'
<disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>
      <source protocol='rbd' name='libvirt-pool/test.img' index='1'>
        <host name='10.66.146.31'/>
        <host name='10.66.145.47'/>
        <host name='10.66.146.115'/>
        <auth username='admin'>
          <secret type='ceph' usage='libvirtceph'/>
        </auth>
      </source>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <serial>8a96ac00-96fc-452d-ae42-eb2a8bd27c43</serial>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
2. Create a external snapshot.
# virsh snapshot-create-as lmn s1 --diskspec sda,file=/var/lib/libvirt/images/rbd.s1 --disk-only --no-metadata
Domain snapshot s1 created
3. Do blockcommit.
# virsh blockcommit lmn sda --active --wait --pivot
Successfully pivoted

Move this bug to be verified according to the test result.

Comment 13 errata-xmlrpc 2020-11-17 17:44:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5137


Note You need to log in before you can comment on or make changes to this bug.