Bug 1466177
| Summary: | libvirt cannot parse back file json format of multi-servers with index | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux Advanced Virtualization | Reporter: | Han Han <hhan> |
| Component: | libvirt | Assignee: | Peter Krempa <pkrempa> |
| Status: | CLOSED ERRATA | QA Contact: | Meina Li <meili> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 8.0 | CC: | chhu, dyuan, hhan, jdenemar, lmen, xuzhang, yanqzhan, yisun |
| Target Milestone: | rc | Keywords: | Triaged |
| Target Release: | 8.1 | Flags: | pm-rhel:
mirror+
|
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-6.2.0-1.el8 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-11-17 17:44:45 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
For the impact, it will block blockcommit from libvirt generated external snapshot to gluster connection with multi-servers:
Reproduce it on libvirt-2.0.0-10.el7_3.9.x86_64:
1. Start a VM with gluster multi-server
# virsh dumpxml gls| awk '/<disk/,/<\/disk/'
<disk type='network' device='disk' snapshot='no'>
<driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>
<source protocol='gluster' name='vol1/raw'>
<host name='A.A.A.A'/>
<host name='B.B.B.B'/>
<host name='C.C.C.C'/>
</source>
<backingStore/>
<target dev='sda' bus='scsi'/>
<serial>8a96ac00-96fc-452d-ae42-eb2a8bd27c43</serial>
<alias name='scsi0-0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
2. Create a external snapshot by libvirt:
# virsh snapshot-create-as gls s1 --diskspec sda,file=/var/lib/libvirt/images/gls.s1 --disk-only --no-metadata
Domain snapshot s1 created
3. Do blockcommit
# virsh blockcommit gls sda --active --pivot
error: invalid argument: missing 'volume', 'path' or 'server' attribute in JSON backing definition for gluster volume
The qemu-img info:
# qemu-img info /var/lib/libvirt/images/gls.s1
image: /var/lib/libvirt/images/gls.s1
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 772K
cluster_size: 65536
backing file: json:{"driver": "raw", "file": {"server.0.host": "A.A.A.A", "server.1.host": "B.B.B.B", "server.2.host": "C.C.C.C", "driver": "gluster", "path": "/raw", "server.0.type": "tcp", "server.1.type": "tcp", "server.2.type": "tcp", "server.0.port": "24007", "server.1.port": "24007", "server.2.port": "24007", "volume": "vol1"}}
backing file format: raw
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
So I suggest to fix it in 7.3.z. If have no time to fix it on 7.4. Please write it as known issue in release note.
This bug is going to be addressed in next major release. Fixed upstream:
commit 096e94cef6edf453bb6196d05e9eedbe16771ba9
Author: Peter Krempa <pkrempa>
Date: Thu Dec 14 18:01:05 2017 +0100
tests: virstoragetest: validate that array deflattening works for gluster
Validate that we are able to parse back the dotted syntax arrays we were
generating in the pre-blockdev era.
Signed-off-by: Peter Krempa <pkrempa>
Reviewed-by: Michal Privoznik <mprivozn>
commit cdf82b60fd6db5e959cd4cfb0a742dcdd53d8bfe
Author: Peter Krempa <pkrempa>
Date: Wed Mar 18 17:02:42 2020 +0100
jsontest: Add test cases for deflattening of arrays
Signed-off-by: Peter Krempa <pkrempa>
Reviewed-by: Michal Privoznik <mprivozn>
commit 070310a680b0ea16af98efb592a1f4771022e791
Author: Peter Krempa <pkrempa>
Date: Thu Dec 14 18:03:04 2017 +0100
virjson: Deflatten arrays generated by the json->commandline generator
For the few instances where we'd generate an array in dotted syntax we
should be able to parse it back. Add another step in deflattening of the
dotted syntax which reconstructs the arrays so that the backing store
parser can parse it.
https://bugzilla.redhat.com/show_bug.cgi?id=1466177
Signed-off-by: Peter Krempa <pkrempa>
Reviewed-by: Michal Privoznik <mprivozn>
commit df304575878e0d5b8b27c73d554beec5882ef6b8
Author: Peter Krempa <pkrempa>
Date: Thu Mar 19 18:11:48 2020 +0100
util: json: Extract deflattening of keys into a separate function
Extract the code so that there's a clean separation once we'll want do
do other steps.
Signed-off-by: Peter Krempa <pkrempa>
Reviewed-by: Michal Privoznik <mprivozn>
commit fca6f593c9408c53d7fc52c483069ccba1cd7215
Author: Peter Krempa <pkrempa>
Date: Wed Mar 18 16:38:16 2020 +0100
virJSONValueObjectDeflattenWorker: Refactor cleanup
Use automatic memory handling to remove the cleanup section.
Signed-off-by: Peter Krempa <pkrempa>
Reviewed-by: Michal Privoznik <mprivozn>
commit 3ceb6951bdb11f303f3d9f62fcdf363306fcd873
Author: Peter Krempa <pkrempa>
Date: Wed Mar 18 15:51:14 2020 +0100
virBitmapNewEmpty: Use g_new0 to allocate and remove error checking
virBitmapNewEmpty can't fail now so we can make it obvious and fix all
callers.
Signed-off-by: Peter Krempa <pkrempa>
Reviewed-by: Michal Privoznik <mprivozn>
Verified Version:
libvirt-6.5.0-1.module+el8.3.0+7323+d54bb644.x86_64
qemu-kvm-5.0.0-2.module+el8.3.0+7379+0505d6ca.x86_64
Verified Steps:
SC1: Do blockcommit for gluster network disk with multi-servers
1. Start a VM with gluster multi-server
# virsh dumpxml lmn | awk '/<disk/,/<\/disk/'
<disk type='network' device='disk' snapshot='no'>
<driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>
<source protocol='gluster' name='gv0/test.img' index='1'>
<host name='10.0.137.154' port='24007'/>
<host name='10.0.139.68' port='24007'/>
<host name='10.0.139.5' port='24007'/>
</source>
<backingStore/>
<target dev='sda' bus='scsi'/>
<serial>8a96ac00-96fc-452d-ae42-eb2a8bd27c43</serial>
<alias name='scsi0-0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
2. Create a external snapshot.
# virsh snapshot-create-as lmn s1 --diskspec sda,file=/var/lib/libvirt/images/gfs.s1 --disk-only
Domain snapshot s1 created
3. Do blockcommit.
# virsh blockcommit lmn sda --active --wait --pivot
Successfully pivoted
SC2: Do pool-refresh after creating gluster image with multi-server in dir pool
1. Create a back file json format of multi-servers image to default dir pool.
# qemu-img create -f qcow2 -b 'json:{"file": {"server.0":{"host": "10.0.137.154", "type": "tcp", "port":"24007"}, "server.1":{"host": "10.0.139.68", "type": "tcp", "port":"24007"}, "server.2":{"host": "10.0.139.5", "type": "tcp","port":"24007"}, "driver": "gluster", "path": "/test.img", "volume": "gv0"}}' -o backing_fmt=raw /var/lib/libvirt/images/gls.s2
Formatting '/var/lib/libvirt/images/gls.s2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 backing_file=json:{"file": {"server.0":{"host": "10.0.137.154",, "type": "tcp",, "port":"24007"},, "server.1":{"host": "10.0.139.68",, "type": "tcp",, "port":"24007"},, "server.2":{"host": "10.0.139.5",, "type": "tcp",,"port":"24007"},, "driver": "gluster",, "path": "/test.img",, "volume": "gv0"}} backing_fmt=raw lazy_refcounts=off refcount_bits=16
2. Do pool-refresh the dir pool.
# virsh pool-refresh default
Pool default refreshed
# virsh vol-list default
Name Path
------------------------------------------------
gls.s2 /var/lib/libvirt/images/gls.s2
lmn.qcow2 /var/lib/libvirt/images/lmn.qcow2
SC1: Do blockcommit for rbd network disk with multi-servers
1. Start a VM with rbd multi-server
# virsh dumpxml lmn | awk '/<disk/,/<\/disk/'
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>
<source protocol='rbd' name='libvirt-pool/test.img' index='1'>
<host name='10.66.146.31'/>
<host name='10.66.145.47'/>
<host name='10.66.146.115'/>
<auth username='admin'>
<secret type='ceph' usage='libvirtceph'/>
</auth>
</source>
<backingStore/>
<target dev='sda' bus='scsi'/>
<serial>8a96ac00-96fc-452d-ae42-eb2a8bd27c43</serial>
<alias name='scsi0-0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
2. Create a external snapshot.
# virsh snapshot-create-as lmn s1 --diskspec sda,file=/var/lib/libvirt/images/rbd.s1 --disk-only --no-metadata
Domain snapshot s1 created
3. Do blockcommit.
# virsh blockcommit lmn sda --active --wait --pivot
Successfully pivoted
Move this bug to be verified according to the test result.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5137 |
Description of problem: As subject Version-Release number of selected component (if applicable): libvirt-3.2.0-14..el7.x86_64 How reproducible: 100% Steps to Reproduce: 1. Create a back file json format of multi-servers with index to dir pool # virsh pool-dumpxml default <pool type='dir'> <name>default</name> <uuid>8b10594a-bd07-4cd4-b237-2621735f00e5</uuid> <capacity unit='bytes'>535632543744</capacity> <allocation unit='bytes'>62641999872</allocation> <available unit='bytes'>472990543872</available> <source> </source> <target> <path>/var/lib/libvirt/images</path> <permissions> <mode>0711</mode> <owner>0</owner> <group>0</group> <label>system_u:object_r:virt_image_t:s0</label> </permissions> </target> </pool> # qemu-img create -f qcow2 -b 'json:{"file": {"server.0":{"host": "B.B.B.B", "type": "tcp", "port":"24007"}, "server.1":{"host": "A.A.A.A", "type": "tcp", "port":"24007"}, "server.2":{"host": "C.C.C.C", "type": "tcp","port":"24007"}, "driver": "gluster", "path": "/raw", "volume": "vol1"}}' -o backing_fmt=raw /var/lib/libvirt/images/gls.s3 Formatting '/var/lib/libvirt/images/gls.s3', fmt=qcow2 size=10737418240 backing_file=json:{"file": {"server.0":{"host": "B.B.B.B",, "type": "tcp",, "port":"24007"},, "server.1":{"host": "A.A.A.A",, "type": "tcp",, "port":"24007"},, "server.2":{"host": "C.C.C.C",, "type": "tcp",,"port":"24007"},, "driver": "gluster",, "path": "/raw",, "volume": "vol1"}} backing_fmt=raw encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 2. Do pool-refresh # virsh pool-refresh default error: Failed to refresh pool default error: invalid argument: missing 'volume', 'path' or 'server' attribute in JSON backing definition for gluster volume Actual results: As above Expected results: Libvirt could parse the back file json format of multi-servers with index Additional info: Rbd may be also affected because it has similar json structure with gluster. You should be aware of BZ1464821 in case server key with index with unflattened "server" structure cannot be parsed. It's better to fix them together. Fix all the issues of protocols using 'server' in json meta-data. It can also be reproduced on RHEL7.3