Bug 1375408 - [TestOnly] libvirt json backing support for ssh/ftp/ftps/sheepdog/glusterfs(unix socket)
Summary: [TestOnly] libvirt json backing support for ssh/ftp/ftps/sheepdog/glusterfs(u...
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.3
Hardware: x86_64
OS: Linux
Target Milestone: rc
: ---
Assignee: Peter Krempa
QA Contact: Han Han
Depends On: 1372143 1373710 1373786 1457088
TreeView+ depends on / blocked
Reported: 2016-09-13 03:26 UTC by Han Han
Modified: 2018-04-10 11:12 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2018-04-10 11:12:16 UTC

Attachments (Terms of Use)

Description Han Han 2016-09-13 03:26:03 UTC
Description of problem:
From the patches of BZ1134878, libvirt adds json backing images support. However, due to some known issues in libvirt/qemu-img/qemu-kvm, we are unable to test if ssh/ftp/ftps/tftp/sheepdog/glusterfs(unix socket) 
works well. Pls test them by attach/detach/pool-refresh after the depended bugs fixed, just like what we did in https://bugzilla.redhat.com/show_bug.cgi?id=1134878#c29 .

Comment 1 Xuesong Zhang 2017-06-16 06:54:49 UTC
Move this BZ to 7.5, due to the depend on BZ1140166 is moved to next release.

Comment 2 Han Han 2018-01-05 05:49:28 UTC
Failed QA since the depend on BZ1140166 is moved to next release.

Comment 3 Peter Krempa 2018-01-05 09:01:36 UTC
The JSON part of the SSH protocol was already implemented:

commit 29f06ff548e8910ba48c207591788b32c3544e40
Author: Peter Krempa <pkrempa@redhat.com>
Date:   Tue Sep 9 17:56:04 2014 +0200

    util: storage: Add 'ssh' network storage protocol
    Allow using 'ssh' protocol in backing chains and later for disks

And with the JSON syntax since commit 

commit bc225b1b5fe3e02eac76f12792aac16d00a7de1d
Author: Peter Krempa <pkrempa@redhat.com>
Date:   Fri Jul 15 15:12:10 2016 +0200

    util: storage: Add JSON backing volume parser for 'ssh' protocol

gluster with unix socket transport works since:

commit 2ed772cd635bfc0fa332c1122aea3f489b879b1a
Author: Peter Krempa <pkrempa@redhat.com>
Date:   Wed Jul 13 15:35:29 2016 +0200

    util: storage: Add json pseudo protocol support for gluster volumes
    Along with the legacy URI based syntax add support for the brand-new
    fully object based syntax.

And (t)ftp(s)/http(s):

commit ba05b5b7e78476497fb2222f449b8cad2738acb1
Author: Peter Krempa <pkrempa@redhat.com>
Date:   Tue Jul 12 17:59:58 2016 +0200

    util: storage: Add support for URI based backing volumes in qemu's JSON pseudo-protocol
    http(s), ftp(s) and tftp use URIs for volume definitions in the JSON
    pseudo protocol so it's pretty straightforward to add support for them.

Removing the dependency on BZ1140166 and moving back.

Comment 4 Han Han 2018-01-25 07:57:08 UTC
tftp has been dropped from http://lists.nongnu.org/archive/html/qemu-block/2016-11/msg00054.html
So I will not test tftp json backing.

Comment 5 Han Han 2018-01-30 08:54:21 UTC
I will not test attach/detach. Because ftp/ftps/sheepdog are not support by qemu-kvm-rhev and ssh is not supported due to BZ1140166. gluster unix hot-plug fails due to the permission of gluster.socket.
Verify on libvirt-3.9.0-9.el7.x86_64 qemu-kvm-rhev-2.10.0-18.el7.x86_64
1. Verify ftp protocol:
Prepare a ftp server with pyftpdlib and create a image on ftp server
Create backing image:
# qemu-img create --object secret,id=sec,data=$(echo -n "12345"|base64),format=base64 -b 'json:{"file.driver":"ftp", "file.url":"","file.username":"user", "file.password-secret":"sec"}' /var/lib/libvirt/images/ftp.qcow2 -f qcow2 
Formatting '/var/lib/libvirt/images/ftp.qcow2', fmt=qcow2 size=104857600 backing_file=json:{"file.driver":"ftp",, "file.url":"",,"file.username":"user",, "file.password-secret":"sec"} cluster_size=65536 lazy_refcounts=off refcount_bits=16

Refresh the pool:
# virsh pool-dumpxml default 
<pool type='dir'>
  <capacity unit='bytes'>150297640960</capacity>
  <allocation unit='bytes'>109206401024</allocation>
  <available unit='bytes'>41091239936</available>

# virsh pool-refresh default
Pool default refreshed

ftp protocol works well.

2. Verify sheepdog protocol:
i. Prepare a sheepdog env from the docker of sheepdog
ii. Create a sheepdog image and refresh the pool
# qemu-img create sheepdog: 100M
Formatting 'sheepdog:', fmt=raw size=104857600

# virsh pool-refresh default  
Pool default refreshed

3. Verify gluster(unix) socket
i. Prepare a gluster server on the local host
ii. Create the glutser unix json backing file and refresh the pool
# qemu-img create -b 'json:{"file":{"driver":"gluster","server":[{"type":"unix", "socket":"/var/run/glusterd.socket"}],"volume":"gv0","path":"a"}}' -o backing_fmt=raw /var/lib/libvirt/images/gluster_unix.qcow2 -f qcow2
Formatting '/var/lib/libvirt/images/gluster_unix.qcow2', fmt=qcow2 size=104857600 backing_file=json:{"file":{"driver":"gluster",,"server":[{"type":"unix",, "socket":"/var/run/glusterd.socket"}],,"volume":"gv0",,"path":"a"}} backing_fmt=raw cluster_size=65536 lazy_refcounts=off refcount_bits=16

# virsh pool-refresh default
Pool default refreshed

Comment 6 Han Han 2018-01-30 10:30:02 UTC
Verify ftps protocal
1. Prepare a ftps server accourding to https://www.digitalocean.com/community/tutorials/how-to-configure-vsftpd-to-use-ssl-tls-on-a-centos-vps with implicit_ssl yes. Then upload a image to the ftps server.

2. Create the json backing file:
# qemu-img create --object secret,id=sec,data=$(echo -n "12345"|base64),format=base64 -b 'json:{"file.driver":"ftps", "file.url":"ftps://","file.username":"ftpuser", "file.password-secret":"sec", "file.sslverify":"off"}' -o backing_fmt=raw /var/lib/libvirt/images/ftps.qcow2 -f qcow2
Formatting '/var/lib/libvirt/images/ftps.qcow2', fmt=qcow2 size=27832320 backing_file=json:{"file.driver":"ftps",, "file.url":"ftps://",,"file.username":"ftpuser",, "file.password-secret":"sec",, "file.sslverify":"off"} backing_fmt=raw cluster_size=65536 lazy_refcounts=off refcount_bits=16

Refresh the pool
# virsh pool-refresh default
Pool default refreshed

Comment 7 Han Han 2018-01-31 04:41:06 UTC
Verified ssh on openssh-clients-7.4p1-16.el7.x86_64:
1. Prepare ssh env
# eval `ssh-agent`
Agent pid 19057

# ssh-add ~/.ssh/id_rsa
Enter passphrase for /root/.ssh/id_rsa: 
Identity added: /root/.ssh/id_rsa (/root/.ssh/id_rsa)

2. Create ssh backing image and refresh the pool
# qemu-img info --image-opts driver=raw,file.driver=ssh,file.host=xx.xx.xx.xx,file.path=/tmp/a,file.host_key_check=no,file.user=hhan
image: json:{"driver": "raw", "file": {"host": "xx.xx.xx.xx", "host_key_check": "no", "driver": "ssh", "path": "/tmp/a", "user": "hhan"}}
file format: raw
virtual size: 100M (104857600 bytes)
disk size: unavailable

# virsh pool-refresh default
Pool default refreshed

Note You need to log in before you can comment on or make changes to this bug.