Bug 1247521

libvirt-daemon-1.2.8-16.el7_1.3.x86_64 is not an upstream release, moving to the downstream tracker.

We are lacking qemu support. I've filed 1247933 to track the qemu addition.

Patches Send to QEMU: (Under Review)

[Qemu-devel] [PATCH 1/1] block/gluster: add support for multiple gluster
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02016.html

[Qemu-devel] [PATCH v2 1/1] block/gluster: add support for multiple glus
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02437.html

Just send the patch to libvirt:
https://www.redhat.com/archives/libvir-list/2015-October/msg00106.html

Progress upstream?

Still waiting on qemu upstream to deal with some design issues:

http://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg01328.html

Can someone explain, please - why multiple volumes support became blocker for libgfapi calls, while it wasn't blocker while single fuse mountpoint usage was not considered blocker?

Is it because it was supposed, that some network failover solution is supposed to be used in the middle between fuse mountpoint and qemu process? But such network failover was always a workaround, isn't it? 

Why current blocker could not be workarounded same way?
If "localhost:volume" is most natural way of configuration, case, when gluster dies on localhost leaving all other processes intact is quite rare, I believe.

So, feature could be already released, with a caution.
Or am I wrong?

Wsorry for mistakes and lexical duplicates :/

(In reply to Andrejs Baulins from comment #10)

From a Gluster point of view, you are not wrong. I do not see a strong requirement to have multiple hosts listed in the libvirt xml. Once one server (could be "localhost" running GlusterD) is connected with qemu/libgfapi, the Gluster client (qemu+gfapi) is aware of all the bricks participating in the volume, and a connection is made to all of those bricks. The host in the libvirt xml is only used for the initial retrieving of the layout of the Gluster volume.

(In reply to Niels de Vos from comment #12)
> (In reply to Andrejs Baulins from comment #10)
> 
> From a Gluster point of view, you are not wrong. I do not see a strong
> requirement to have multiple hosts listed in the libvirt xml. Once one
> server (could be "localhost" running GlusterD) is connected with
> qemu/libgfapi, the Gluster client (qemu+gfapi) is aware of all the bricks
> participating in the volume, and a connection is made to all of those
> bricks. The host in the libvirt xml is only used for the initial retrieving
> of the layout of the Gluster volume.

I', also waiting really for a way to use the native qmui+gfapi way of mounting my VM's. 
I did a long way and switched from OpenNebula (which supports native gluster mounts!) to Ovirt to finally see my VM XML's to use it the fuse way. 
I'm really disappointed and would also use a workaround to be able to specify "Yes I want gfapi". Since I started gluster I used round-robin a DNS entry for my gluster volume servers includincg all "entry points" to my gluster I would be ok to specify only _one_ gluster host.

Now OVirt 4.0 is out and still nothing new ?

There are 2 scenarios I feel confused.

version:
1)test host:
libvirt-2.0.0-6.el7.x86_64
qemu-kvm-rhev-2.6.0-22.el7.x86_64

2)glusterfs server :
glusterfs-server-3.7.9-12.el7rhgs.x86_64


scenario1:
1.prepare  glusterfs environment in glusterfs servers：
[root@localhost ~]# gluster volume status
Status of volume: test
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 10.66.4.213:/opt/test1                49154     0          Y       2545 
Brick 10.66.4.105:/opt/test2                49152     0          Y       24386
Brick 10.66.4.148:/opt/test3                49153     0          Y       20994

2.in the test host(10.66.70.107)
[root@localhost ~]# qemu-img create -f qcow2 gluster://10.66.4.213/test/test1.img 200M

3.start a guest with xml:
<driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='test/test1.img'>
        <host name='10.66.4.2'/>  --->not exists
        <host name='10.66.4.105'/>    --->one of the above glusterfs servers
        <host name='10.66.4.148'/>    --->one of the above glusterfs servers
      </source>
      <target dev='vda' bus='virtio'/>
      <boot order='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </disk>

[root@localhost ~]# virsh start bios
error: Failed to start domain bios
error: failed to initialize gluster connection (src=0x7f1be48b7130 priv=0x7f1bec254f10): Transport endpoint is not connected

My question:
if the first host can't be found,the second host will not be tried? can we add the function?



scenario2:
1.shutdown the first glusterfs server(10.66.4.213)
2.start the guest with xml:
<driver name='qemu' type='qcow2' cache='none'/>
      <source protocol='gluster' name='test/test1.img'>
  ***      <host name='10.66.4.105'/>    
        <host name='10.66.4.148'/>  ***  They are other two glusterfs server available
      </source>
      <target dev='vda' bus='virtio'/>
      <boot order='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </disk>

[root@localhost ~]# virsh start bios      --->guest can be started,but very slow,about 6 mins.

3.destroy the guest
[root@localhost ~]# virsh destroy bios
error: Disconnected from qemu:///system due to keepalive timeout
error: Failed to destroy domain bios
error: internal error: connection closed due to keepalive timeout      -->is the error expected?

Additional info:
I have also tried the command in the test host:
[root@localhost ~]# mount -t glusterfs 10.66.4.105:/test /var/lib/libvirt/images/
It is also very slow.

Both of those originate from the glusterfs library since libvirt just initializes it. There's a discussion on the QEMU bug relevant to this work regarding missing servers and the test cases connected to it.

http://git.qemu.org/?p=qemu.git;a=commit;h=6c7189bb29de9fa2202f613f3c6caf028f96f261 is included in QEMU v2.7.0.

http://libvirt.org/git/?p=libvirt.git;a=commit;h=f1bbc7df4a9959e09679486c769e12f82d443d9a is included in libvirt v2.1.0

It seems that the test-host with these versions is too old:
 - libvirt-2.0.0-6.el7.x86_64
 - qemu-kvm-rhev-2.6.0-22.el7.x86_64

libvirt backported that from the upstream v2.1.0 version to libvirt-2.0.0-4.el7 thus -6 version is good enough.

(In reply to Peter Krempa from comment #20)
> libvirt backported that from the upstream v2.1.0 version to
> libvirt-2.0.0-4.el7 thus -6 version is good enough.

Yes, the missing part seems to be the patch(es) for QEMU.

hi ,Peter,

for scenario 1:
I have confirmed with qemu qe,if the first glusterfs server IP specified in vm does not exist(or can't be connected),and the second and third IP are right,the guest can boot up successfully using qemu cli.

the cli is:
...-drive file.driver=gluster,file.volume=test,file.path=/test.img,file.server.0.type=tcp,file.server.0.host=10.66.4.1,file.server.0.port=24007,file.server.1.type=tcp,file.server.1.host=10.66.4.105,file.server.1.port=24007,format=raw,if=none,id=drive-virtio-disk1,cache=none .....

I think there is no difference from what libvirt used.

But in libvirt,the vm can't boot up,as scenario1 in comment 16.

qemu qe and I use the same glusterfs servers.

Is the phenomenon in libvirt normal? Do we need to improve it?

for scenario2:
It is my fault. My test host maybe have some issue. I have used other test host,I can't reproduce the scenario again. Sorry.

the scenario1 in comment16 will be tracked in bug 1377663.
change the bug status as verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2577.html