Bug 1373786

The bug will block some test scenarios for BZ1134878

This worked for me when running QEMU by itself.  I was able to reproduce the exact bug description when using libvirt / virsh, however.

The issue is default permissions of the gluster socket in /var/run.  In the example in the description, "/var/run/glusterd.socket" should be owned by qemu, not by root:

My test image:

qemu-img info --backing-chain /home/jcody/images/gluster_unix.img
image: /home/jcody/images/gluster_unix.img
file format: qcow2
virtual size: 100M (104857600 bytes)
disk size: 196K
cluster_size: 65536
backing file: json:{
    "file.driver": "gluster",
    "file.path": "test.qcow2",
    "file.server": [
        {
            "type": "unix",
            "socket": "/var/run/glusterd.socket"
        }
    ],
    "file.volume": "gv0"
}
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

image: json:{"driver": "qcow2", "file": {"driver": "gluster", "path": "test.qcow2", "server.0.type": "unix", "server.0.socket": "/var/run/glusterd.socket", "volume": "gv0"}}
file format: qcow2
virtual size: 100M (104857600 bytes)
disk size: 193K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false




Before attending to the permissions of the socket:

# virsh attach-disk c /home/jcody/images/gluster_unix.img vdc --subdriver qcow2
error: Failed to attach disk
error: internal error: unable to execute QEMU command '__com.redhat_drive_add': Device 'drive-virtio-disk2' could not be initialized



Changing owner of the gluster socket, and retrying:

# chown qemu.qemu /var/run/glusterd.socket 

# virsh attach-disk c /home/jcody/images/gluster_unix.img vdc --subdriver qcow2
Disk attached successfully


I believe this is the root of the error, so I will close this as NOTABUG, as it is likely due to steps in the testing scenario.

However, first I'd like to make sure that libvirt (if it currently supports gluster unix sockets) sets the owner / permissions correctly on the socket in this scenario. Eric, do you know if this is the case?

I think libvirt intends to support unix sockets for gluster (at least this comment in storage_backend_gluster.c is telling:
     /* Accept the same URIs as qemu's block/gluster.c:
      * gluster[+transport]://[server[:port]]/vol/[dir/]image[?socket=...] */
), so it's probably best to reassign this to libvirt for evaluation.  Either libvirt is not properly managing the socket permissions, or it is indeed a test setup bug, but it needs evaluation from the libvirt point of view.

(In reply to Eric Blake from comment #6)
> I think libvirt intends to support unix sockets for gluster (at least this
> comment in storage_backend_gluster.c is telling:
>      /* Accept the same URIs as qemu's block/gluster.c:
>       * gluster[+transport]://[server[:port]]/vol/[dir/]image[?socket=...] */
> ), so it's probably best to reassign this to libvirt for evaluation.  Either
> libvirt is not properly managing the socket permissions, or it is indeed a
> test setup bug, but it needs evaluation from the libvirt point of view.

Thanks Eric.  Re-assigning to libvirt.

We can't just chown the socket to qemu:qemu, since it would possibly break other configurations. I think it should be left as is and the admin has to set up permissions correctly.

The only sane solution I can see here is to use FD passing if that's possible for gluster connections.

The qemu gluster driver passes the socket path directly to glfs_set_volfile_server so it's not possible to use qemu's fd passing mechanism.

Since libvirt can't modify the socket permissions as the socket is shared, the user needs to set up the access to the gluster socket properly.

Hi Peter,
Since glusterd socket and nbd socket cannot be used by libvirt directly, how abourt disabling them?

It can be used directly if you properly configure permissions for the unix socket. Since it's a shared resource, libvirt can't modify them on the VMs behalf as it would break other clients connecting.