Bug 1052014

Summary: guest fail to start with permission denied error when with gluster volume
Product: Red Hat Enterprise Linux 6 Reporter: Shanzhi Yu <shyu>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.6CC: acathrow, bili, chhu, dyuan, mazhang, mzhan, scohen, shyu, vbellur, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1052114 (view as bug list) Environment:
Last Closed: 2014-04-04 21:26:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1052114    
Bug Blocks:    

Description Shanzhi Yu 2014-01-13 06:19:47 UTC
Description of problem:

guest fail to start with permission denied error when with gluster volume

Version-Release number of selected component (if applicable):

qemu-kvm-rhev-0.12.1.2-2.415.el6_5.3.x86_64
libvirt-0.10.2-29.el6_5.2.x86_64

How reproducible:
100%

Steps to Reproduce:
1.prepare gluster server and volume gluster-vol1

[gluster server]# gluster volume info

Volume Name: gluster-vol1
Type: Distribute
Volume ID: 32fe2d1d-7a2a-45d9-8a96-faab68253edd
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: 10.66.106.20:/mnt/gluster-volume1
Brick2: 10.66.106.22:/mnt/gluster-volume1
Options Reconfigured:
server.allow-insecure: on

2.create an image by qemu-img on gluster client

[gluster client~]#qemu-img create -f qcow2 gluster://gluster server/gluster-vol1/rhel6.4-qcow2.img 10G

[gluster client~]#qemu-img info gluster://gluster server/gluster-vol1/rhel6.4-qcow2.img 
image: gluster://gluster server/gluster-vol1/rhel6.4-qcow2.img
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 136K
cluster_size: 65536

3.define an guest with the disk created in step 2

[gluster client~]#virsh dumpxml rhel6.4-qcow2|grep disk -A 6

<disk type='network' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source protocol='gluster' name='gluster-vol1/rhel6.4-qcow2.img'>
        <host name='gluster server' port='24007'/>
      </source>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>

4.start the guest

[gluster client~]#virsh start rhel6.4-qcow2

error: Failed to start domain rhel6.4-qcow2
error: internal error Process exited while reading console log output: char device redirected to /dev/pts/3
qemu-kvm: -drive file=gluster+tcp://10.66.106.20:24007/gluster-vol1/rhel6.4-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2: could not open disk image gluster+tcp://10.66.106.20:24007/gluster-vol1/rhel6.4-qcow2.img: Permission denied

Actual results:


Expected results:

should succeed starting guest in step 4.


Additional info:

even though i set storage.owner-uid/gid on gluster server, it doesn't work.
error log:
2014-01-13 06:14:06.953+0000: 2622: error : qemuProcessWaitForMonitor:1801 : internal error process exited while connecting to monitor: char device redirected to /dev/pts/3
qemu-kvm: -drive file=gluster+tcp://10.66.106.20:24007/gluster-vol1/rhel6.4-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2: could not open disk image gluster+tcp://10.66.106.20:24007/gluster-vol1/rhel6.4-qcow2.img: Permission denied

Comment 2 Shanzhi Yu 2014-01-13 08:19:59 UTC
there are two workaround to make it work
1)login to glusterfs server and change owner and group of disk(created from gluster client,here is rhel6.4-qcow2.img) to qemu:qemu
2)login to gluster server and change permission of disk to 666(the default is 600)

Comment 3 Shanzhi Yu 2014-01-13 10:41:12 UTC
sorry for forgot third workaround 

3)configure user = "root" and group = "root" in /etc/libvirt/qemu.conf on gluster client, restart libvirt. guest will start without any error.

Comment 4 Vijay Bellur 2014-02-19 04:47:52 UTC
Can you please also attach the gluster server and client logs to this?

Comment 5 Peter Krempa 2014-02-19 13:02:42 UTC
The issue is that libvirt isn't chowning/chmoding the files before starting qemu that will use them. Gluster is performing okay in this case.

Comment 7 RHEL Program Management 2014-04-04 21:26:53 UTC
Development Management has reviewed and declined this request.
You may appeal this decision by reopening this request.