Bug 1244594
Summary: | Permission denied when writing files to mounted glusterfs volumes from pod | ||
---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Jianwei Hou <jhou> |
Component: | Storage | Assignee: | hchen |
Status: | CLOSED CURRENTRELEASE | QA Contact: | Jianwei Hou <jhou> |
Severity: | medium | Docs Contact: | |
Priority: | medium | ||
Version: | 3.0.0 | CC: | dmcphers, hchen, jhou, jialiu, jkrieger, libra-bugs, mliyazud, mturansk, swagiaal |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2015-11-23 14:26:27 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Jianwei Hou
2015-07-20 06:18:30 UTC
This problem also exits when writing mounted iscsi volume from pod(after setting selinux to permissive): bash-4.2$ ls /mnt/iscsi/ -ldZ drwxr-xr-x. root root system_u:object_r:unlabeled_t:s0 /mnt/iscsi/ bash-4.2$ ls /mnt/iscsi/ lost+found bash-4.2$ touch /mnt/iscsi/testfile touch: cannot touch '/mnt/iscsi/testfile': Permission denied Huamin, can you give this a look? Jianwei, this could be a selinux issue, see https://bugzilla.redhat.com/show_bug.cgi?id=1231936 @hchen, following bug 1231936, I did 'setsebool virt_sandbox_use_fusefs 1', the mount dir is read accessible, but write inaccessible Here is the SELinux context of the mount dir bash-4.3$ ls -Zd /mnt/gluster/ drwxr-xr-x. root root system_u:object_r:fusefs_t:s0 /mnt/gluster/ bash-4.3$ touch /mnt/gluster/flie touch: cannot touch 'flie': Permission denied Jianwei, thanks for the upodate. I think you might want to try svirt_sandbox_file_t instead of virt_sandbox_use_fusefs. See if you can try "chcon -R -t svirt_sandbox_file_t /path_to_your_gluster_mount_on_your_host" and see any difference. On node: [root@openshift-115 ~]# getsebool virt_sandbox_use_fusefs virt_sandbox_use_fusefs --> off On master: [root@openshift-114 tmp]# oc exec gluster -n jhou -it -- bash bash-4.2$ ls /mnt/gluster/ ls: cannot open directory /mnt/gluster/: Permission denied Turn virt_sandbox_use_fusefs on [root@openshift-115 ~]# setsebool virt_sandbox_use_fusefs 1 [root@openshift-115 ~]# getsebool virt_sandbox_use_fusefs virt_sandbox_use_fusefs --> on [root@openshift-114 tmp]# oc exec gluster -n jhou -it -- bash bash-4.2$ ls /mnt/gluster/ hello index.html test bash-4.2$ touch /mnt/gluster/t1 touch: cannot touch '/mnt/gluster/t1': Permission denied bash-4.2$ ls /mnt/gluster/ -Zd drwxr-xr-x. root root system_u:object_r:fusefs_t:s0 /mnt/gluster/ The result is same with comment 4, could read, couldn't write. chcon operation not supported bash-4.2$ chcon -R -t svirt_sandbox_file_t /mnt/gluster/ chcon: failed to change context of 'index.html' to 'system_u:object_r:svirt_sandbox_file_t:s0': Operation not supported chcon: failed to change context of 'hello' to 'system_u:object_r:svirt_sandbox_file_t:s0': Operation not supported chcon: failed to change context of 'test' to 'system_u:object_r:svirt_sandbox_file_t:s0': Operation not supported chcon: failed to change context of '/mnt/gluster/' to 'system_u:object_r:svirt_sandbox_file_t:s0': Operation not supported Also tried svirt_sandbox_file_t, it seems on the system this boolean is not defined. [root@openshift-115 ~]# getsebool svirt_sandbox_file_t Error getting active value for svirt_sandbox_file_t Jianwei, Did you run "chcon" inside container? That command should be run on host. Can you check your environment, get your glusterfs volume label, and run the following docker exec test? Here is my gluster volume label: # ls -Zd /var/lib/openshift/openshift.local.volumes/pods/b1de7110-524f-11e5-bf46-b8ca3a627d6c/volumes/kubernetes.io~glusterfs drwxr-x---. root root system_u:object_r:svirt_sandbox_file_t:s0 /var/lib/openshift/openshift.local.volumes/pods/b1de7110-524f-11e5-bf46-b8ca3a627d6c/volumes/kubernetes.io~glusterfs My environment is: # oc version oc v3.0.1.0-528-g8c2fe51 kubernetes v1.0.0 # getenforce Permissive # docker inspect 71f90f66890e |grep gluster "/usr/share/nginx/html/test": "/var/lib/openshift/openshift.local.volumes/pods/b1de7110-524f-11e5-bf46-b8ca3a627d6c/volumes/kubernetes.io~glusterfs/gluster-default-volume", # docker exec -ti 71f90f66890e touch /usr/share/nginx/html/test/glusterfs/foobar # docker exec -ti 71f90f66890e ls -l /usr/share/nginx/html/test/glusterfs/foobar -rw-r--r--. 1 root root 0 Sep 24 09:59 /usr/share/nginx/html/test/glusterfs/foobar I did "chcon" on the from the node where the container is hosted, but the operation is not allowed [root@openshift-117 ~]# chcon -R -t svirt_sandbox_file_t /var/lib/openshift/openshift.local.volumes/pods/acfbc766-6d62-11e5-97e3-fa163e53da5a/volumes/kubernetes.io~glusterfs/gluster chcon: failed to change context of ‘index.html’ to ‘system_u:object_r:svirt_sandbox_file_t:s0’: Operation not supported chcon: failed to change context of ‘hello’ to ‘system_u:object_r:svirt_sandbox_file_t:s0’: Operation not supported chcon: failed to change context of ‘test’ to ‘system_u:object_r:svirt_sandbox_file_t:s0’: Operation not supported chcon: failed to change context of ‘/var/lib/openshift/openshift.local.volumes/pods/acfbc766-6d62-11e5-97e3-fa163e53da5a/volumes/kubernetes.io~glusterfs/gluster’ to ‘system_u:object_r:svirt_sandbox_file_t:s0’: Operation not supported Jianwei, thanks for the update. Your "chcon" went one level deeper. You have to apply the label to /var/lib/openshift/openshift.local.volumes/pods/acfbc766-6d62-11e5-97e3-fa163e53da5a/volumes/kubernetes.io~glusterfs/, rather than /var/lib/openshift/openshift.local.volumes/pods/acfbc766-6d62-11e5-97e3-fa163e53da5a/volumes/kubernetes.io~glusterfs/gluster. For more information, please check "Mounting External Volumes" from docker-run manpage. We don't pass security label during mount. There are upstream efforts to do this. For now, please make your pod privileged to write to gluster volume. I don't think you would be able to do a chcon on the client side for GlusterFS try running the above chcon on the server where Gluster is being served from Thank you all for the updates, I have verified that with privileged pod, I'm able to r/w to the mount dir of glusterfs with selinux enforcing. Inside container: The security context is [root@gluster ~]# ls -lZd /mnt/gluster/ drwxr-xr-x. root root system_u:object_r:fusefs_t:s0 /mnt/gluster/ From node: [root@openshift-114 ~]# mount|grep gluster 10.66.79.108:testvol on /var/lib/origin/openshift.local.volumes/pods/743092a5-796e-11e5-9ca2-fa163ee4ad04/volumes/kubernetes.io~glusterfs/gluster1 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) [root@openshift-114 ~]# ls -lZd /var/lib/origin/openshift.local.volumes/pods/743092a5-796e-11e5-9ca2-fa163ee4ad04/volumes/kubernetes.io~glusterfs/ drwxr-x---. root root system_u:object_r:svirt_sandbox_file_t:s0 /var/lib/origin/openshift.local.volumes/pods/743092a5-796e-11e5-9ca2-fa163ee4ad04/volumes/kubernetes.io~glusterfs/ [root@openshift-114 ~]# ls -lZd /var/lib/origin/openshift.local.volumes/pods/743092a5-796e-11e5-9ca2-fa163ee4ad04/volumes/kubernetes.io~glusterfs/gluster1/ drwxr-xr-x. root root system_u:object_r:fusefs_t:s0 /var/lib/origin/openshift.local.volumes/pods/743092a5-796e-11e5-9ca2-fa163ee4ad04/volumes/kubernetes.io~glusterfs/gluster1/ This bug can be verified according to comment 14 This fix is available in OpenShift Enterprise 3.1. |