Bug 1231936

Summary: Docker contaier can't access gluster (fuse) volumes
Product: Red Hat Enterprise Linux 7 Reporter: Lokesh Mandvekar <lsm5>
Component: dockerAssignee: Daniel Walsh <dwalsh>
Status: CLOSED ERRATA QA Contact: Luwen Su <lsu>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.1CC: admiller, dwalsh, extras-qa, ichavero, jcajka, jchaloup, jhou, jsafrane, jvance, lsm5, miminar, mjenner, mturansk, sghosh, vbatts
Target Milestone: rcKeywords: Extras
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: docker-1.6.2-11.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1231134 Environment:
Last Closed: 2015-06-23 09:30:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1231134    
Bug Blocks:    

Description Lokesh Mandvekar 2015-06-15 16:14:33 UTC
+++ This bug was initially created as a clone of Bug #1231134 +++

SELinux denies any access from inside a Docker container to a mounted GlusterFS volume, even though virt_use_fusefs boolean is set to 'on'.

AVC gathered with disabled dontaudit rules (why dontaudit this ????):
type=SYSCALL msg=audit(1434098028.965:515): arch=c000003e syscall=257 success=no exit=-13 a0=ffffffffffffff9c a1=194bc60 a2=90800 a3=0 items=0 ppid=4012 pid=4054 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts3 ses=4294967295 comm="ls" exe="/usr/bin/ls" subj=system_u:system_r:svirt_lxc_net_t:s0:c131,c679 key=(null)
type=AVC msg=audit(1434098028.965:515): avc:  denied  { read } for  pid=4054 comm="ls" name="/" dev="fuse" ino=1 scontext=system_u:system_r:svirt_lxc_net_t:s0:c131,c679 tcontext=system_u:object_r:fusefs_t:s0 tclass=dir permissive=0


Version-Release number of selected component (if applicable):
selinux-policy-targeted-3.13.1-105.13.fc21.noarch

How reproducible:
always

Steps to Reproduce:
1. mount a GlusterFS volume to /mnt/test (or use any other fuse filesystem)
mount -t glusterfs my-gluster-server:test_vol /mnt/test

2. use it as a container volume 
docker run -ti -v /mnt/test:/mnt/test fedora:21 bash

3. (inside the container) look at /mnt/test
ls /mnt/test

Actual results:
ls: cannot open directory /mnt/test: Permission denied

Expected results:
see content of the directory

Additional info:
As stated above, 'setsebool -P virt_use_fusefs on' does not help. On the other way, 'setsebool -P virt_use_nfs on' helps to get the same working for NFS.

Also, these dontaudit rules only complicate things, it's hard to guess who rejects access in a complex environment where Docker, Kubernetes, SElinux, kernel namespaces and Gluster are involved.

--- Additional comment from Jeff Vance on 2015-06-12 16:07:27 CDT ---

I've observed that, although you cannot ls the mounted dir (as if r access to the dir is denied), you can pass through it (x access) and cat files in the gluster volume.

--- Additional comment from Daniel Walsh on 2015-06-15 08:12:42 CDT ---

I just added a 

virt_sandbox_use_fusefs boolean to docker-selinux.

If we got proper labeling on gluster files via XAttrs we would not have this problem.

This boolean will allow docker containers to attack any fusefs files on the system, including all gluster mounted drives, if it can get access to them.

Comment 3 Luwen Su 2015-06-16 04:27:51 UTC
Due to the limitation of my environment, i don't have a glusterfs to use, so i found an experiment fuse program from 
https://stuff.mit.edu/iap/2009/fuse/examples/

#mkdir /root/fuse/tmp
#python hello.py /root/fuse/tmp 
(pip install fuse-python, if can't find the import)
#ls -aZ /root/fuse
drwxr-xr-x. root root system_u:object_r:fusefs_t:s0    tmp

# getsebool -a | grep -i fuse
ftpd_use_fusefs --> off
glance_use_fusefs --> off
httpd_use_fusefs --> off
mailman_use_fusefs --> off
samba_share_fusefs --> off
sanlock_use_fusefs --> off
use_fusefs_home_dirs --> off
virt_sandbox_use_fusefs --> off
virt_use_fusefs --> off

#docker run -v /root/fuse/tmp:/fuse rhel7 ls /fuse
ls: cannot open directory /fuse: Permission denied
#setsebool virt_sandbox_use_fusefs 1
# docker run -v /root/fuse/tmp:/fuse rhel7 ls -aZ /fuse
drwxr-xr-x. root root system_u:object_r:fusefs_t:s0    .
drwxr-xr-x. root root system_u:object_r:svirt_sandbox_file_t:s0:c22,c732 ..
-r--r--r--. root root system_u:object_r:fusefs_t:s0    hello
# docker run -v /root/fuse/tmp:/fuse rhel7 cat /fuse/hello
Hello World!

In docker-1.6.2-14.el7.x86_64, move to verified

Comment 6 Mark Turansky 2015-06-19 13:06:06 UTC
Cut and pasted from an email from Jan Safranek who is working with gluster:

To follow up on current development, docker-1.6.2-14.el7 adds a new
boolean to SELinux policy. This should allow docker container to access
gluster volume (optionally with -P for persistent change):

setsebool virt_sandbox_use_fusefs 1

dwalsh told me docker-1.6.2-14.el7 is heading to RHEL 7.1.3, i.e. to be
released in couple of days (2015-Jun-23?) in RHEL7 extras channel.

So what we need is just to document the boolean and encourage customers
to check docker package version (and wait for 1.6.2-14).

Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1231936
Docker build: https://brewweb.devel.redhat.com/buildinfo?buildID=441835
Errata: https://errata.devel.redhat.com/advisory/20663

Comment 7 Mark Turansky 2015-06-19 13:21:23 UTC
*** Bug 1233540 has been marked as a duplicate of this bug. ***

Comment 8 Mark Turansky 2015-06-19 13:24:18 UTC
*** Bug 1233482 has been marked as a duplicate of this bug. ***

Comment 9 Mark Turansky 2015-06-19 13:25:50 UTC
Disregard the "dupe 1233540".  I linked the wrong issue accidentally.

Comment 11 errata-xmlrpc 2015-06-23 09:30:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1167.html