Bug 852150

+++ This bug was initially created as a clone of Bug #811672 +++

Description of problem:

Mountbroker is a service provided by glusterd that can be used to request certain (pre-configured) glusterfs mounts and unmount them.

Mountbroker mounts succeed, but the unmount part fails with EACCES on RHEL-6.2 -- it seems to be caused by RHEL-specific security settings. Invoking umount(8) from shell (with same arguments as passed by glusterd) we succeed.

Version-Release number of selected component (if applicable):


How reproducible:

Deterministically.

Steps to Reproduce:

1. Set up mountbroker as described in RHS User Guide 9.2.5.2. (you may omit the creation of geogroup and the corresponding "option geo-replication-log-group geogroup" volume option)

2. mount the volume (here I use "slavevol", as in above doc) through mountbroker with following command:

# gluster system:: mount geoaccount user-map-root=geoaccount xlator-option=\*-dht.assert-no-child-down=true volfile-server=localhost volfile-id=slavevol client-pid=-1

This will give you back a path of the form /var/mountbroker-root/mb_hive/<mount id>

3. take down the above mount through mountbroker with following command:

# gluster system:: umount /var/mountbroker-root/mb_hive/<mount id>
  
Actual results:

We get back the message "umount failed" and the above command exits with 1.

Expected results:

We don't get any output and the above command exits with 0.

Additional info:

Stracing glusterd with following command:

# strace -s500 -f -eumount -p `cat /var/run/glusterd.pid`

displays

... umount("/var/mountbroker-root/mb_hive/mntTOKUsE", 0) = -1 EACCES (Permission denied)

--- Additional comment from csaba on 2012-04-11 12:59:48 EDT ---

For your ease, the RHS url:

http://docs.redhat.com/docs/en-US/Red_Hat_Storage/2/html/User_Guide/ch09s02s05s02.html

--- Additional comment from jdarcy on 2012-04-11 13:07:42 EDT ---

Anything in the audit log?

--- Additional comment from enakai on 2012-05-09 05:07:31 EDT ---

I suspect this is caused by SELinux. Here's the audit log.

-----
May  9 09:01:30 rhs20b2-02 kernel: type=1400 audit(1336554090.683:5): avc:  denied  { read } for  pid=2130 comm="umount" name="mnt48dS2M" dev=vda2 ino=29940 scontext=unconfined_u:system_r:mount_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=lnk_file
-----

And because of this, geo-replication with an unprivileged user, such as below, fails.

# gluster vol geo-replication vol01 geoaccount@rhs20b2-02::vol01_slave


A workaround is to "setenforce 0", but the final resolution should be an appropriate context labeling....

--- Additional comment from vbellur on 2012-05-14 03:31:11 EDT ---

Workaround available. Removing 3.3.0 milestone from this bug.

--- Additional comment from csaba on 2012-06-07 05:40:40 EDT ---

Etsuji: can you help us in dechipering the audit log entry? What is weird that the mount in question is created by glusterd (that is uid=0) and is to be taken down by glusterd too.

So it's a case of:
- root cannot do an umount
- the one who asked the system for some resource is not authorized to ask the system to release the resource

-- both seems contraintuitive from a naive POV. Which part of the context does imply this?

--- Additional comment from enakai on 2012-06-09 07:14:14 EDT ---

Ok... I did some investigation, and my hypothesis is:

- "umount" is forked from "glusterd", which is started by the upstart sequence.
- "mount" is forked from "gluterfs", which is forked from "glusterd".

Because of the difference of fork chain, "umount" and "mount" have differenct seculity contexts, and only "umount" is denied by SElinux.

Following is the result of my investigation which lead to the above hypothesis. 

====
1. After setting SELinux to Permissive mode and starting psacct on slave, I started georep from master.

2. According to the audit log on slave, "umount" has pid=1675, and its security context is "scontext=system_u:system_r:mount_t:s0".

Jun  9 10:37:56 slave01 kernel: type=1400 audit(1339238276.342:5): avc:  denied  { read } for  pid=1675 comm="umount" name="mnt3Tj3nM" dev=vda2 ino=30095 scontext=system_u:system_r:mount_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=lnk_file

3. According to the psacct log shown by "lastcomm --pid" on slave, the "umount" (pid=1675) is forked from pid=1217.

---------------
gluster               X georep01 __         0.00 secs Sat Jun  9 10:37     1671     1646
umount            S     root     __         0.00 secs Sat Jun  9 10:37     1675     1217
python                  georep01 __         0.03 secs Sat Jun  9 10:37     1673     1671
gluster               X georep01 __         0.00 secs Sat Jun  9 10:37     1656     1646
glusterfs         S     root     __         0.01 secs Sat Jun  9 10:37     1660     1217
mount             S     root     __         0.00 secs Sat Jun  9 10:37     1661     1660
python                  georep01 __         0.02 secs Sat Jun  9 10:37     1658     1656
---------------

And, pid=1271 is a "glusterd" launched by the upstart sequence, which has the security context "system_u:system_r:initrc_t:s0" as below.

# ps -efZ | grep 121[7]
system_u:system_r:initrc_t:s0   root      1217     1  0 10:30 ?        00:00:00 /usr/sbin/glusterd --pid-file=/var/run/glusterd.pid

4. On the otherhand, according to the same psacct log, "mount" (not denied) is forked from "glusterfs", not directly from "glusterd".
=====

To confirm the hypothesis, I stopped "glusterd" and started it again directly from the command line. Then "glusterd" has a different security context as below.

# ps -efZ | grep 1730
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 root 1730 1  1 10:45 ? 00:00:00 /usr/sbin/glusterd --pid-file=/var/run/glusterd.pid

In this case, georep succeeded without any SElinux prevention.

--- Additional comment from vinaraya on 2012-06-24 21:12:37 EDT ---

This bug does not block 820843, removing the dependency.