Bug 1441216

Summary: Restricted SCC prevents execute on Gluster PVs
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Matthew Robson <mrobson>
Component: CNS-deploymentAssignee: Humble Chirammal <hchiramm>
Status: CLOSED CURRENTRELEASE QA Contact: Anoop <annair>
Severity: high Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: akhakhar, annair, aos-bugs, bchilds, fkrska, hchiramm, jarrpa, jokerman, madam, misalunk, mliyazud, mmccomas, mrobson, mzywusko, pprakash, rcyriac, rhs-bugs, rreddy, rtalur, vinug, wmeng
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1445226 (view as bug list) Environment:
Last Closed: 2017-08-02 15:46:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1445226    
Bug Blocks:    
Attachments:
Description Flags
POD example none

Description Matthew Robson 2017-04-11 12:45:26 UTC 
Description of problem:
A POD using the default restricted SCC can not execute files on a Gluster backed PV.  Read and Write are not an issue.  Same scenario works for NFS.

A privileged container can execute correctly.

The gluster brick mounted directly to the node can also execute correctly.

[root@node-d-133 log]# getsebool virt_use_fusefs
virt_use_fusefs --> on
[root@node-d-133 log]# getsebool virt_sandbox_use_fusefs
virt_sandbox_use_fusefs --> on

Privileged pod is able to execute.

[root@node-d-100 ~]# oc create -f foobla.yaml
pod "foobla" created
[root@node-d-100 ~]# oc rsh foobla
sh-4.2$ id
uid=1001(default) gid=0(root) groups=0(root)
sh-4.2$ cd /asdf/
sh-4.2$ ./foo.sh
TEST
sh-4.2$ ls -ldZ /asdf
drwxrwxrwx. root root system_u:object_r:fusefs_t:s0    /asdf
sh-4.2$ ls -lZ /asdf
-rwxr-xr-x. default root system_u:object_r:fusefs_t:s0    foo.sh
drwx------. 1000660000 root system_u:object_r:fusefs_t:s0    userdata

< edit foobla.yaml to have privileged: false >

[root@node-d-100 ~]# oc delete pod foobla
pod "foobla" deleted
[root@node-d-100 ~]# oc create -f foobla.yaml
pod "foobla" created
[root@node-d-100 ~]# oc rsh foobla
sh-4.2$ id
uid=1001(default) gid=0(root) groups=0(root)
sh-4.2$ cd /asdf/
sh-4.2$ ./foo.sh
sh: ./foo.sh: Permission denied
sh-4.2$ ls -ldZ /asdf/
drwxrwxrwx. root root system_u:object_r:fusefs_t:s0    /asdf/
sh-4.2$ ls -lZ /asdf/
-rwxr-xr-x. default root system_u:object_r:fusefs_t:s0    foo.sh
drwx------. 1000660000 root system_u:object_r:fusefs_t:s0    userdata

Brick Details:

# i='app_pv_100_5G'
# lvcreate -V 5G -T space01/userpool -n $i 
# mkfs.xfs -f -i size=512 -n size=8192 /dev/mapper/space01-$i 
# mkdir -p /mnt/userspace/$i 
# echo "/dev/mapper/space01-app_pv_100_5G /mnt/userspace/app_pv_100_5G xfs defaults 0 0" >> /etc/fstab
# mount –a
#  mkdir -p /mnt/userspace/$i/brick
# chmod 777 /mnt/userspace/$i/brick
# gluster volume create $i replica 2 transport tcp 192.168.111.140:/mnt/userspace/$i/brick 192.168.11.10:/mnt/userspace/$i/brick
# gluster volume start $i

Version-Release number of selected component (if applicable):
3.3.1.7

How reproducible:
100%

Steps to Reproduce:
1. Deploy a basic pod with any gluster PV, such a using the Jenkins persistent template or the attached POD 
2. Create a bash script on the mount
3. Try execute, Permission Denied

Actual results:

Permission denied

Expected results:

Executes


Additional info:
Comment 2 Matthew Robson 2017-04-11 12:48:01 UTC 
Created attachment 1270775 [details]
POD example
Comment 12 Miheer Salunke 2017-05-12 01:50:32 UTC 
So we installed selinux modules on your Openshift node which helped.


Summary ->

we oc rsh'ed in to jenkins pod

executed the script which gave permission denied 

We checked the audit logs

cat /var/log/audit.log | grep denied
We saw the following

 avc:  denied  { execute } 

# getsebool virt_sandbox_use_fusefs virt_use_fusefs

# grep fusefs /var/log/audit/audit.log | grep execute | audit2allow -M fusefstoexecutetemp

Installing the module  on OpenShift node->

# semodule -i fusefstoexecutetemp.pp

Then again we oc rsh'ed to the jenkins pod and executed the script. 

We saw the audit logs again and we saw ->
avc:  denied  { execute_no_trans }

- Checking and deleting earlier fusefstoexecutetemp

# semodule -l | grep fusefstoexecutetemp

# semodule -d fusefstoexecutetemp

Copy and add the contents to the new file as per the latest avc denial
# cp fusefstoexecutetemp.te fusefstoexecute.te

# Add the following to fusefstoexecute.te
##########################################################################
module fusefstoexecute 1.0;

require {
    type svirt_lxc_net_t;
    type fusefs_t;
    class file { execute execute_no_trans };
}

#============= svirt_lxc_net_t ==============
allow svirt_lxc_net_t fusefs_t:file execute_no_trans;
allow svirt_lxc_net_t fusefs_t:file execute;

###########################################################################

Save the file and run the following.

# checkmodule -M -m -o fusefstoexecute.mod fusefstoexecute.te

# semodule_package -m fusefstoexecute.mod -o fusefstoexecute.pp

Install the module the same way as before.

# semodule -i fusefstoexecute.pp

Again we oc rsh'ed into the pod and then we were able to execute the script.


After the package will be released, told them that they can simply remove the workaround by running (on every node)


# semodule -l | grep fuse
fusefstoexecutetemp     1.0
fusefstoexecute     1.0

# semodule -d fusefstoexecutetemp 
# semodule -d fusefstoexecute
Comment 14 Humble Chirammal 2017-08-02 15:46:46 UTC 
I am closing this bug as the subjected fix is already available and shipped. Please revert if the issue is still present.