Description of problem: After creating a gluster volume on top of xfs partitions, and mounting that volume, I'm unable to change the security context of files on the mounted filesystem. Version-Release number of selected component: Tested with both glusterfs-3.6.4-1.el6.x86_64 glusterfs-api-3.6.4-1.el6.x86_64 glusterfs-cli-3.6.4-1.el6.x86_64 glusterfs-debuginfo-3.6.4-1.el6.x86_64 glusterfs-extra-xlators-3.6.4-1.el6.x86_64 glusterfs-fuse-3.6.4-1.el6.x86_64 glusterfs-libs-3.6.4-1.el6.x86_64 glusterfs-server-3.6.4-1.el6.x86_64 glusterfs-3.7.3-1.el6.x86_64 glusterfs-api-3.7.3-1.el6.x86_64 glusterfs-cli-3.7.3-1.el6.x86_64 glusterfs-client-xlators-3.7.3-1.el6.x86_64 glusterfs-debuginfo-3.7.3-1.el6.x86_64 glusterfs-fuse-3.7.3-1.el6.x86_64 glusterfs-libs-3.7.3-1.el6.x86_64 glusterfs-server-3.7.3-1.el6.x86_64 How reproducible: Always Steps to Reproduce: (for 3.7.3) Built the latest RPM's from source from: http://dl.fedoraproject.org/pub/epel/6/SRPMS/userspace-rcu-0.7.7-1.el6.src.rpm http://download.gluster.org/pub/gluster/glusterfs/LATEST/RHEL/epel-6.6/SRPMS/glusterfs-3.7.3-1.el6.src.rpm rpm -ivh glusterfs-3.7.3-1.el6.src.rpm cd rpmbuild rpmbuild -ba SPECS/glusterfs.spec |& tee log (put the rpm's in a private repository) Create two test VM's, "ga" and "gb" using rhel66. Each VM has partitions: Filesystem 1K-blocks Used Available Use% Mounted on /dev/vda8 6558440 2802892 3415736 46% / tmpfs 510028 0 510028 0% /dev/shm /dev/vda3 200000 10400 189600 6% /b1 /dev/vda5 200000 10400 189600 6% /b2 /dev/vda6 200000 10400 189600 6% /b3 /dev/vda7 200000 10400 189600 6% /b4 /dev/vda1 243823 28113 202910 13% /boot /dev/vda1: UUID="d3642293-57b1-4988-ac4f-85b0635e64c6" TYPE="ext4" /dev/vda2: UUID="f08a07bf-9222-45c7-9fd1-4d33207a8b86" TYPE="swap" /dev/vda3: UUID="ae1d0314-c22d-401e-a1b9-cf30fa4c6542" TYPE="xfs" /dev/vda5: UUID="5d32686c-5f00-4a00-ac06-539d2b85110d" TYPE="xfs" /dev/vda6: UUID="97e56854-76e6-4484-9afb-c5f56907df6e" TYPE="xfs" /dev/vda7: UUID="9e4d2bd4-de96-43f0-9aa3-3769cb22d508" TYPE="xfs" /dev/vda8: UUID="0be135b1-9bd5-49f5-8b21-8f43367a825b" TYPE="ext4" For the test we're only using the /b1 partitions on each host as our test bricks. yum install glusterfs glusterfs-api glusterfs-cli glusterfs-client-xlators glusterfs-debuginfo glusterfs-fuse glusterfs-libs glusterfs-server Create the a replicated volume: chkconfig glusterd on service glusterd start gluster peer probe ga gluster peer probe gb gluster volume create gvol replica 2 transport tcp ga:/b1 gb:/b1 force gluster volume start gvol gluster volume set gvol auth.allow ga,gb gluster volume set gvol nfs.disable on gluster volume info Volume Name: gvol Type: Replicate Volume ID: 4eeb493c-ed5f-4c3b-8945-4d14848a95d5 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: ga:/b1 Brick2: gb:/b1 Options Reconfigured: nfs.disable: on auth.allow: ga,gb performance.readdir-ahead: on On each host, mount the volume mkdir /data mount -t glusterfs -o selinux localhost:/gvol /data Check that the --selinux switch is asserted ... # ps -eo args |grep glust /usr/sbin/glusterd --pid-file=/var/run/glusterd.pid /usr/sbin/glusterfsd -s gb --volfile-id gvol.gb.b1 -p /var/lib/glusterd/vols/gvol/run/gb-b1.pid -S /var/run/gluster/8dd23446126b2065164fdba21397998f.socket --brick-name /b1 -l /var/log/glusterfs/bricks/b1.log --xlator-option *-posix.glusterd-uuid=72fb7173-c1d3-4edd-b7e6-e7633e33358e --brick-port 49152 --xlator-option gvol-server.listen-port=49152 /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /var/lib/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/a8c70c7c13620b79a8b5d26757294453.socket --xlator-option *replicate*.node-uuid=72fb7173-c1d3-4edd-b7e6-e7633e33358e /usr/sbin/glusterfs --selinux --volfile-server=localhost --volfile-id=/gvol /data Make some directories and files: mkdir -p /data/a/b/c echo test file > /data/a/b/myfile Now for the test ... Actual results: [root@ga ~]# ls -Z /data/a/b/myfile -rw-r--r--. root root system_u:object_r:fusefs_t:s0 /data/a/b/myfile [root@ga ~]# chcon -t tftpdir_rw_t /data/a/b/myfile chcon: failed to change context of `/data/a/b/myfile' to `system_u:object_r:tftpdir_rw_t:s0': Operation not supported [root@ga ~]# ls -Z /data/a/b/myfile -rw-r--r--. root root system_u:object_r:fusefs_t:s0 /data/a/b/myfile Expected results: [root@ga ~]# ls -Z /data/a/b/myfile -rw-r--r--. root root system_u:object_r:fusefs_t:s0 /data/a/b/myfile [root@ga ~]# chcon -t tftpdir_rw_t /data/a/b/myfile [root@ga ~]# ls -Z /data/a/b/myfile -rw-r--r--. root root system_u:object_r:tftpdir_rw_t:s0 /data/a/b/myfile Additional info: Works on other file systems: [root@ga ~]# touch /tmp/x [root@ga ~]# ls -Z /tmp/x -rw-r--r--. root root unconfined_u:object_r:user_tmp_t:s0 /tmp/x [root@ga ~]# chcon -t tftpdir_rw_t /tmp/x [root@ga ~]# ls -Z /tmp/x -rw-r--r--. root root unconfined_u:object_r:tftpdir_rw_t:s0 /tmp/x Both hosts have selinux enabled in permissive mode The mount has selinux capability enabled. Is there anything on the server side that needs to be configured to enable selinux capability?
The man pages for glusterd, glusterfs, and glusterfsd processes indicate that they take a --selinux flag. I tried applying this by hacking the glusterd code to add it - without success. I tried: /etc/sysconfig/glusterd add GLUSTERD_OPTIONS='--selinux' Patch the glusterfs-3.7.3 build: --- SPECS/glusterfs.spec_orig 2015-08-25 00:38:13.610000109 +0000 +++ SPECS/glusterfs.spec 2015-08-26 00:26:26.793000286 +0000 @@ -165,7 +165,7 @@ %if ( 0%{_for_fedora_koji_builds} ) Name: glusterfs Version: 3.7.3 -Release: 1%{?prereltag:.%{prereltag}}%{?dist} +Release: 4%{?prereltag:.%{prereltag}}%{?dist}rda Vendor: Fedora Project %else Name: @PACKAGE_NAME@ @@ -187,6 +187,8 @@ Source0: @PACKAGE_NAME@-@PACKAGE_VERSION@.tar.gz %endif +Patch0: glusterd-3.7.3-selinux.patch + BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX) %if ( 0%{?rhel} && 0%{?rhel} <= 5 ) @@ -595,6 +597,7 @@ %prep %setup -q -n %{name}-%{version}%{?prereltag} +%patch0 -p1 -b .selinux %build # For whatever reason, install-sh is sometimes missing. When this gets fixed, ----------------------------------------- $ cat SOURCES/glusterd-3.7.3-selinux.patch --- ./xlators/mgmt/glusterd/src/glusterd-quota.c_orig 2015-08-26 00:21:01.186000302 +0000 +++ ./xlators/mgmt/glusterd/src/glusterd-quota.c 2015-08-26 00:22:04.272000274 +0000 @@ -246,6 +246,7 @@ runinit (&runner); runner_add_args (&runner, SBIN_DIR"/glusterfs", + "--selinux", "-s", "localhost", "--volfile-id", volname, "--use-readdirp=no", --- ./xlators/mgmt/glusterd/src/glusterd-volume-ops.c_orig 2015-08-26 00:21:01.195000301 +0000 +++ ./xlators/mgmt/glusterd/src/glusterd-volume-ops.c 2015-08-26 00:22:42.866000330 +0000 @@ -2770,7 +2770,7 @@ runinit (&runner); glusterd_get_trusted_client_filepath (client_volfpath, volinfo, volinfo->transport_type); - runner_add_args (&runner, SBIN_DIR"/glusterfs", "-f", NULL); + runner_add_args (&runner, SBIN_DIR"/glusterfs", "-f", NULL, "--selinux"); runner_argprintf (&runner, "%s", client_volfpath); runner_add_arg (&runner, "-l"); runner_argprintf (&runner, DEFAULT_LOG_FILE_DIRECTORY --- ./xlators/mgmt/glusterd/src/glusterd-mountbroker.c_orig 2015-08-26 00:21:01.185000302 +0000 +++ ./xlators/mgmt/glusterd/src/glusterd-mountbroker.c 2015-08-26 00:21:54.417000313 +0000 @@ -659,6 +659,7 @@ runinit (&runner); runner_add_arg (&runner, SBIN_DIR"/glusterfs"); + runner_add_arg (&runner, "--selinux"); seq_dict_foreach (argdict, _runner_add, &runner); runner_add_arg (&runner, mtptemp); ret = runner_run_reuse (&runner); --- ./xlators/mgmt/glusterd/src/glusterd-rebalance.c_orig 2015-08-26 00:21:01.188000301 +0000 +++ ./xlators/mgmt/glusterd/src/glusterd-rebalance.c 2015-08-26 00:22:14.668000295 +0000 @@ -261,6 +261,7 @@ snprintf (volname, sizeof(volname), "rebalance/%s", volinfo->volname); runner_add_args (&runner, SBIN_DIR"/glusterfs", + "--selinux", "-s", "localhost", "--volfile-id", volname, "--xlator-option", "*dht.use-readdirp=yes", "--xlator-option", "*dht.lookup-unhashed=yes", --- ./xlators/mgmt/glusterd/src/glusterd-replace-brick.c_orig 2015-08-26 00:21:01.189000301 +0000 +++ ./xlators/mgmt/glusterd/src/glusterd-replace-brick.c 2015-08-26 00:22:18.535000291 +0000 @@ -83,6 +83,7 @@ runinit (&runner); runner_add_args (&runner, SBIN_DIR"/glusterfs", + "--selinux", "-s", "localhost", "--volfile-id", volinfo->volname, "--client-pid", pid, --- ./xlators/mgmt/glusterd/src/glusterd-snapd-svc.c_orig 2015-08-26 00:21:01.191000299 +0000 +++ ./xlators/mgmt/glusterd/src/glusterd-snapd-svc.c 2015-08-26 00:22:22.317000303 +0000 @@ -287,6 +287,7 @@ snprintf (snapd_id, sizeof (snapd_id), "snapd-%s", volinfo->volname); runner_add_args (&runner, SBIN_DIR"/glusterfsd", + "--selinux", "-s", svc->proc.volfileserver, "--volfile-id", svc->proc.volfileid, "-p", svc->proc.pidfile, --- ./xlators/mgmt/glusterd/src/glusterd-utils.c_orig 2015-08-26 00:21:01.193000300 +0000 +++ ./xlators/mgmt/glusterd/src/glusterd-utils.c 2015-08-26 00:22:32.278000299 +0000 @@ -1620,6 +1620,7 @@ (void) snprintf (glusterd_uuid, 1024, "*-posix.glusterd-uuid=%s", uuid_utoa (MY_UUID)); runner_add_args (&runner, SBIN_DIR"/glusterfsd", + "--selinux", "-s", brickinfo->hostname, "--volfile-id", volfile, "-p", pidfile, "-S", socketpath, "--brick-name", brickinfo->path, --- ./xlators/mgmt/glusterd/src/glusterd-svc-mgmt.c_orig 2015-08-26 00:21:01.192000299 +0000 +++ ./xlators/mgmt/glusterd/src/glusterd-svc-mgmt.c 2015-08-26 00:22:28.821000303 +0000 @@ -182,6 +182,7 @@ } runner_add_args (&runner, SBIN_DIR"/glusterfs", + "--selinux", "-s", svc->proc.volfileserver, "--volfile-id", svc->proc.volfileid, "-p", svc->proc.pidfile, ----------------------------------------- Although this applies --selinux to all processes: # ps -eo args |grep glust /usr/sbin/glusterd --pid-file=/var/run/glusterd.pid --selinux /usr/sbin/glusterfsd --selinux -s ga --volfile-id gvol.ga.b1 -p /var/lib/glusterd/vols/gvol/run/ga-b1.pid -S /var/run/gluster/11753d16ee8a048e5f9b2331cbcfd4c7.socket --brick-name /b1 -l /var/log/glusterfs/bricks/b1.log --xlator-option *-posix.glusterd-uuid=6f491c3b-53d5-4928-8435-6c3d84f3ce53 --brick-port 49152 --xlator-option gvol-server.listen-port=49152 /usr/sbin/glusterfs --selinux -s localhost --volfile-id gluster/glustershd -p /var/lib/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/6502d8ef42d50130bd676cf9ef26c76d.socket --xlator-option *replicate*.node-uuid=6f491c3b-53d5-4928-8435-6c3d84f3ce53 /usr/sbin/glusterfs --selinux --volfile-server=localhost --volfile-id=/gvol /data .. I still see the same error using chcon. So something deeper in the code seems to be missing. Anyone have an idea where the disconnect is?
Ping, this is being referenced externally as well: https://github.com/CiscoCloud/microservices-infrastructure/issues/867#issuecomment-159730627
Not that this bug depends on #1318100. Until that is resolved, Gluster will not be able to change the SELinux labels. The standard SELinux mount options to set specific labels for the whole mounted Gluster volume can be used (like "context", see #1287877 and http://review.gluster.org/12870). https://bugzilla.redhat.com/show_bug.cgi?id=1318100#c1 also contains some more details, and points to a discussion on the mailinglist: > At the moment it is not possible to set the SELinux context on a mounted > Gluster Volume. We intend to have this functionality added to the Gluster core, > and from there on add support to additional layers (FUSE, Labelled NFS, ...). > > More details are listed in a conversation on the Gluster developers list: > http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/13071e This is not something that we'll be able to backport to 3.7 or 3.8. We might be able to get initial support in GlusterFS 3.9. Changes to the kernel to support SELinux over FUSE might not be ready by that time though. I'm closing this as DEFERRED, because SELinux support on Gluster volumes will not happen in 3.7.