Following the OpenShift Enterprise guide, having added a mount point to /var/lib/openshift and executing quotacheck -cmug /var/lib/openshift returns permission errors. (see attachments. # rpm -qa selinux\* selinux-policy-targeted-3.7.19-155.el6_3.8.noarch selinux-policy-3.7.19-155.el6_3.8.noarch How reproducible: Follow guide on https://openshift.redhat.com/community/wiki/build-your-own Add a mount point for /var/lib/openshift for the "node 2" setup, and try to run the quotacheck -cmug command suggested. Using audit2allow didn't produce the complete policy to allow for the action. Only placing quota_t into permissive mode allows you to successfully complete the command.
Created attachment 652254 [details] Output from grep denied /var/log/audit/audit.log Details of the SELinux errors.
Ok, I see the error in the guide. restorecon -rv /var/run restorecon -rv /usr/share/rubygems/gems/passenger-* restorecon -rv /usr/sbin/mcollectived /var/log/mcollective.log /var/run/mcollectived.pid restorecon -rv /var/lib/openshift /etc/openshift/node.conf /etc/httpd/conf.d/openshift The problem is "-R" should be used instead of "-r". So restorecon -R -v /var/run restorecon -R -v /usr/share/rubygems/gems/passenger-* restorecon -R -v /usr/sbin/mcollectived /var/log/mcollective.log /var/run/mcollectived.pid restorecon -R -v /var/lib/openshift /etc/openshift/node.conf /etc/httpd/conf.d/openshift should fix this issue. Could you test it?
Does "-R" behave differently than "-r"? The man page for restorecon doesn't imply that it does. -R -r change files and directories file labels recursively
Yes. Forget it. Peter, did you run it?
I did a fixfiles onboot and that didn't solve any selinux problems.
What does # ls -alZ /var/lib/openshift
This is the output with semanage permissive -a quota_t Meaning I no longer get a permission denied, but the audit.log still shows the issue. # ls -aZl /var/lib/openshift/. total 32 drwxr-xr-x. 3 system_u:object_r:openshift_var_lib_t:s0 root root 4096 Nov 26 14:16 . drwxr-xr-x. 31 system_u:object_r:var_lib_t:s0 root root 4096 Nov 24 13:58 .. -rw-------. 1 unconfined_u:object_r:quota_db_t:s0 root root 6144 Nov 26 14:16 aquota.user drwx------. 2 system_u:object_r:openshift_var_lib_t:s0 root root 16384 Nov 26 14:15 lost+found What "checkquota -cmug" does is try to create the aquota.user file which due to the selinux policy issues is denied. The SE contexts on the directory etc. are all as they were before checkquota was run.
Miroslav I think 5d5c6b1669abcecf49e95157d1f342b5b1a995cb Will fix the problem.
Yes, the restorecon masked the quota_db was created as openshift_var_lib_t.
Peter, could you test it with the latest policy builds. https://brewweb.devel.redhat.com/buildinfo?buildID=245393
(In reply to comment #11) > Peter, > could you test it with the latest policy builds. > > https://brewweb.devel.redhat.com/buildinfo?buildID=245393 [root@node1 ~]# rpm --upgrade selinux-policy-3.7.19-183.el6.noarch.rpm selinux-policy-targeted-3.7.19-183.el6.noarch.rpm libsemanage.get_home_dirs: e2ddef0fc72144e5bc9530bbb0d68496 homedir /var/lib/openshift/e2ddef0fc72144e5bc9530bbb0d68496 or its parent directory conflicts with a file context already specified in the policy. This usually indicates an incorrectly defined system account. If it is a system account please make sure its uid is less than 500 or its login shell is /sbin/nologin. [root@node1 ~]# quotacheck -cmug /mnt/test quotacheck: Cannot create new quotafile /mnt/test/aquota.user.new: Permission denied quotacheck: Cannot initialize IO on new quotafile: Permission denied Still fails. I'm getting the strange error in between - not sure if it means anything. sealert suggests I do a full relabel. Will do that and comment again.
Btw - /mnt/test is my test using the openshift_var_lib_t context: # ls -ladZ /mnt/test drwxr-xr-x. root root system_u:object_r:openshift_var_lib_t:s0 /mnt/test
full relabel (fixfiles onboot) did not resolve the problem either.
Steps followed: semanage permissive -d quota_t lvcreate -n lvtest -L 4g vg_broker mkfs.ext4 -m 0 /dev/vg_broker/lvtest mkdir /mnt/test mount -o usrquota /dev/vg_broker/lvtest /mnt/test rpm --upgrade selinux-policy-3.7.19-183.el6.noarch.rpm selinux-policy-targeted-3.7.19-183.el6.noarch.rpm # get "strange message" chcon -t openshift_var_lib_t /mnt/test quotacheck -cmug /mnt/test # fails
Ok, the first issue is with a user with homedir in /var/lib/openshift. "Please make sure its uid is less than 500 or its login shell is /sbin/nologin"
If you execute # setenforce 0 # quotacheck -cmug /mnt/test # ausearch -m avc -ts recent
(In reply to comment #17) > Ok, the first issue is with a user with homedir in /var/lib/openshift. > > "Please make sure its uid is less than 500 or its login shell is > /sbin/nologin" I see this entry in /etc/passwd: e2ddef0fc72144e5bc9530bbb0d68496:x:500:500:OpenShift guest:/var/lib/openshift/e2ddef0fc72144e5bc9530bbb0d68496:/usr/bin/oo-trap-user Is /etc/login.defs supposed to be changed/modified as part of the install? This user was definitely not created by me (directly).
(In reply to comment #18) > If you execute > > # setenforce 0 > # quotacheck -cmug /mnt/test > # ausearch -m avc -ts recent [root@node1 ~]# quotacheck -cmug /mnt/test [root@node1 ~]# ausearch -m avc -ts recent ---- time->Thu Nov 29 08:32:31 2012 type=SYSCALL msg=audit(1354195951.886:7041): arch=c000003e syscall=2 success=yes exit=3 a0=7fff2a9a2dc0 a1=c2 a2=180 a3=7fff2a9a2b10 items=0 ppid=4855 pid=6759 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1160 comm="quotacheck" exe="/sbin/quotacheck" subj=unconfined_u:unconfined_r:quota_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(1354195951.886:7041): avc: denied { write } for pid=6759 comm="quotacheck" name="aquota.user.new" dev=dm-2 ino=12 scontext=unconfined_u:unconfined_r:quota_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:file_t:s0 tclass=file type=AVC msg=audit(1354195951.886:7041): avc: denied { create } for pid=6759 comm="quotacheck" name="aquota.user.new" scontext=unconfined_u:unconfined_r:quota_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:file_t:s0 tclass=file type=AVC msg=audit(1354195951.886:7041): avc: denied { add_name } for pid=6759 comm="quotacheck" name="aquota.user.new" scontext=unconfined_u:unconfined_r:quota_t:s0-s0:c0.c1023 tcontext=system_u:object_r:file_t:s0 tclass=dir ---- time->Thu Nov 29 08:32:31 2012 type=SYSCALL msg=audit(1354195951.888:7042): arch=c000003e syscall=82 success=yes exit=0 a0=7fff2a9a2dc0 a1=7f01fd7ed900 a2=1000 a3=7fff2a9a2aa0 items=0 ppid=4855 pid=6759 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1160 comm="quotacheck" exe="/sbin/quotacheck" subj=unconfined_u:unconfined_r:quota_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(1354195951.888:7042): avc: denied { rename } for pid=6759 comm="quotacheck" name="aquota.user.new" dev=dm-2 ino=12 scontext=unconfined_u:unconfined_r:quota_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:file_t:s0 tclass=file type=AVC msg=audit(1354195951.888:7042): avc: denied { remove_name } for pid=6759 comm="quotacheck" name="aquota.user.new" dev=dm-2 ino=12 scontext=unconfined_u:unconfined_r:quota_t:s0-s0:c0.c1023 tcontext=system_u:object_r:file_t:s0 tclass=dir ---- time->Thu Nov 29 08:32:31 2012 type=SYSCALL msg=audit(1354195951.889:7043): arch=c000003e syscall=90 success=yes exit=0 a0=7f01fd7ed900 a1=180 a2=1000 a3=7fff2a9a2aa0 items=0 ppid=4855 pid=6759 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1160 comm="quotacheck" exe="/sbin/quotacheck" subj=unconfined_u:unconfined_r:quota_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(1354195951.889:7043): avc: denied { setattr } for pid=6759 comm="quotacheck" name="aquota.user" dev=dm-2 ino=12 scontext=unconfined_u:unconfined_r:quota_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:file_t:s0 tclass=file
(In reply to comment #19) > (In reply to comment #17) > > Ok, the first issue is with a user with homedir in /var/lib/openshift. > > > > "Please make sure its uid is less than 500 or its login shell is > > /sbin/nologin" > > I see this entry in /etc/passwd: > e2ddef0fc72144e5bc9530bbb0d68496:x:500:500:OpenShift > guest:/var/lib/openshift/e2ddef0fc72144e5bc9530bbb0d68496:/usr/bin/oo-trap- > user Could you fix uid to see if we can get this working. > Is /etc/login.defs supposed to be changed/modified as part of the install? > This user was definitely not created by me (directly). The question for OpenShift folks. How did it work for you?
I've had the broker/node running all weekend with no intervention (this is Christmas month so I don't have much time to do weekend computer stuff). I'm very puzzled - the user mentioned in comment 19 is no longer defined, and the selinux error is gone. I'll retry the patch fix. Instead of testing on a new mount-point, I'm going to remove quota and then add it again to /var/lib/openshift.
I can now add/remove quota with SElinux in enforcing mode on a mounted /var/lib/openshift. Somehow I'm not able to do that on the /mnt/test - but removing quota and re-creating it for /var/lib/openshift works.
I tested the following packages today: selinux-policy-targeted-3.7.19-183.el6.noarch selinux-policy-3.7.19-183.el6.noarch Here's what I did: * Created a volume and mounted it at /var/lib/openshift with usrquota turned on * Tried running 'quotacheck -cmug /var/lib/openshift' before upgrading the packages. I received permission denied * I updated the packages. At this point I saw the same warnings about having an incorrectly defined system account. In the case of OpenShift I think that warning is incorrect. /var/lib/openshift is where the Gear ssh account home directories are stored. * I ran the quotacheck command again and it worked. At this point I did some general checking of the system and things appear to be working correctly. Is there any way we could get a build of the policy for RHEL 6.3 that our QE team could perform regression testing on?
You should be able to run the same policy on both RHEL6.3 and 6.4
This policy definitely works on RHEL6.3. I was mostly curious about if we could get a backported policy for RHEL6.3 that we could possibly z-stream. Would that be possible?
If you want a z-stream, you need to request it.
While we're waiting for the zstream for RHEL 6.3, it's worth noting a workaround. Seems only the quota file creation is blocked. 1. setenforce 0 2. quotacheck -cmug /var/lib/openshift 3. setenforce 1 4. restorecon /var/lib/openshift/aquota.user 5. reboot (there's probably a better way but a simple remount didn't work) 6. New gears will get the quotas. edquota <uuid> for any gears already created, set limits according to gear size.
I'm nominating this bug for 6.3.Z, on behalf of the open shift team. For questions please email bleanhar
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0314.html