Description of problem: After update selinux-policy to latest version (selinux-policy-3.7.19-195.el6_4.1.noarch), openshift cron job generated avc denied message. Version-Release number of selected component (if applicable): 1.1.z/2013-02-20.1 How reproducible: Always Steps to Reproduce: 1.Install RHEL6.4 system, it installed selinux-policy-3.7.19-195.el6.noarch. 2.Setup node env on this system 3./etc/cron.minutely/openshift-facts will be executed every min, and monitor audit.log to make sure there is no avc denied message generated by this cron job. 4.Update selinux-policy to latest version (selinux-policy-3.7.19-195.el6_4.1.noarch). # yum update selinux-policy 5.Monitor audit.log. Actual results: cron job will generate avc denied message. # grep denied /var/log/audit/audit.log*|grep cron /var/log/audit/audit.log.1:type=AVC msg=audit(1361875921.479:776343667): avc: denied { read } for pid=13799 comm="ruby" name="localtime" dev=dm-0 ino=130732 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:locale_t:s0 tclass=file /var/log/audit/audit.log.1:type=AVC msg=audit(1361875922.270:776344837): avc: denied { read } for pid=13799 comm="ruby" name="localtime" dev=dm-0 ino=130732 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:locale_t:s0 tclass=file Expected results: cron job should not generate this avc denied message. Additional info:
"grep openshift_cron_t /var/log/audit/audit.log |audit2allow -R -M cronpol" generated the following: policy_module(cronpol, 1.0) require { type openshift_cron_t; class capability { dac_read_search dac_override }; } #============= openshift_cron_t ============== allow openshift_cron_t self:capability { dac_read_search dac_override }; kernel_search_xen_state(openshift_cron_t) miscfiles_read_localization(openshift_cron_t) The dac_override part seems really odd. These denials were not happening with selinux-policy-3.7.19-195.el6 and seemed to have been introduced with selinux-policy-3.7.19-195.el6_4.1. Miroslav, do you have any suggestions to help us debug this further?
allow openshift_cron_t self:capability { dac_read_search dac_override }; Could you re-test it with full auditing # echo "-w /etc/shadow -p w" >> /etc/audit/audit.rules # service auditd restart
Created attachment 703504 [details] Denial with full audit logging on
*** Bug 921125 has been marked as a duplicate of this bug. ***
In bug 921125 I'm pretty sure I can identify the code that is prompting this (since I can see it's failing to run, although errors are swallowed). The exact same code runs on OS Online devenvs without apparent problem, however that may have custom policy in place. I'd be happy to provide logins for both to compare and debug.
Could it be tested with selinux-policy-3.7.19-195.el6_4.3? Basically just dac_* caps should remain. Unfortunatelly I see nothing in audit.log. If you re-test it with the lastest rhel-6.4 policy, are these caps needed to make it working?
I installed selinux-policy-3.7.19-195.el6_4.3.noarch on 10.4.59.158 before commenting on this ticket - the problem still exists there (the cron job runs minutely so there should be no shortage of entries). Please let me know what more I can do to help pinpoint this.
Now the avc denial message like this: type=AVC msg=audit(1363850222.268:75794): avc: denied { dac_override } for pid=15030 comm="ruby" capability=1 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tclass=capability type=AVC msg=audit(1363850222.268:75794): avc: denied { dac_override } for pid=15030 comm="ruby" capability=1 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tclass=capability
Ok, I finally found it type=PATH msg=audit(1363858501.401:141587): item=0 name="/var/lib/openshift/1e4aa50fadf34b1187917c49cebc0e46/app-root/runtime/.state" type=PATH msg=audit(1363858501.401:141587): item=1 name=(null) inode=665159 dev=fd:00 mode=040750 ouid=1021 ogid=1021 rdev=00:00 obj=unconfined_u:object_r:openshift_var_lib_t:s0:c0,c1021 type=AVC msg=audit(1363858501.401:141588): avc: denied { dac_override } for pid=30651 comm="ruby" capability=1 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tclass=capability type=SYSCALL msg=audit(1363858501.401:141588): arch=c000003e syscall=6 success=no exit=-13 a0=23e7330 a1=7fffe5afd960 a2=7fffe5afd960 a3=2 items=2 ppid=30648 pid=30651 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=2422 comm="ruby" exe="/usr/bin/ruby" subj=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 key=(null) So the problem is with permissions on /var/lib/openshift/1e4aa50fadf34b1187917c49cebc0e46/app-root/runtime/.state file.
It means it is owned by 1e4aa50fadf34b1187917c49cebc0e46 user with -rw-r-----. 1 1e4aa50fadf34b1187917c49cebc0e46 1e4aa50fadf34b1187917c49cebc0e46 8 Mar 21 05:50 /var/lib/openshift/1e4aa50fadf34b1187917c49cebc0e46/app-root/runtime/.state but and we have a process with openshift_cron_t:s0-s0:c0.c1023 running with type=SYSCALL msg=audit(1363858501.401:141588): arch=c000003e syscall=6 success=no exit=-13 a0=23e7330 a1=7fffe5afd960 a2=7fffe5afd960 a3=2 items=2 ppid=30648 pid=30651 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=2422 comm="ruby" exe="/usr/bin/ruby"
So, we definitely want the cron job to be able to read that file. And I'm fairly certain it could prior to the RHEL 6.4 release. Is this an selinux policy fix that needs to be made or is there something else we need to consider changing about OpenShift?
Ok, basically dac_override is pretty powerful. So the cron job "just" reads this file?
Correct, the cron job just reads the .state file (and presumably the directories above).
Ok. Usually we try to avoid dac_override as much as possible by changing of permissions. If we want to get it to rhel6.4, we need to open a new selinux-policy bug with this issue.
I am adding fixes to RHEL6.5.
Sorry for the delay on my end. I meant to look into how this is being handled upstream and whether there's some way to work around it. Hasn't happened yet. But, if you're adding fixes for RHEL6.5, then that's the path forward - can we please requests a zstream for RHEL 6.4 as well? We do not want this broken for months.
Temporary workaround while we await the RHEL policy fix: a custom policy on nodes. # cat > openshift_cron.te #copy and paste everything until the ^D module openshift_cron 1.0; require { type openshift_cron_t; class capability dac_override; } #============= openshift_cron_t ============== allow openshift_cron_t self:capability dac_override; ^D # actually type Ctrl-D to exit cat; the rest is to use this file # make -f /usr/share/selinux/devel/Makefile # semodule -i openshift_cron.pp
Should be fixed in the latest selinux-policy RHEL6.5 build.
I didn't see 915701 in the RPM changelog, but this version seems to have shut up the errors: selinux-policy-3.7.19-198.el6.noarch
I'm testing on a recent RHEL 6.4 OSE 1.2 build and I no longer see the openshift_cron_t AVCs ("grep openshift_cron_t /var/log/audit/audit.log" returns no output) Release .2 is installed: selinux-policy-3.7.19-195.el6_4.2.noarch selinux-policy-targeted-3.7.19-195.el6_4.2.noarch What I see how is output like: type=CRED_DISP msg=audit(1368545403.343:72670): user pid=13727 uid=0 auid=0 ses=180 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1368545403.344:72671): user pid=13727 uid=0 auid=0 ses=180 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' Those don't look SELinux related. I also tested with the latest RHEL 6.4 builds I found in brew (.5) and they produce the same results. I'm going to move this to ON_QA for re-testing.
I see selinux-policy-3.7.19-195.el6_4.3 contains "Backport openshfit fixes". Though it wasn't until .4 that the fix mentioned in this bug was applied: * Wed Apr 24 2013 Miroslav Grepl <mgrepl> 3.7.19-195.el6_4.4 - Allow dirsrv-admin server to be restarted from console Resolves:#955703 - Allow dac_override for openshift-cron Miroslav, Did anything change in selinux-policy-3.7.19-195 .2 or .3 that would address this bug? It appears to no longer be reproducible with those builds.
@Brenton, I could reproduce this issue with selinux-policy-3.7.19-195.el6_4.3. type=AVC msg=audit(1368605221.435:149): avc: denied { dac_read_search } for pid=4736 comm="ruby" capability=2 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tclass=capability type=AVC msg=audit(1368605221.436:150): avc: denied { dac_override } for pid=4736 comm="ruby" capability=1 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tclass=capability type=AVC msg=audit(1368605221.436:150): avc: denied { dac_read_search } for pid=4736 comm="ruby" capability=2 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tclass=capability After I update selinux-policy to selinux-policy-3.7.19-195.el6_4.4, the "dac_read_search" avc message is disappeared. So this bug is fixed on selinux-policy-3.7.19-195.el6_4.4, move it to verified status. Actually still some other avc message are seen. I will open a new bug to tracking it.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2013-1032.html