Description of problem: In openshift node, there is a cron job to update mcollective facts. # cat /etc/cron.minutely/openshift-facts #!/bin/bash PREFIX="" if [ -f /opt/rh/ruby193/root/usr/libexec/mcollective/update_yaml.rb ]; then PREFIX="/opt/rh/ruby193/root" fi oo-exec-ruby ${PREFIX}/usr/libexec/mcollective/update_yaml.rb ${PREFIX}/etc/mcollective/facts.yaml &> /tmp/facts.log Monitor /var/log/audit/audit.log, and wait for a moment. # tailf /var/log/audit/audit.log|grep avc After some minutes, some avc message is seen. type=AVC msg=audit(1382603882.299:82004): avc: denied { search } for pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir type=AVC msg=audit(1382603882.300:82005): avc: denied { search } for pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir type=AVC msg=audit(1382603882.300:82006): avc: denied { search } for pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir type=AVC msg=audit(1382603882.300:82007): avc: denied { getattr } for pid=31675 comm="ruby" path="/var/log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir type=AVC msg=audit(1382603882.301:82008): avc: denied { search } for pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir type=AVC msg=audit(1382603882.301:82009): avc: denied { search } for pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir type=AVC msg=audit(1382603882.301:82010): avc: denied { search } for pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir type=AVC msg=audit(1382603882.301:82011): avc: denied { getattr } for pid=31675 comm="ruby" path="/var/log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir The following process generate this message. root 31675 31672 0 01:38 ? 00:00:00 ruby /opt/rh/ruby193/root/usr/libexec/mcollective/update_yaml.rb /opt/rh/ruby193/root/etc/mcollective/facts.yaml Version-Release number of selected component (if applicable): 2.0/2013-10-21.3 libselinux-devel-2.0.94-5.3.el6_4.1.x86_64 libselinux-utils-2.0.94-5.3.el6_4.1.x86_64 selinux-policy-3.7.19-222.el6.noarch selinux-policy-targeted-3.7.19-222.el6.noarch libselinux-2.0.94-5.3.el6_4.1.x86_64 ruby193-ruby-selinux-2.0.94-3.el6op.x86_64 libselinux-python-2.0.94-5.3.el6_4.1.x86_64 How reproducible: Always Steps to Reproduce: 1.Setup openshift node env. 2.Monitor /var/log/audit/audit.log, and wait for a moment. # tailf /var/log/audit/audit.log|grep avc 3. Actual results: Some avc message is seen in the log. Expected results: There is no any avc message. Additional info:
We have this allow rule in Fedora allow openshift_cron_t var_log_t:dir { search getattr };
After I update my env to 2.0/2013-10-23.2 puddle include update all rhel packages, this issue disappeared.
I am seeing the same issue on: rhel 6.5 beta OSE 2.0 beta1 oo-exec-ruby ${PREFIX}/usr/libexec/mcollective/update_yaml.rb ${PREFIX}/etc/mcollective/facts.yaml &> /tmp/facts.log Nov 6 15:24:01 lae-node-1 kernel: type=1400 audit(1383773041.805:134): avc: denied { write } for pid=26402 comm="openshift-facts" name="facts.log" dev=dm-0 ino=532714 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:tmp_t:s0 tclass=file ruby193-mcollective-2.2.3-3.el6op.noarch ruby193-mcollective-common-2.2.3-3.el6op.noarch ruby193-mcollective-client-2.2.3-3.el6op.noarch openshift-origin-msg-node-mcollective-1.16.0-1.git.428.c2e3f5d.el6op.noarch rubygem-openshift-origin-container-selinux-0.3.0-1.git.193.95fbbc6.el6op.noarch libselinux-devel-2.0.94-5.3.el6_4.1.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 libselinux-python-2.0.94-5.3.el6_4.1.x86_64 libselinux-ruby-2.0.94-5.3.el6_4.1.x86_64 selinux-policy-targeted-3.7.19-222.el6.noarch selinux-policy-3.7.19-222.el6.noarch ruby193-ruby-selinux-2.0.94-3.el6op.x86_64 libselinux-utils-2.0.94-5.3.el6_4.1.x86_64 libselinux-2.0.94-5.3.el6_4.1.i686
This is a different issue. Where is "facts.log" located?
Like specified in the cron.minutely, its trying to write in the /tmp folder as /tmp/facts.log. Apparently crond cant write to it. I have to execute the script manually to get my openshift install up and running.
openshift_cron_t is not supposed to write generict tmp_t content. # sesearch -A -s openshift_cron_t -c file -p write
I see, but the openshift cron entry "${PREFIX}/etc/mcollective/facts.yaml &> /tmp/facts.log" needs the access to /tmp. If /tmp should not be write accessible for openshift_cron, shouldnt this entry then be changed?
On Fedora 21 I have. sesearch -A -s openshift_cron_t -c file -p write | grep tmp_t allow openshift_cron_t openshift_cron_tmp_t : file { ioctl read write create getattr setattr lock append unlink link rename open } ; Is openshift_cron_t running as a gear or UID=0?
It's being run as root by crond: Nov 11 13:52:01 CROND[1505]: (root) CMD (run-parts /etc/cron.minutely) Nov 11 13:52:01 run-parts[1505]: (/etc/cron.minutely) starting openshift-facts
This is the error message on /var/log/message every minute that the cron is trying to run. kernel: type=1400 audit(1384280461.045:14280): avc: denied { write } for pid=20544 comm="openshift-facts" name="facts.log" dev=dm-0 ino=532714 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:tmp_t:s0 tclass=file sesearch -A -s openshift_cron_t -c file -p write | grep tmp_t allow openshift_cron_t openshift_cron_tmp_t : file { ioctl read write create getattr setattr lock append unlink link rename open } ; CROND[20720]: (root) CMD (run-parts /etc/cron.minutely) sau-node-1 run-parts[20720]: (/etc/cron.minutely) starting openshift-facts
I spent some time looking through the git logs and previously this cronjob was redirecting output to /dev/null. The change to /tmp/facts.log was actually a recent mistake when a fix for another bug was addressed.
Tried it again with below versions and its resolved. rhel 6.5 beta OSE 2.0 beta2 (previous install was OSE 2.0 beta1) ruby193-mcollective-2.2.3-3.el6op.noarch ruby193-mcollective-common-2.2.3-3.el6op.noarch openshift-origin-msg-node-mcollective-1.17.0-1.git.71.15e0f7e.el6op.noarch libselinux-devel-2.0.94-5.3.el6_4.1.x86_64 ruby193-ruby-selinux-2.0.94-3.el6op.x86_64 rubygem-openshift-origin-container-selinux-0.4.0-1.git.236.de290c6.el6op.noarch libselinux-2.0.94-5.3.el6_4.1.x86_64 libselinux-python-2.0.94-5.3.el6_4.1.x86_64 selinux-policy-targeted-3.7.19-222.el6.noarch selinux-policy-3.7.19-222.el6.noarch libselinux-utils-2.0.94-5.3.el6_4.1.x86_64 libselinux-2.0.94-5.3.el6_4.1.i686 The beta1 version has /tmp/facts.log permissions set as: -rw-r--r--. root root system_u:object_r:tmp_t:s0 facts.log Beta2 /tmp/facts.log is set as: -rw-r--r--. root root system_u:object_r:openshift_cron_tmp_t:s0 facts.log
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/f1abe972794e35a4bfba597694ce829990f14d39 Bug 1022889 - AVC message is seen when mcolletive facts update cron job is running. Looking back at the history showed that the switch to writing to /tmp/facts.log was actually a mistake. I'm reverting that change which will address this bug.
Verified this but with openshift-origin-msg-node-mcollective-1.17.2-2.el6op.noarch, and PASS. # cat /etc/cron.minutely/openshift-facts #!/bin/bash PREFIX="" if [ -f /opt/rh/ruby193/root/usr/libexec/mcollective/update_yaml.rb ]; then PREFIX="/opt/rh/ruby193/root" fi oo-exec-ruby ${PREFIX}/usr/libexec/mcollective/update_yaml.rb ${PREFIX}/etc/mcollective/facts.yaml &> /dev/null Open two terminals, one is monitoring /var/log/cron, another one is monitoring /var/log/audit/audit.log. When cron job is finished, no any avc message is seen in /var/log/audit/audit.log # tailf /var/log/cron Nov 18 03:27:01 node1 CROND[14837]: (root) CMD (run-parts /etc/cron.minutely) Nov 18 03:27:01 node1 run-parts[14837]: (/etc/cron.minutely) starting openshift-facts Nov 18 03:27:04 node1 run-parts[15202]: (/etc/cron.minutely) finished openshift-facts Nov 18 03:27:04 node1 run-parts[14837]: (/etc/cron.minutely) starting openshift-origin-cron-minutely Nov 18 03:27:04 node1 run-parts[15236]: (/etc/cron.minutely) finished openshift-origin-cron-minutely # grep /var/log/audit/audit.log |grep avc <nothing>
ee4a1dbbce0b8207e29b325dabe9e86c2a3a0c57 fixes this in git. It adds a label to /var/log/openshift # semanage fcontext -a -t openshift_log_t "/var/log/openshift(/.*)?" # restorecon -R -v /var/log/openshift SHould do this for now. But You will still need a policy change to allow openshift_cron_t to append to openshift_log_t.
Should we clone this bug to have this policy added to a RHEL 6.5 Z-stream?
We need to have RHEL6.6 clone which needs to be fixed and then we can request RHEL6.5.z.
The issue mentioned in Comment 38 could be reproduced in puddle 2.2/2014-12-02.3 with RHEL6.6 When a gear lost its gear directory on node somehow, such avc message would appear in the log: ... type=AVC msg=audit(1417603923.207:6141): avc: denied { write } for pid=28055 comm="ruby" name="log" dev=devtmpfs ino=11347 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devlog_t:s0 tclass=sock_file
OpenShift Enterprise v2 has officially reached EoL. This product is no longer supported and bugs will be closed. Please look into the replacement enterprise-grade container option, OpenShift Container Platform v3. https://www.openshift.com/container-platform/ More information can be found here: https://access.redhat.com/support/policy/updates/openshift/