Bug 1022889

We have this allow rule in Fedora

allow openshift_cron_t var_log_t:dir { search getattr };

After I update my env to 2.0/2013-10-23.2 puddle include update all rhel packages, this issue disappeared.

I am seeing the same issue on:
rhel 6.5 beta
OSE 2.0 beta1

oo-exec-ruby ${PREFIX}/usr/libexec/mcollective/update_yaml.rb ${PREFIX}/etc/mcollective/facts.yaml &> /tmp/facts.log

Nov  6 15:24:01 lae-node-1 kernel: type=1400 audit(1383773041.805:134): avc:  denied  { write } for  pid=26402 comm="openshift-facts" name="facts.log" dev=dm-0 ino=532714 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:tmp_t:s0 tclass=file

ruby193-mcollective-2.2.3-3.el6op.noarch
ruby193-mcollective-common-2.2.3-3.el6op.noarch
ruby193-mcollective-client-2.2.3-3.el6op.noarch
openshift-origin-msg-node-mcollective-1.16.0-1.git.428.c2e3f5d.el6op.noarch


rubygem-openshift-origin-container-selinux-0.3.0-1.git.193.95fbbc6.el6op.noarch
libselinux-devel-2.0.94-5.3.el6_4.1.x86_64
libselinux-2.0.94-5.3.el6_4.1.x86_64
libselinux-python-2.0.94-5.3.el6_4.1.x86_64
libselinux-ruby-2.0.94-5.3.el6_4.1.x86_64
selinux-policy-targeted-3.7.19-222.el6.noarch
selinux-policy-3.7.19-222.el6.noarch
ruby193-ruby-selinux-2.0.94-3.el6op.x86_64
libselinux-utils-2.0.94-5.3.el6_4.1.x86_64
libselinux-2.0.94-5.3.el6_4.1.i686

This is a different issue. Where is "facts.log" located?

Like specified in the cron.minutely, its trying to write in the /tmp folder as /tmp/facts.log. Apparently crond cant write to it. I have to execute the script manually to get my openshift install up and running.

openshift_cron_t is not supposed to write generict tmp_t content.

# sesearch -A -s openshift_cron_t -c file -p write

I see, but the openshift cron entry "${PREFIX}/etc/mcollective/facts.yaml &> /tmp/facts.log" needs the access to /tmp. If /tmp should not be write accessible for openshift_cron, shouldnt this entry then be changed?

On Fedora 21 I have.



sesearch -A -s openshift_cron_t -c file -p write | grep tmp_t
   allow openshift_cron_t openshift_cron_tmp_t : file { ioctl read write create getattr setattr lock append unlink link rename open } ; 


Is openshift_cron_t running as a gear or UID=0?

It's being run as root by crond:

Nov 11 13:52:01  CROND[1505]: (root) CMD (run-parts /etc/cron.minutely)
Nov 11 13:52:01 run-parts[1505]: (/etc/cron.minutely) starting openshift-facts

This is the error message on /var/log/message every minute that the cron is trying to run.
kernel: type=1400 audit(1384280461.045:14280): avc:  denied  { write } for  pid=20544 comm="openshift-facts" name="facts.log" dev=dm-0 ino=532714 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:tmp_t:s0 tclass=file


sesearch -A -s openshift_cron_t -c file -p write | grep tmp_t
   allow openshift_cron_t openshift_cron_tmp_t : file { ioctl read write create getattr setattr lock append unlink link rename open } ;


CROND[20720]: (root) CMD (run-parts /etc/cron.minutely)
sau-node-1 run-parts[20720]: (/etc/cron.minutely) starting openshift-facts

I spent some time looking through the git logs and previously this cronjob was redirecting output to /dev/null.  The change to /tmp/facts.log was actually a recent mistake when a fix for another bug was addressed.

Tried it again with below versions and its resolved.
rhel 6.5 beta
OSE 2.0 beta2 (previous install was OSE 2.0 beta1)

ruby193-mcollective-2.2.3-3.el6op.noarch
ruby193-mcollective-common-2.2.3-3.el6op.noarch
openshift-origin-msg-node-mcollective-1.17.0-1.git.71.15e0f7e.el6op.noarch

libselinux-devel-2.0.94-5.3.el6_4.1.x86_64
ruby193-ruby-selinux-2.0.94-3.el6op.x86_64
rubygem-openshift-origin-container-selinux-0.4.0-1.git.236.de290c6.el6op.noarch
libselinux-2.0.94-5.3.el6_4.1.x86_64
libselinux-python-2.0.94-5.3.el6_4.1.x86_64
selinux-policy-targeted-3.7.19-222.el6.noarch
selinux-policy-3.7.19-222.el6.noarch
libselinux-utils-2.0.94-5.3.el6_4.1.x86_64
libselinux-2.0.94-5.3.el6_4.1.i686

The beta1 version has /tmp/facts.log permissions set as:
-rw-r--r--. root root system_u:object_r:tmp_t:s0       facts.log

Beta2 /tmp/facts.log is set as:
-rw-r--r--. root root system_u:object_r:openshift_cron_tmp_t:s0 facts.log

Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/f1abe972794e35a4bfba597694ce829990f14d39
Bug 1022889 - AVC message is seen when mcolletive facts update cron job is running.

Looking back at the history showed that the switch to writing to /tmp/facts.log
was actually a mistake.  I'm reverting that change which will address this bug.

Verified this but with openshift-origin-msg-node-mcollective-1.17.2-2.el6op.noarch, and PASS.

# cat /etc/cron.minutely/openshift-facts 
#!/bin/bash

PREFIX=""

if [ -f /opt/rh/ruby193/root/usr/libexec/mcollective/update_yaml.rb ]; then
  PREFIX="/opt/rh/ruby193/root"
fi

oo-exec-ruby ${PREFIX}/usr/libexec/mcollective/update_yaml.rb ${PREFIX}/etc/mcollective/facts.yaml &> /dev/null

Open two terminals, one is monitoring /var/log/cron, another one is monitoring /var/log/audit/audit.log.


When cron job is finished, no any avc message is seen in /var/log/audit/audit.log
# tailf /var/log/cron
Nov 18 03:27:01 node1 CROND[14837]: (root) CMD (run-parts /etc/cron.minutely)
Nov 18 03:27:01 node1 run-parts[14837]: (/etc/cron.minutely) starting openshift-facts
Nov 18 03:27:04 node1 run-parts[15202]: (/etc/cron.minutely) finished openshift-facts
Nov 18 03:27:04 node1 run-parts[14837]: (/etc/cron.minutely) starting openshift-origin-cron-minutely
Nov 18 03:27:04 node1 run-parts[15236]: (/etc/cron.minutely) finished openshift-origin-cron-minutely
# grep /var/log/audit/audit.log |grep avc
<nothing>

ee4a1dbbce0b8207e29b325dabe9e86c2a3a0c57 fixes this in git.

It adds a label to /var/log/openshift

# semanage fcontext -a -t openshift_log_t "/var/log/openshift(/.*)?"
# restorecon -R -v /var/log/openshift

SHould do this for now.

But You will still need a policy change to allow openshift_cron_t to append to openshift_log_t.

Should we clone this bug to have this policy added to a RHEL 6.5 Z-stream?

We need to have RHEL6.6 clone which needs to be fixed and then we can request RHEL6.5.z.

The issue mentioned in Comment 38 could be reproduced in puddle 2.2/2014-12-02.3 with RHEL6.6

When a gear lost its gear directory on node somehow, such avc message would appear in the log:
...
type=AVC msg=audit(1417603923.207:6141): avc:  denied  { write } for  pid=28055 comm="ruby" name="log" dev=devtmpfs ino=11347 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devlog_t:s0 tclass=sock_file

OpenShift Enterprise v2 has officially reached EoL.  This product is no longer supported and bugs will be closed.

Please look into the replacement enterprise-grade container option, OpenShift Container Platform v3.  https://www.openshift.com/container-platform/

More information can be found here: https://access.redhat.com/support/policy/updates/openshift/