Bug 1022889 - AVC message is seen when mcolletive facts update cron job is running.
AVC message is seen when mcolletive facts update cron job is running.
Status: CLOSED EOL
Product: OpenShift Container Platform
Classification: Red Hat
Component: Containers (Show other bugs)
2.2.0
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Brenton Leanhardt
libra bugs
: Reopened
Depends On: 1034206
Blocks: CVE-2013-4561
  Show dependency treegraph
 
Reported: 2013-10-24 04:57 EDT by Johnny Liu
Modified: 2017-01-13 17:12 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1034206 (view as bug list)
Environment:
Last Closed: 2017-01-13 17:12:47 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Johnny Liu 2013-10-24 04:57:48 EDT
Description of problem:
In openshift node, there is a cron job to update mcollective facts.
# cat /etc/cron.minutely/openshift-facts 
#!/bin/bash

PREFIX=""

if [ -f /opt/rh/ruby193/root/usr/libexec/mcollective/update_yaml.rb ]; then
  PREFIX="/opt/rh/ruby193/root"
fi

oo-exec-ruby ${PREFIX}/usr/libexec/mcollective/update_yaml.rb ${PREFIX}/etc/mcollective/facts.yaml &> /tmp/facts.log


Monitor /var/log/audit/audit.log, and wait for a moment.
# tailf /var/log/audit/audit.log|grep avc

After some minutes, some avc message is seen.
type=AVC msg=audit(1382603882.299:82004): avc:  denied  { search } for  pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir
type=AVC msg=audit(1382603882.300:82005): avc:  denied  { search } for  pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir
type=AVC msg=audit(1382603882.300:82006): avc:  denied  { search } for  pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir
type=AVC msg=audit(1382603882.300:82007): avc:  denied  { getattr } for  pid=31675 comm="ruby" path="/var/log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir
type=AVC msg=audit(1382603882.301:82008): avc:  denied  { search } for  pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir
type=AVC msg=audit(1382603882.301:82009): avc:  denied  { search } for  pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir
type=AVC msg=audit(1382603882.301:82010): avc:  denied  { search } for  pid=31675 comm="ruby" name="log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir
type=AVC msg=audit(1382603882.301:82011): avc:  denied  { getattr } for  pid=31675 comm="ruby" path="/var/log" dev=dm-0 ino=682 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=dir


The following process generate this message.
root     31675 31672  0 01:38 ?        00:00:00 ruby /opt/rh/ruby193/root/usr/libexec/mcollective/update_yaml.rb /opt/rh/ruby193/root/etc/mcollective/facts.yaml


Version-Release number of selected component (if applicable):
2.0/2013-10-21.3
libselinux-devel-2.0.94-5.3.el6_4.1.x86_64
libselinux-utils-2.0.94-5.3.el6_4.1.x86_64
selinux-policy-3.7.19-222.el6.noarch
selinux-policy-targeted-3.7.19-222.el6.noarch
libselinux-2.0.94-5.3.el6_4.1.x86_64
ruby193-ruby-selinux-2.0.94-3.el6op.x86_64
libselinux-python-2.0.94-5.3.el6_4.1.x86_64


How reproducible:
Always

Steps to Reproduce:
1.Setup openshift node env.
2.Monitor /var/log/audit/audit.log, and wait for a moment.
# tailf /var/log/audit/audit.log|grep avc
3.

Actual results:
Some avc message is seen in the log.


Expected results:
There is no any avc message.

Additional info:
Comment 5 Simon Sekidde 2013-10-28 09:52:28 EDT
We have this allow rule in Fedora

allow openshift_cron_t var_log_t:dir { search getattr };
Comment 14 Johnny Liu 2013-10-29 04:45:13 EDT
After I update my env to 2.0/2013-10-23.2 puddle include update all rhel packages, this issue disappeared.
Comment 15 saurabh sharma 2013-11-07 14:00:27 EST
I am seeing the same issue on:
rhel 6.5 beta
OSE 2.0 beta1

oo-exec-ruby ${PREFIX}/usr/libexec/mcollective/update_yaml.rb ${PREFIX}/etc/mcollective/facts.yaml &> /tmp/facts.log

Nov  6 15:24:01 lae-node-1 kernel: type=1400 audit(1383773041.805:134): avc:  denied  { write } for  pid=26402 comm="openshift-facts" name="facts.log" dev=dm-0 ino=532714 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:tmp_t:s0 tclass=file

ruby193-mcollective-2.2.3-3.el6op.noarch
ruby193-mcollective-common-2.2.3-3.el6op.noarch
ruby193-mcollective-client-2.2.3-3.el6op.noarch
openshift-origin-msg-node-mcollective-1.16.0-1.git.428.c2e3f5d.el6op.noarch


rubygem-openshift-origin-container-selinux-0.3.0-1.git.193.95fbbc6.el6op.noarch
libselinux-devel-2.0.94-5.3.el6_4.1.x86_64
libselinux-2.0.94-5.3.el6_4.1.x86_64
libselinux-python-2.0.94-5.3.el6_4.1.x86_64
libselinux-ruby-2.0.94-5.3.el6_4.1.x86_64
selinux-policy-targeted-3.7.19-222.el6.noarch
selinux-policy-3.7.19-222.el6.noarch
ruby193-ruby-selinux-2.0.94-3.el6op.x86_64
libselinux-utils-2.0.94-5.3.el6_4.1.x86_64
libselinux-2.0.94-5.3.el6_4.1.i686
Comment 16 Miroslav Grepl 2013-11-08 03:23:28 EST
This is a different issue. Where is "facts.log" located?
Comment 17 saurabh sharma 2013-11-08 03:29:13 EST
Like specified in the cron.minutely, its trying to write in the /tmp folder as /tmp/facts.log. Apparently crond cant write to it. I have to execute the script manually to get my openshift install up and running.
Comment 18 Miroslav Grepl 2013-11-11 08:07:09 EST
openshift_cron_t is not supposed to write generict tmp_t content.

# sesearch -A -s openshift_cron_t -c file -p write
Comment 19 saurabh sharma 2013-11-11 13:16:54 EST
I see, but the openshift cron entry "${PREFIX}/etc/mcollective/facts.yaml &> /tmp/facts.log" needs the access to /tmp. If /tmp should not be write accessible for openshift_cron, shouldnt this entry then be changed?
Comment 20 Daniel Walsh 2013-11-11 13:38:38 EST
On Fedora 21 I have.



sesearch -A -s openshift_cron_t -c file -p write | grep tmp_t
   allow openshift_cron_t openshift_cron_tmp_t : file { ioctl read write create getattr setattr lock append unlink link rename open } ; 


Is openshift_cron_t running as a gear or UID=0?
Comment 21 Brenton Leanhardt 2013-11-11 13:53:38 EST
It's being run as root by crond:

Nov 11 13:52:01  CROND[1505]: (root) CMD (run-parts /etc/cron.minutely)
Nov 11 13:52:01 run-parts[1505]: (/etc/cron.minutely) starting openshift-facts
Comment 28 saurabh sharma 2013-11-12 13:26:33 EST
This is the error message on /var/log/message every minute that the cron is trying to run.
kernel: type=1400 audit(1384280461.045:14280): avc:  denied  { write } for  pid=20544 comm="openshift-facts" name="facts.log" dev=dm-0 ino=532714 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:tmp_t:s0 tclass=file


sesearch -A -s openshift_cron_t -c file -p write | grep tmp_t
   allow openshift_cron_t openshift_cron_tmp_t : file { ioctl read write create getattr setattr lock append unlink link rename open } ;


CROND[20720]: (root) CMD (run-parts /etc/cron.minutely)
sau-node-1 run-parts[20720]: (/etc/cron.minutely) starting openshift-facts
Comment 29 Brenton Leanhardt 2013-11-12 14:24:56 EST
I spent some time looking through the git logs and previously this cronjob was redirecting output to /dev/null.  The change to /tmp/facts.log was actually a recent mistake when a fix for another bug was addressed.
Comment 30 saurabh sharma 2013-11-12 14:37:31 EST
Tried it again with below versions and its resolved.
rhel 6.5 beta
OSE 2.0 beta2 (previous install was OSE 2.0 beta1)

ruby193-mcollective-2.2.3-3.el6op.noarch
ruby193-mcollective-common-2.2.3-3.el6op.noarch
openshift-origin-msg-node-mcollective-1.17.0-1.git.71.15e0f7e.el6op.noarch

libselinux-devel-2.0.94-5.3.el6_4.1.x86_64
ruby193-ruby-selinux-2.0.94-3.el6op.x86_64
rubygem-openshift-origin-container-selinux-0.4.0-1.git.236.de290c6.el6op.noarch
libselinux-2.0.94-5.3.el6_4.1.x86_64
libselinux-python-2.0.94-5.3.el6_4.1.x86_64
selinux-policy-targeted-3.7.19-222.el6.noarch
selinux-policy-3.7.19-222.el6.noarch
libselinux-utils-2.0.94-5.3.el6_4.1.x86_64
libselinux-2.0.94-5.3.el6_4.1.i686

The beta1 version has /tmp/facts.log permissions set as:
-rw-r--r--. root root system_u:object_r:tmp_t:s0       facts.log

Beta2 /tmp/facts.log is set as:
-rw-r--r--. root root system_u:object_r:openshift_cron_tmp_t:s0 facts.log
Comment 32 openshift-github-bot 2013-11-12 16:24:44 EST
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/f1abe972794e35a4bfba597694ce829990f14d39
Bug 1022889 - AVC message is seen when mcolletive facts update cron job is running.

Looking back at the history showed that the switch to writing to /tmp/facts.log
was actually a mistake.  I'm reverting that change which will address this bug.
Comment 33 Johnny Liu 2013-11-18 03:30:09 EST
Verified this but with openshift-origin-msg-node-mcollective-1.17.2-2.el6op.noarch, and PASS.

# cat /etc/cron.minutely/openshift-facts 
#!/bin/bash

PREFIX=""

if [ -f /opt/rh/ruby193/root/usr/libexec/mcollective/update_yaml.rb ]; then
  PREFIX="/opt/rh/ruby193/root"
fi

oo-exec-ruby ${PREFIX}/usr/libexec/mcollective/update_yaml.rb ${PREFIX}/etc/mcollective/facts.yaml &> /dev/null

Open two terminals, one is monitoring /var/log/cron, another one is monitoring /var/log/audit/audit.log.


When cron job is finished, no any avc message is seen in /var/log/audit/audit.log
# tailf /var/log/cron
Nov 18 03:27:01 node1 CROND[14837]: (root) CMD (run-parts /etc/cron.minutely)
Nov 18 03:27:01 node1 run-parts[14837]: (/etc/cron.minutely) starting openshift-facts
Nov 18 03:27:04 node1 run-parts[15202]: (/etc/cron.minutely) finished openshift-facts
Nov 18 03:27:04 node1 run-parts[14837]: (/etc/cron.minutely) starting openshift-origin-cron-minutely
Nov 18 03:27:04 node1 run-parts[15236]: (/etc/cron.minutely) finished openshift-origin-cron-minutely
# grep /var/log/audit/audit.log |grep avc
<nothing>
Comment 40 Daniel Walsh 2013-11-20 15:14:03 EST
ee4a1dbbce0b8207e29b325dabe9e86c2a3a0c57 fixes this in git.

It adds a label to /var/log/openshift

# semanage fcontext -a -t openshift_log_t "/var/log/openshift(/.*)?"
# restorecon -R -v /var/log/openshift

SHould do this for now.

But You will still need a policy change to allow openshift_cron_t to append to openshift_log_t.
Comment 41 Brenton Leanhardt 2013-11-23 09:33:19 EST
Should we clone this bug to have this policy added to a RHEL 6.5 Z-stream?
Comment 42 Miroslav Grepl 2013-11-25 07:19:32 EST
We need to have RHEL6.6 clone which needs to be fixed and then we can request RHEL6.5.z.
Comment 44 Gaoyun Pei 2014-12-03 05:55:56 EST
The issue mentioned in Comment 38 could be reproduced in puddle 2.2/2014-12-02.3 with RHEL6.6

When a gear lost its gear directory on node somehow, such avc message would appear in the log:
...
type=AVC msg=audit(1417603923.207:6141): avc:  denied  { write } for  pid=28055 comm="ruby" name="log" dev=devtmpfs ino=11347 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devlog_t:s0 tclass=sock_file
Comment 45 Rory Thrasher 2017-01-13 17:12:47 EST
OpenShift Enterprise v2 has officially reached EoL.  This product is no longer supported and bugs will be closed.

Please look into the replacement enterprise-grade container option, OpenShift Container Platform v3.  https://www.openshift.com/container-platform/

More information can be found here: https://access.redhat.com/support/policy/updates/openshift/

Note You need to log in before you can comment on or make changes to this bug.