Created attachment 932800 [details] Example Steps from customer Description of problem: We have installed OpenShift Enterprise (2.1) as part of a supported proof of concept project. I have performed a manual installation using the process deocumented in the deployment guide ; https://access.redhat.com/documentation/en-US/OpenShift_Enterprise/2/html/Deployment_Guide/index.html. The installtion looks to be working, however when switching on SELinux to Enforcing mode on the gear node we are seeing the following AVC denied messages logged when a new gear is created ; type=AVC msg=audit(1405701881.059:866): avc: denied { read } for pid=12638 comm="httpd" path="/tmp/systemu_lonlx90602_1507_12620_0.4908402953688549_1/stdin" dev=sda2 ino=802503 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:openshift_initrc_tmp_t:s0 tclass=file type=AVC msg=audit(1405701881.117:867): avc: denied { read } for pid=12639 comm="httpd" path="/tmp/systemu_lonlx90602_1507_12620_0.4908402953688549_1/stdin" dev=sda2 ino=802503 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:openshift_initrc_tmp_t:s0 tclass=file type=AVC msg=audit(1405701881.117:867): avc: denied { write } for pid=12639 comm="httpd" path="/tmp/systemu_lonlx90602_1507_12620_0.4908402953688549_1/stdout" dev=sda2 ino=802505 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:openshift_initrc_tmp_t:s0 tclass=file type=AVC msg=audit(1405701881.117:867): avc: denied { write } for pid=12639 comm="httpd" path="/tmp/systemu_lonlx90602_1507_12620_0.4908402953688549_1/stderr" dev=sda2 ino=802506 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:openshift_initrc_tmp_t:s0 tclass=file The gear is still created and looks to be functioning however we need to understand if there is some SELinux mis-configuration which needs correcting. oo-accept-node seems to Pass. oo-diagnostics generates no errors (4 warnings which don't seem to be assocaited with SE Linux permissions). Version-Release number of selected component (if applicable): ruby193-ruby-selinux-2.0.94-3.el6op.x86_64 rubygem-openshift-origin-container-selinux-0.8.1.1-1.el6op.noarch selinux-policy-3.7.19-231.el6_5.3.noarch How reproducible: (see attachment) Actual results: $ cat ausearch_-m_avc_user_avc_-ts_today ---- time->Tue Aug 12 13:05:08 2014 type=USER_AVC msg=audit(1407845108.059:200353): user pid=2084 uid=28 auid=4294967295 ses=4294967295 subj=system_u:system_r:nscd_t:s0 msg='avc: received setenforce notice (enforcing=0) exe="/usr/sbin/nscd" sauid=28 hostname=? addr=? terminal=?' ---- time->Tue Aug 12 13:09:42 2014 type=SYSCALL msg=audit(1407845382.706:200403): arch=c000003e syscall=59 success=yes exit=0 a0=8ef350 a1=8ef3c0 a2=8ec9a0 a3=18 items=0 ppid=27718 pid=27722 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="httpd" exe="/usr/sbin/httpd" subj=system_u:system_r:httpd_t:s0 key=(null) type=AVC msg=audit(1407845382.706:200403): avc: denied { read } for pid=27722 comm="httpd" path="/tmp/systemu_lonlx80069_1886_27704_0.17922529204064752_1/stdin" dev=sda3 ino=50347 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:openshift_initrc_tmp_t:s0 tclass=file ---- time->Tue Aug 12 13:09:42 2014 type=SYSCALL msg=audit(1407845382.779:200404): arch=c000003e syscall=59 success=yes exit=0 a0=8ef5f0 a1=8e9cf0 a2=8ec9a0 a3=20 items=0 ppid=27718 pid=27725 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="httpd" exe="/usr/sbin/httpd" subj=system_u:system_r:httpd_t:s0 key=(null) type=AVC msg=audit(1407845382.779:200404): avc: denied { write } for pid=27725 comm="httpd" path="/tmp/systemu_lonlx80069_1886_27704_0.17922529204064752_1/stdout" dev=sda3 ino=50350 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:openshift_initrc_tmp_t:s0 tclass=file Expected results: No AVC's
The /tmp/systemu* files are likely being created by that library. There are two tools that use systemu in OpenShift Enterprise. One is MCollective and the other is called oo-httpd-singular. I'm guessing somehow the latter is involved since we see exe="/usr/sbin/httpd" in the AVC mentioned above. What version of the following packages are you running?: openshift-origin-node-util ruby193-rubygem-systemu Can you verify that no other ruby193 systemu gems are installed? One way to do this is: scl enable ruby193 "gem list systemu" rpm -qV ruby193-rubygem-systemu
I was able to figure out how to reproduce this. It happens whenever rubygem-openshift-origin-frontend-apache-vhost is in use. The rubygem-openshift-origin-frontend-apache-mod-rewrite plugin is the default an is probably what most people who were trying to reproduce this bug were using. So people are in the loop, here's the chain of events that lead to this: 1) Broker submits an application create message to the Node 2) Node (mcollective) is running one of the apache frontend plugins 3) The frontend plugin calls /usr/sbin/oo-httpd-singular to gracefully restart apache 4) Apache restarts #3 only happens when the vhost plugin is in use. For the rewrite plugin it's not needed on application create. One workaround it to replace the use of systemu in that script with another library. We already have openshift-origin-node/utils/shell_exec so I will propose we switch to that. If you would like to try out a modified version in a non-production environment I will attach it to this bug.
Created attachment 934631 [details] /usr/sbin/oo-httpd-singular For non-production use only. You can overwrite the default contents of /usr/sbin/oo-httpd-singular with this file.
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/06a5cdb5fa50188a014cf4b672ef776bb0d97a5e Bug 1135617 - AVC denied messages when creating new gears
Recreated on ose-2.1 puddle-2-1-2014-09-04, and verified puddle-2-1-2014-09-09 Recreated steps. 1) setenforce 0 2) rhc app-create supportcase jbosseap-6 3) grep -i denied audit.log [root@node1 audit]# grep -i denied audit.log type=AVC msg=audit(1410329916.366:9711): avc: denied { read } for pid=18524 comm="httpd" path="/tmp/systemu_node1.ose21z-manual.com.cn_1121_18506_0.42876805572889265_1/stdin" dev=dm-0 ino=145617 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:openshift_initrc_tmp_t:s0 tclass=file type=AVC msg=audit(1410329916.415:9712): avc: denied { write } for pid=18525 comm="httpd" path="/tmp/systemu_node1.ose21z-manual.com.cn_1121_18506_0.42876805572889265_1/stdout" dev=dm-0 ino=145618 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:openshift_initrc_tmp_t:s0 tclass=file Verified steps: 1) setenforce 0 2) rhc app-create supportcase jbosseap-6 3) grep -i denied audit.log No denied message can be found. 4) rhc app restart supportcase and check audit.log No denied message can be found
*** Bug 1134842 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-1353.html