Bug 1003645 - cron jobs fail due to selinux context mismatch
cron jobs fail due to selinux context mismatch
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Containers (Show other bugs)
1.2.0
All Linux
high Severity high
: ---
: ---
Assigned To: Miciah Dashiel Butler Masters
libra bugs
:
Depends On: 988519
Blocks:
  Show dependency treegraph
 
Reported: 2013-09-02 10:36 EDT by Nikhil Mone
Modified: 2017-03-08 12 EST (History)
12 users (show)

See Also:
Fixed In Version: rubygem-openshift-origin-node-1.9.14.4-1
Doc Type: Bug Fix
Doc Text:
Due to bugs in /usr/bin/oo-scheduled-jobs commands that called upon “ oo_spawn”, for example 'gear' or 'ctl_app' commands did not work inside cron jobs. This was because the “user” part of SELinux context should be 'unconfined_u', but in cron it is 'system_u'. With this fix cron jobs now run as expected.
Story Points: ---
Clone Of: 988519
Environment:
Last Closed: 2013-09-25 11:31:05 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Nikhil Mone 2013-09-02 10:36:44 EDT
+++ This bug was initially created as a clone of Bug #988519 +++

Description of problem:

Commands which call "oo_spawn" (e.g., "gear" or "ctl_app" commands) do not work inside cron jobs.  The reason is that the "user" part of selinux context is expected to be "unconfined_u", but under cron it is "system_u"

Steps to Reproduce:
1. create an app which has the "cron" cartridge installed
2. add a script called .openshift/cron/minutely/restart.sh to your git repo
with these contents:

#!/bin/sh
gear stop
gear start

Actual results:

The script will fail with a "could not open session" error

Expected results:

The script should succeed

Additional info:

I tracked this down to the oo_spawn function in node/lib/openshift-origin-node/utils/shell_exec.rb

The code runs:

              current_context  = SELinux.getcon
              target_context   = SELinux.context_from_defaults(SELinux.get_mcs_label(options[:uid]))

and compares the two.  if they do not match, then it tries to spawn the specified command under runuser / runcon; with our PAM configuration, this is only allowed for root.  When a command fails in this way under cron, you'll see messages like this in /var/log/secure:

Jul 25 14:35:16 ex-std-nodeXX runuser: pam_unix(runuser:session): session opened for user XXXXXXXXX by (uid=XXXX)
Jul 25 14:35:16 ex-std-nodeXX runuser: pam_namespace(runuser:session): Error opening config file /etc/security/namespace.d/shm.conf
Jul 25 14:35:16 ex-std-nodeXX runuser: PAM audit_open() failed: Permission denied
Jul 25 14:35:16 ex-std-nodeXX runuser: pam_unix(runuser:session): session closed for user XXXXXXXXX

--- Additional comment from Andy Grimm on 2013-07-25 15:21:35 EDT ---

I don't know whether it's the _correct_ fix, but one possible fix would be to change the runcon invocation in daemon_as_user in oo-scheduled-jobs.  It currently has:

daemon --user="$uuid" runcon -r system_r -t openshift_t -l $mcs_level "$@"

which could be:

daemon --user="$uuid" runcon -u unconfined_u -r system_r -t openshift_t -l $mcs_level "$@"

--- Additional comment from Rob Millner on 2013-07-25 16:42:04 EDT ---

Implemented the above fix:
https://github.com/openshift/origin-server/pull/3173


Stopping gear...
CLIENT_RESULT: Stopping CRON cart
Waiting for stop to finish
Starting gear...
App container start_gear
CLIENT_RESULT: Starting CRON cart
__________________________________________________________________________
Thu Jul 25 16:40:11 EDT 2013: END minutely cron run - status=0
__________________________________________________________________________

--- Additional comment from openshift-github-bot on 2013-07-25 21:06:08 EDT ---

Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/d1f0ecc25c07ef8c1c9aaabc616b3e4353bedd22
Bug 988519 - Ensure that the gear task runs as unconfined_u.

--- Additional comment from chunchen on 2013-07-29 01:45:28 EDT ---

It's fixed, verified on devenv_3572, please refer to the following results:

1. create an app which has the "cron" cartridge installed

2. add a script called .openshift/cron/minutely/restart.sh to your git repo
with these contents and git push:
#!/bin/sh
gear stop
gear start

3. ssh into this app and check the cron log
\> cat ./cron/log/cron.minutely.log
Stopping gear...
CLIENT_RESULT: Stopping CRON cart
Waiting for stop to finish
Starting gear...
CLIENT_RESULT: Starting CRON cart
__________________________________________________________________________
Mon Jul 29 01:40:14 EDT 2013: END minutely cron run - status=0
__________________________________________________________________________
Comment 2 Miciah Dashiel Butler Masters 2013-09-04 13:41:07 EDT
Here is a pull request to cherry-pick the fix to origin-server:

https://github.com/openshift/enterprise-server/pull/129
Comment 3 Miciah Dashiel Butler Masters 2013-09-05 15:03:18 EDT
The fix has been merged into enterprise-server:


https://github.com/openshift/enterprise-server/commit/ab28ca44fe4bbd24660c7b3a7ccac8db83e8ee96
Comment 6 Gaoyun Pei 2013-09-06 04:29:28 EDT
Verify this bug on puddle: 1.2/2013-09-05.1

1. create a python app with cron-1.4 embedded.

2. add a script called .openshift/cron/minutely/restart.sh to your git repo
with these contents and git push:
#!/bin/sh
gear stop
gear start

3. ssh into this app and check the cron log
[python-11.osetestv2auto.com 52298faa6088676af6000023]\> cat ./cron/log/cron.minutely.log
__________________________________________________________________________
Fri Sep  6 04:25:05 EDT 2013: START minutely cron run
__________________________________________________________________________
/var/lib/openshift/52298faa6088676af6000023/app-root/runtime/repo//.openshift/cron/minutely/restart.sh:

Stopping gear...
CLIENT_RESULT: Stopping CRON cart
Stopping PYTHON cart
Waiting for stop to finish
Starting gear...
CLIENT_RESULT: Starting CRON cart
Starting PYTHON cart

__________________________________________________________________________
Fri Sep  6 04:25:28 EDT 2013: END minutely cron run - status=0
__________________________________________________________________________
Comment 9 errata-xmlrpc 2013-09-25 11:31:05 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1275.html

Note You need to log in before you can comment on or make changes to this bug.