+++ This bug was initially created as a clone of Bug #988519 +++ Description of problem: Commands which call "oo_spawn" (e.g., "gear" or "ctl_app" commands) do not work inside cron jobs. The reason is that the "user" part of selinux context is expected to be "unconfined_u", but under cron it is "system_u" Steps to Reproduce: 1. create an app which has the "cron" cartridge installed 2. add a script called .openshift/cron/minutely/restart.sh to your git repo with these contents: #!/bin/sh gear stop gear start Actual results: The script will fail with a "could not open session" error Expected results: The script should succeed Additional info: I tracked this down to the oo_spawn function in node/lib/openshift-origin-node/utils/shell_exec.rb The code runs: current_context = SELinux.getcon target_context = SELinux.context_from_defaults(SELinux.get_mcs_label(options[:uid])) and compares the two. if they do not match, then it tries to spawn the specified command under runuser / runcon; with our PAM configuration, this is only allowed for root. When a command fails in this way under cron, you'll see messages like this in /var/log/secure: Jul 25 14:35:16 ex-std-nodeXX runuser: pam_unix(runuser:session): session opened for user XXXXXXXXX by (uid=XXXX) Jul 25 14:35:16 ex-std-nodeXX runuser: pam_namespace(runuser:session): Error opening config file /etc/security/namespace.d/shm.conf Jul 25 14:35:16 ex-std-nodeXX runuser: PAM audit_open() failed: Permission denied Jul 25 14:35:16 ex-std-nodeXX runuser: pam_unix(runuser:session): session closed for user XXXXXXXXX --- Additional comment from Andy Grimm on 2013-07-25 15:21:35 EDT --- I don't know whether it's the _correct_ fix, but one possible fix would be to change the runcon invocation in daemon_as_user in oo-scheduled-jobs. It currently has: daemon --user="$uuid" runcon -r system_r -t openshift_t -l $mcs_level "$@" which could be: daemon --user="$uuid" runcon -u unconfined_u -r system_r -t openshift_t -l $mcs_level "$@" --- Additional comment from Rob Millner on 2013-07-25 16:42:04 EDT --- Implemented the above fix: https://github.com/openshift/origin-server/pull/3173 Stopping gear... CLIENT_RESULT: Stopping CRON cart Waiting for stop to finish Starting gear... App container start_gear CLIENT_RESULT: Starting CRON cart __________________________________________________________________________ Thu Jul 25 16:40:11 EDT 2013: END minutely cron run - status=0 __________________________________________________________________________ --- Additional comment from openshift-github-bot on 2013-07-25 21:06:08 EDT --- Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/d1f0ecc25c07ef8c1c9aaabc616b3e4353bedd22 Bug 988519 - Ensure that the gear task runs as unconfined_u. --- Additional comment from chunchen on 2013-07-29 01:45:28 EDT --- It's fixed, verified on devenv_3572, please refer to the following results: 1. create an app which has the "cron" cartridge installed 2. add a script called .openshift/cron/minutely/restart.sh to your git repo with these contents and git push: #!/bin/sh gear stop gear start 3. ssh into this app and check the cron log \> cat ./cron/log/cron.minutely.log Stopping gear... CLIENT_RESULT: Stopping CRON cart Waiting for stop to finish Starting gear... CLIENT_RESULT: Starting CRON cart __________________________________________________________________________ Mon Jul 29 01:40:14 EDT 2013: END minutely cron run - status=0 __________________________________________________________________________
Here is a pull request to cherry-pick the fix to origin-server: https://github.com/openshift/enterprise-server/pull/129
The fix has been merged into enterprise-server: https://github.com/openshift/enterprise-server/commit/ab28ca44fe4bbd24660c7b3a7ccac8db83e8ee96
Verify this bug on puddle: 1.2/2013-09-05.1 1. create a python app with cron-1.4 embedded. 2. add a script called .openshift/cron/minutely/restart.sh to your git repo with these contents and git push: #!/bin/sh gear stop gear start 3. ssh into this app and check the cron log [python-11.osetestv2auto.com 52298faa6088676af6000023]\> cat ./cron/log/cron.minutely.log __________________________________________________________________________ Fri Sep 6 04:25:05 EDT 2013: START minutely cron run __________________________________________________________________________ /var/lib/openshift/52298faa6088676af6000023/app-root/runtime/repo//.openshift/cron/minutely/restart.sh: Stopping gear... CLIENT_RESULT: Stopping CRON cart Stopping PYTHON cart Waiting for stop to finish Starting gear... CLIENT_RESULT: Starting CRON cart Starting PYTHON cart __________________________________________________________________________ Fri Sep 6 04:25:28 EDT 2013: END minutely cron run - status=0 __________________________________________________________________________
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-1275.html