Bug 912215 - builds fail on jboss with oo-ruby: Permission denied [was: open4 LoadError continuously seen when oo-cgroup-read action is performed]
Summary: builds fail on jboss with oo-ruby: Permission denied [was: open4 LoadError co...
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: OKD
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Rob Millner
QA Contact: libra bugs
URL:
Whiteboard:
: 915370 (view as bug list)
Depends On:
Blocks: 924556
TreeView+ depends on / blocked
 
Reported: 2013-02-18 06:40 UTC by Jianwei Hou
Modified: 2015-05-14 23:05 UTC (History)
10 users (show)

Fixed In Version: devenv_2826+
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-03-09 02:32:42 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 915370 0 high CLOSED oo-cgroup-read fails, generates AVC 2021-02-22 00:41:40 UTC

Internal Links: 915370

Description Jianwei Hou 2013-02-18 06:40:39 UTC
Description of problem:
When performing oo-cgroup-read, the "open4 (LoadError)" is repeatedly seen. This in turn blocked many operations that calls "oo-cgroup-read", such as building a jbossas application, performing snapshot and restore.

Version-Release number of selected component (if applicable):
On devenv_2823

How reproducible:
Always

Steps to Reproduce:
1. Try building a jbossas application
2. run 'oo-cgroup-read"

  
Actual results:
After step1: 
remote: /usr/bin/oo-cgroup-read:7:in `require': no such file to load -- open4 (LoadError)
remote:         from /usr/bin/oo-cgroup-read:7
remote: expr: syntax error
remote: (standard_in) 1: syntax error
remote: Sourcing pre_build_jbossas-7
remote: Found pom.xml... attempting to build with 'mvn -e clean package -Popenshift -DskipTests'
remote: Invalid maximum heap size: -Xmxm
remote: Error: Could not create the Java Virtual Machine.
remote: Error: A fatal exception has occurred. Program will exit.

After step2:
[root@ip-10-202-49-211 ~]# oo-cgroup-read 
/usr/bin/oo-cgroup-read:7:in `require': no such file to load -- open4 (LoadError)
        from /usr/bin/oo-cgroup-read:7


Expected results:
The LoadError is not expected.


Additional info:

Comment 1 Dan Mace 2013-02-18 14:56:26 UTC
Resolved by: https://github.com/openshift/origin-server/pull/1392

Comment 2 openshift-github-bot 2013-02-18 15:49:23 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/2b6e149517902f29314d6bd784b449262abeeb39
Bug 912215: Use oo-ruby for interpreter

Comment 3 Jianwei Hou 2013-02-19 02:09:51 UTC
Tested on devenv_2829

Now, the LoadError is gone, the code uses oo-ruby instead of ruby to resolve the problem.
But SELinux issues arose. The gear owner can not perform oo-cgroup-read

[jbossas1-2829t.dev.rhcloud.com ~]\> oo-cgroup-read memory.memsw.limit_in_bytes
/usr/bin/env: oo-ruby: Permission denied

[root@ip-10-151-84-246 ~]# ll -Z /usr/bin/oo-cgroup-read
-rwxr-xr-x. root root system_u:object_r:openshift_cgroup_read_exec_t:s0 /usr/bin/oo-cgroup-read


Can not build a jbossas app due to this error:
remote: Running .openshift/action_hooks/pre_build
remote: /usr/bin/env: oo-ruby: Permission denied
remote: expr: syntax error
remote: (standard_in) 1: syntax error
remote: Sourcing pre_build_jbossas-7
remote: Found pom.xml... attempting to build with 'mvn -e clean package -Popenshift -DskipTests'
remote: Invalid maximum heap size: -Xmxm
remote: Error: Could not create the Java Virtual Machine.
remote: Error: A fatal exception has occurred. Program will exit.
remote: Invalid maximum heap size: -Xmxm
remote: Error: Could not create the Java Virtual Machine.
remote: Error: A fatal exception has occurred. Program will exit.
remote: Running .openshift/action_hooks/build
remote: Running .openshift/action_hooks/deploy
remote: hot_deploy_added=false

Comment 4 Mark Lamourine 2013-02-19 17:41:08 UTC
The recent change added a call out to cgget, where before it used a file read.
This requires that the script be able to execute an external program (cgget).

I think we need the rule described below added to the openshift-origin policy.

jhonce got us this diagnostic:

type=AVC msg=audit(1361291623.691:128914): avc:  denied  { execute } for  pid=4623 comm="oo-cgroup-read" name="bash" dev=xvde1 ino=29 scontext=unconfined_u:system_r:openshift_cgroup_read_t:s0:c0,c501 tcontext=system_u:object_r:shell_exec_t:s0 tclass=file
        Was caused by:
                Missing type enforcement (TE) allow rule.

                You can use audit2allow to generate a loadable module to allow this access.


#============= openshift_cgroup_read_t ==============
allow openshift_cgroup_read_t shell_exec_t:file execute;

Comment 5 Zhe Wang 2013-02-20 05:33:12 UTC
Reproduced this bug in devenv_2836.

Jenkins build for jboss apps (jbossap, jbosseap, jbossews) also failed.

Comment 6 Zhe Wang 2013-02-25 06:39:41 UTC
It seems that this bug only affects building an app only (such as git-pushing some changes), and there were no problems for saving and restoring db cartridges embedded into jboss apps (ap, eap, ews).

Comment 7 Mengjiao Gao 2013-02-25 10:43:46 UTC
When tested quickstarts on devenv_2855 with following steps, found that Capedwraf, Spring, and JEE FULL PROFILE ON JBOSE cannot work well. These quickstarts are all based on jboss eap,and jboss as. 

Steps to Reproduce:
1.Go to <server>/app/console
2.Create an app of Capedwraf, Spring, or JEE FULL PROFILE ON JBOSE
3.Check the app url
  
Actual results:
3.At first, I get a 503 service temporarily unavailable, after several minutes, get 404 not found error.

Expected results:
3.The app page should be shown correctly .

Comment 8 Rob Millner 2013-02-26 20:02:03 UTC
*** Bug 915370 has been marked as a duplicate of this bug. ***

Comment 10 Rob Millner 2013-02-27 00:31:30 UTC
Waiting on the updated OpenShift policy for RHEL 6 and Fedora 18.

Comment 11 Xiaoli Tian 2013-02-27 09:24:40 UTC
Most jboss related cases are blocked by this bug.

Comment 12 zhaozhanqi 2013-03-01 06:10:54 UTC
Reproduced this bug on INT:

create a jbossas app, do some change, and then git push:


test@ubuntu:~/20130301/zqjbossas/src/main/webapp$ git push
Counting objects: 11, done.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 427 bytes, done.
Total 6 (delta 4), reused 0 (delta 0)
remote: restart_on_add=false
remote: Sending SIGTERM to jboss:4515 ...
remote: kill -TERM 4792
remote: kill -TERM 4515
remote: Done
remote: restart_on_add=false
remote: Running .openshift/action_hooks/pre_build
remote: /usr/bin/env: oo-ruby: Permission denied
remote: expr: syntax error
remote: (standard_in) 1: syntax error
remote: Sourcing pre_build_jbossas-7
remote: Found pom.xml... attempting to build with 'mvn -e clean package -Popenshift -DskipTests'
remote: Invalid maximum heap size: -Xmxm
remote: Error: Could not create the Java Virtual Machine.
remote: Error: A fatal exception has occurred. Program will exit.
remote: Invalid maximum heap size: -Xmxm
remote: Error: Could not create the Java Virtual Machine.
remote: Error: A fatal exception has occurred. Program will exit.
remote: Running .openshift/action_hooks/build
remote: Running .openshift/action_hooks/deploy
remote: hot_deploy_added=false
remote: Found 127.3.222.129:8080 listening port
remote: Done
remote: Running .openshift/action_hooks/post_deploy
To ssh://5130424a03ef647002000282.rhcloud.com/~/git/zqjbossas.git/
   19991a7..1a3383a  master -> master

Comment 13 Rob Millner 2013-03-01 22:04:04 UTC
Worked around the SELinux policy issues.

https://github.com/openshift/origin-server/pull/1511

Comment 14 Rob Millner 2013-03-01 22:37:21 UTC
Rerouted to pull request:
https://github.com/openshift/origin-server/pull/1514

Comment 15 openshift-github-bot 2013-03-02 03:31:15 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/a3fa114a24065df3450053329676ddaf80843d39
Bug 912215 - Workaround broken SELinux policy.

Comment 16 Meng Bo 2013-03-04 06:13:55 UTC
Checked on devenv_2888, jboss apps can be built now successfully.

Checked for all the jboss app types. All the cartridges can be built and can be accessed after built.

Mark bug as verified.

Comment 17 Luke Meyer 2013-03-25 15:26:50 UTC
Should we consider reopening or cloning this so that the SELinux policy gets fixed, rather than worked around with a somewhat fragile hack? (see bug 924556)


Note You need to log in before you can comment on or make changes to this bug.