Description of problem: When app symlink is created, openshift cron job will generate avc denied message. Version-Release number of selected component (if applicable): selinux-policy-3.7.19-195.el6_4.4.noarch selinux-policy-targeted-3.7.19-195.el6_4.4.noarch 1.2/2013-05-14.1 How reproducible: Always Steps to Reproduce: 1.Setup ose env 2.Modify /etc/openshift/node.conf, set CREATE_APP_SYMLINKS=1 3.Create an app, a symlink will be created for this app in /var/lib/openshift 4.Run the following command to monitor avc message on node. # tailf /var/log/audit/audit.log|grep avc Actual results: The following avc error message is seen: type=AVC msg=audit(1368606181.591:538): avc: denied { read } for pid=12677 comm="ruby" name="jbossewstest-jialiu" dev=dm-0 ino=147917 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:openshift_var_lib_t:s0 tclass=lnk_file type=AVC msg=audit(1368606181.592:539): avc: denied { read } for pid=12677 comm="ruby" name="hotdeploywtau-sun005" dev=dm-0 ino=141383 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:openshift_var_lib_t:s0 tclass=lnk_file Expected results: No error message is seen. Additional info:
This happens because the updates /etc/cron.minutely/openshift-facts loads facter. If CREATE_APP_SYMLINKS is enabled the following code in /opt/rh/ruby193/root/usr/share/ruby/vendor_ruby/facter/openshift_facts.rb will trigger the denial: 70 Dir.glob("/var/lib/openshift/*").each do |app_dir| 71 if File.directory?(app_dir) && !File.symlink?(app_dir) 72 git_repos_count += Dir.glob(File.join(app_dir, "git/*.git")).count 73 The cronjob tries to read the symlink and it can't. The result is that the gear related facts will always be 0: [root@broker ~]# mco inventory node1.example.com | grep active active_capacity => 0.0 district_active => true gears_active_count => 0 gears_active_usage_pct => 0.0 max_active_gears => 100 Putting SELinux into permissive mode results in the gears_active_count cat audit.log | audit2allow -R require { type openshift_cron_t; type openshift_var_lib_t; class capability dac_override; class lnk_file { read getattr }; } #============= openshift_cron_t ============== allow openshift_cron_t openshift_var_lib_t:lnk_file { read getattr }; allow openshift_cron_t self:capability dac_override I'm setting this to NEEDINFO to get Miroslav's advice.
allow openshift_cron_t self:capability dac_override should be fixed. allow openshift_cron_t openshift_var_lib_t:lnk_file { read getattr }; needs to be fixed. Please clone this bug for selinux-policy.
From /etc/openshift/node.conf, I set CREATE_APP_SYMLINKS=1 (the default is 0). CREATE_APP_SYMLINKS=1 # If set to 1, creates gear-name symlinks to the UUID directories (debugging only) Then I create some new apps and get stuff like this: root@node: ~ # tail -1 /var/log/messages May 16 11:36:54 node setroubleshoot: SELinux is preventing /opt/rh/ruby193/root/usr/bin/ruby from read access on the lnk_file /var/lib/openshift/prf1832-ns. For complete SELinux messages. run sealert -l a26747d5-082c-442e-9446-718453a33222 root@node: ~ # sealert -l a26747d5-082c-442e-9446-718453a33222 SELinux is preventing /opt/rh/ruby193/root/usr/bin/ruby from read access on the lnk_file /var/lib/openshift/prf0210-ns. ***** Plugin catchall (100. confidence) suggests *************************** If you believe that ruby should be allowed read access on the prf0210-ns lnk_file by default. Then you should report this as a bug. You can generate a local policy module to allow this access. Do allow this access for now by executing: # grep ruby /var/log/audit/audit.log | audit2allow -M mypol # semodule -i mypol.pp root@node: ~ # grep prf1832-ns /var/log/audit/audit.log type=AVC msg=audit(1368718567.116:68801): avc: denied { read } for pid=13566 comm="ruby" name="prf1832-ns" dev=dm-0 ino=15736359 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:openshift_var_lib_t:s0 tclass=lnk_file type=AVC msg=audit(1368718629.252:71717): avc: denied { read } for pid=16295 comm="ruby" name="prf1832-ns" dev=dm-0 ino=15736359 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:openshift_var_lib_t:s0 tclass=lnk_file root@node: /var/lib/openshift/prf1832-ns # ls -alZ drwxr-x---. root 5194701e6892dff10c003f5d system_u:object_r:openshift_var_lib_t:s0 . drwxr-xr-x. root root system_u:object_r:openshift_var_lib_t:s0 .. drwxr-xr-x. root 5194701e6892dff10c003f5d system_u:object_r:openshift_var_lib_t:s0:c5,c103 app-root drwxr-x---. root 5194701e6892dff10c003f5d system_u:object_r:openshift_var_lib_t:s0 .env drwxr-xr-x. root root unconfined_u:object_r:openshift_var_lib_t:s0 git -rw-r--r--. root root unconfined_u:object_r:openshift_var_lib_t:s0 .gitconfig -rw-r--r--. root 5194701e6892dff10c003f5d system_u:object_r:openshift_var_lib_t:s0:c5,c103 .pearrc drwxr-xr-x. 5194701e6892dff10c003f5d 5194701e6892dff10c003f5d system_u:object_r:openshift_var_lib_t:s0:c5,c103 php d---------. root root system_u:object_r:openshift_tmp_t:s0 .sandbox drwxr-x---. root 5194701e6892dff10c003f5d system_u:object_r:ssh_home_t:s0 .ssh d---------. root root system_u:object_r:openshift_tmp_t:s0 .tmp root@node: /var/lib/openshift/prf1832-ns # cd ../ root@node: /var/lib/openshift # ls -ald /var/lib/openshift/prf1832-ns lrwxrwxrwx. 1 root root 24 May 16 01:35 /var/lib/openshift/prf1832-ns -> 5194701e6892dff10c003f5d root@node: /var/lib/openshift # ls -aldZ /var/lib/openshift/prf1832-ns lrwxrwxrwx. root root unconfined_u:object_r:openshift_var_lib_t:s0 /var/lib/openshift/prf1832-ns -> 5194701e6892dff10c003f5d root@node: /var/lib/openshift # ls -aldZ /var/lib/openshift/5194701e6892dff10c003f5d drwxr-x---. root 5194701e6892dff10c003f5d system_u:object_r:openshift_var_lib_t:s0 /var/lib/openshift/5194701e6892dff10c003f5d root@node: ~/selinux # time grep ruby /var/log/audit/audit.log | audit2allow -R -M lnk_file root@node: ~/selinux # cat lnk_file.te policy_module(lnk_file, 1.0) require { type openshift_cron_t; type openshift_var_lib_t; class lnk_file read; } #============= openshift_cron_t ============== allow openshift_cron_t openshift_var_lib_t:lnk_file read; root@node: ~/selinux # cat lnk_file.te policy_module(lnk_file, 1.0) require { type openshift_cron_t; type openshift_var_lib_t; class lnk_file read; } #============= openshift_cron_t ============== allow openshift_cron_t openshift_var_lib_t:lnk_file read; Later I found some more selinux errors, and then generated this policy: root@node: ~/selinux # cat lnk_file2.te policy_module(lnk_file2, 1.0) require { type openshift_cron_t; type openshift_var_lib_t; class lnk_file getattr; } #============= openshift_cron_t ============== allow openshift_cron_t openshift_var_lib_t:lnk_file getattr; Loading both of those seems to have done the trick, no more denials so far.
*** Bug 963830 has been marked as a duplicate of this bug. ***
Yes, we added fixes to Fedora and will have a fix in RHEL6 ASAP.
Retest this bug with selinux-policy-3.7.19-199.el6, but failed. [root@node1 ~]# rpm -q selinux-policy selinux-policy-3.7.19-199.el6.noarch [root@node1 ~]# tailf /var/log/audit/audit.log |grep avc <--snip--> type=AVC msg=audit(1370591401.820:115270): avc: denied { read } for pid=19430 comm="ruby" name="jbossews2i25m9pdq-ece03ko19z" dev=dm-0 ino=24531 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:openshift_var_lib_t:s0 tclass=lnk_file type=AVC msg=audit(1370591401.821:115271): avc: denied { read } for pid=19430 comm="ruby" name="51b069ca0f5b152695000010-jzu" dev=dm-0 ino=26232 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:openshift_var_lib_t:s0 tclass=lnk_file type=AVC msg=audit(1370591401.821:115272): avc: denied { read } for pid=19430 comm="ruby" name="h4qbzitnbg-sun000" dev=dm-0 ino=23985 scontext=system_u:system_r:openshift_cron_t:s0-s0:c0.c1023 tcontext=system_u:object_r:openshift_var_lib_t:s0 tclass=lnk_file <--snip--> Actually this bug is fixed in selinux-policy-3.7.19-195.el6_4.6, after I downgrade packages to selinux-policy-3.7.19-195.el6_4.6, this issue disappeared. BTW, selinux-policy-3.7.19-195.el6_4.6 is already available in http://cdn.rcm-qa.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os, once selinux-policy-3.7.19-199.el6 is pulled into ose puddle repo, then selinux-policy-3.7.19-199.el6 will be installed, this issue will reproduce.
It should be in this release. What does # rpm -qa |grep selinux Could you try to re-test it with selinux-policy-targeted-3.7.19-200.el6
Miroslav, Did I tag in the correct build? The RHEL 6.4 bug says it was fixed in selinux-policy-3.7.19-199.el6.
Nevermind, please ignore the NEEDINFO. I see now you already chimed in. :) Jianlin you cate get the packages Miroslav mentioned from here: http://download.devel.redhat.com/brewroot/packages/selinux-policy/3.7.19/200.el6/noarch/
Tagged selinux-policy-3.7.19-195.el6_4.10 into the latest puddle, please verify with this version of selinux-policy
Verify this bug on puddle: http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterprise/1.2/2013-06-07.2/ [root@node2 yum.repos.d]# rpm -qa|grep selinux-policy selinux-policy-3.7.19-195.el6_4.10.noarch selinux-policy-targeted-3.7.19-195.el6_4.10.noarch Modify /etc/openshift/node.conf, set CREATE_APP_SYMLINKS=1 Created a jbosseap-6.0 app, control this app like restart or ssh into it. No avc denials in audit.log
Closing all bugs introduced, fixed, and verified during 1.2 release work (thus never shipped).