Bug 1138424 - Need selinux policy for OpenStack Keystone running in Apache with mod_wsgi
Summary: Need selinux policy for OpenStack Keystone running in Apache with mod_wsgi
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: selinux-policy
Version: 7.1
Hardware: All
OS: Linux
Target Milestone: rc
: ---
Assignee: Lukas Vrabec
QA Contact: Martin Žember
: 1157658 (view as bug list)
Depends On: 1111274 1122764 1122767
Blocks: 1123117 1126594 1154615 1170218 1170223 1170224 1170225 1170370 1170372
TreeView+ depends on / blocked
Reported: 2014-09-04 18:35 UTC by Rich Megginson
Modified: 2016-04-26 14:59 UTC (History)
14 users (show)

Fixed In Version: selinux-policy-3.13.1-16.el7
Doc Type: Bug Fix
Doc Text:
Clone Of: 1122764
: 1154615 1170218 1180230 (view as bug list)
Last Closed: 2015-03-05 10:44:53 UTC
Target Upstream Version:

Attachments (Terms of Use)
proposed policy and scripts to compile and test (1.08 KB, application/x-tgz)
2014-09-05 23:01 UTC, Rich Megginson
no flags Details

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:0458 normal SHIPPED_LIVE selinux-policy bug fix and enhancement update 2015-03-05 15:17:00 UTC

Description Rich Megginson 2014-09-04 18:35:33 UTC
+++ This bug was initially created as a clone of Bug #1122764 +++

Keystone's preferred deployment has changed to running within Apache httpd/mod_wsgi upstream.  This offers better performance, stronger authentication mechanisms, and federation capabilities over using eventlet (keystone-all).

We should deploy Keystone in httpd/mod_wsgi for RHEL OSP 6.0 via all supported installation methods.

This bug will serve as a tracker for the various sub-tasks that are needed to complete this work across components.

--- Additional comment from Nathan Kinder on 2014-08-05 16:11:09 EDT ---

I was able to get this working with RHEL7 by doing the following:

semanage port -m -t http_port_t -p tcp 5000

and using the following policy:

module keystonewsgi 1.0;

require {
	type httpd_t;
	type keystone_log_t;
	type keystone_var_lib_t;
	class dir { search getattr };
	class file open;

#============= httpd_t ==============

allow httpd_t keystone_log_t:dir search;
allow httpd_t keystone_log_t:file open;
allow httpd_t keystone_var_lib_t:dir getattr;
allow httpd_t keystone_var_lib_t:dir search;

apache mod_wsgi runs each keystone daemon (the main and admin daemons) as separate processes with uid keystone.  Instead of relabeling these resources to allow httpd_t access, I think it would be better to transition from httpd_t to keystone_t when apache mod_wsgi starts the keystone daemons.

Comment 1 Rich Megginson 2014-09-04 19:35:47 UTC
What's needed is a transition rule.  When apache (httpd_t) executes /var/www/cgi-bin/keystone/*, transition the new process to keystone_t.

Comment 2 Miroslav Grepl 2014-09-05 12:29:13 UTC
I believe we want to have keystone_cgi_script_t for it.

Comment 3 Rich Megginson 2014-09-05 23:01:30 UTC
Created attachment 934922 [details]
proposed policy and scripts to compile and test

Comment 4 Nathan Fritze 2014-10-03 19:04:09 UTC
Some more information here  (I've been working on this issue with Nathan Kinder as well) - mod_wsgi does not exec the keystone scripts - instead it loads them as data and runs them in python interpreters inside the httpd subprocess.  As a result, a domain autotrans on the exec of the scripts will not trigger. I've written some test code to allow mod_wsgi to dynamically change context after the fork, just as it changes the user, process group, et al, but this brings up other questionable choices.

In particular, SELinux is not in favor of switching within a process.  In this case, we're doing it directly after a fork, but we're still doing it within the process and not on an exec boundary.

A better choice could be to actually cause keystone to do the exec and switch that way, but then it's no longer in the apache process, which may cause issues for things such as the SAML and federation support as of Juno.  This is also quite a bit more work.

Short question - I can make this work this way (dynamic transition on mod_wsgi fork), but is doing that appropriate?

Comment 5 Lukas Vrabec 2014-10-06 14:14:37 UTC
commit 141011af28212092f96b0719c754b91c5077c670
Author: Lukas Vrabec <lvrabec@redhat.com>
Date:   Mon Oct 6 16:13:54 2014 +0200

    Make keystone_cgi_script_t domain. BZ (#1138424)

Comment 9 Nathan Kinder 2014-10-17 00:52:01 UTC
I still see this AVC when running keystone in httpd with selinux-policy-3.13.1-6.el7.noarch:

type=AVC msg=audit(1413506204.808:77): avc:  denied  { open } for  pid=5219 comm="httpd" path="/var/log/keystone/keystone.log" dev="vda1" ino=18431067 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:keystone_log_t:s0 tclass=file

The following rule is recommended by audit2allow:

  allow httpd_t keystone_log_t:file open;

I suspect that this is due to httpd opening keystone.log prior to the transition.

Comment 10 Nathan Kinder 2014-10-17 01:46:44 UTC
If I disable dontaudit rules, I also see this AVC in permissive mode:

type=AVC msg=audit(1413510114.291:718): avc:  denied  { name_bind } for  pid=3929 comm="httpd" src=5000 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:commplex_main_port_t:s0 tclass=tcp_socket

Unfortunately, port 5000 is a standard keystone port, but it's already labeled as commplex_main_port_t in the base policy.  Is it possible to define an alias to allow this port to be used for either purpose?

Comment 11 Nathan Kinder 2014-10-17 02:05:24 UTC
I am able to run keystone in httpd using selinux-policy-3.13.1-6.el7.noarch along with this custom policy module:


require {
    type httpd_t;
    type keystone_log_t;
    type commplex_main_port_t;

typealias commplex_main_port_t alias keystone_main_port_t;

allow httpd_t keystone_log_t:file open;
allow httpd_t keystone_main_port_t:tcp_socket name_bind;

As you can see, I chose to alias commplex_main_port_t.  This required a new type of keystone_main_port_t since I couldn't redefine keystone_port_t as an alias.  This new alias name is appropriate, as port 5000 is referred to as the keystone "main" port, while 35357 is the keystone "admin" port.  If there is a better way to handle this, feel free to change it.  We just need to ensure that httpd_t can name_bind to ports 5000 and 35357.

Comment 12 Nathan Kinder 2014-10-17 17:11:00 UTC
One additional thing to point out is that I think the current rules for port 35357 are incorrect, but they happen to be working by chance.

When running keystone in httpd, we need httpd_t to be able to 'name_bind' to the ports.  The current policy around port 35357 (labeled keystone_port_t) allows keystone_t to name_bind.  This is needed when running keystone_all, but it doesn't work for httpd.

Since Keystone's admin port (35357) is in the ephemeral port range, httpd_t is allowed to bind to it.  I tested this my modifyin gthe httpd config to use port 35358, and it binds to the port just fine in enforcing mode.

We need to allow http_t to name_bind to keystone_port_t and keystone_main_port_t (or whatever alias/label you choose to use for port 5000).

Comment 13 Lukas Vrabec 2014-10-27 11:04:04 UTC
Sent scratch builds to Nathan, waiting for reply to push to selinux-policy build.

Comment 14 Nathan Kinder 2014-10-27 14:15:11 UTC
*** Bug 1157658 has been marked as a duplicate of this bug. ***

Comment 15 Nathan Kinder 2014-10-27 21:01:21 UTC
(In reply to Lukas Vrabec from comment #13)
> Sent scratch builds to Nathan, waiting for reply to push to selinux-policy
> build.

It still fails in enforcing and audit2allow shows the following with
dontaudit rules turned off:

  allow httpd_t commplex_main_port_t:tcp_socket name_bind;
  allow httpd_t keystone_log_t:file open;

These would be solved with the changes I described in comment#11.

Comment 16 Lukas Vrabec 2014-11-10 14:36:44 UTC
commit 21f2ce906ab407ae02bf4ce20c5d0315a364e86f
Author: Lukas Vrabec <lvrabec@redhat.com>
Date:   Mon Nov 10 15:34:53 2014 +0100

    Label keystone cgi files as keystone_cgi_script_exec_t. BZ(1138424)

Comment 18 Nathan Kinder 2014-12-17 17:01:33 UTC
There are some issues being seen with the new policy as mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1174795#c11:

type=AVC msg=audit(1418833963.580:1474): avc:  denied  { getattr } for  pid=15900 comm="keystone-all" path="/usr/share/keystone/keystone-dist.conf" dev="sda1" ino=925373 scontext=system_u:system_r:keystone_t:s0 tcontext=system_u:object_r:keystone_cgi_script_exec_t:s0 tclass=file permissive=1
type=AVC msg=audit(1418833963.580:1475): avc:  denied  { read } for  pid=15900 comm="keystone-all" name="keystone-dist.conf" dev="sda1" ino=925373 scontext=system_u:system_r:keystone_t:s0 tcontext=system_u:object_r:keystone_cgi_script_exec_t:s0 tclass=file permissive=1
type=AVC msg=audit(1418833963.580:1476): avc:  denied  { open } for  pid=15900 comm="keystone-all" path="/usr/share/keystone/keystone-dist.conf" dev="sda1" ino=925373 scontext=system_u:system_r:keystone_t:s0 tcontext=system_u:object_r:keystone_cgi_script_exec_t:s0 tclass=file permissive=1

These AVCs were reported on Fedora (rawhide), but I believe that they apply for RHEL 7.1 as well.

Comment 19 Milos Malik 2014-12-18 10:06:12 UTC
The /usr/share/keystone/keystone-dist.conf file looks like a configuration file, so it should be labeled either etc_t or usr_t. Does it work after execution of following command?

# chcon -t usr_t /usr/share/keystone/keystone-dist.conf

Comment 20 Nathan Kinder 2015-01-08 06:27:03 UTC
(In reply to Milos Malik from comment #19)
> The /usr/share/keystone/keystone-dist.conf file looks like a configuration
> file, so it should be labeled either etc_t or usr_t. Does it work after
> execution of following command?
> # chcon -t usr_t /usr/share/keystone/keystone-dist.conf

No, this wasn't enough to make it work.  I had to recursively label the /usr/share/keystone directory as usr_t to make it work.  Here is the labels prior to using chcon:

[rhosuser@rhos ~]$ ls -laZ /usr/share/keystone
drwxr-xr-x. root root     system_u:object_r:keystone_cgi_script_exec_t:s0 .
drwxr-xr-x. root root     system_u:object_r:usr_t:s0       ..
-rw-r--r--. root keystone system_u:object_r:keystone_cgi_script_exec_t:s0 keystone-dist.conf
-rw-r--r--. root keystone system_u:object_r:keystone_cgi_script_exec_t:s0 keystone-dist-paste.ini
-rw-r--r--. root keystone system_u:object_r:keystone_cgi_script_exec_t:s0 keystone.wsgi
-rw-r--r--. root keystone system_u:object_r:keystone_cgi_script_exec_t:s0 policy.v3cloudsample.json
-rwxr-xr-x. root root     system_u:object_r:keystone_cgi_script_exec_t:s0 sample_data.sh
-rw-r--r--. root keystone system_u:object_r:keystone_cgi_script_exec_t:s0 wsgi-keystone.conf

The keystone_cgi_script_exec_t label for /usr/share/keystone and it's contents works fine when running keystone in httpd, but not when it uses eventlet (keystone-all).  Both httpd and eventlet deployment methods work with a label of usr_t for /usr/share/keystone and it's contents, so I think that is the correct solution.

I do however still encounter failures starting keystone in httpd in enforcing mode.  The start failures are due to the following AVC messages as seen when starting keystone in httpd in permissive mode (with dontaudit rules disabled):

type=AVC msg=audit(1420693901.582:15510): avc:  denied  { name_bind } for  pid=1147 comm="httpd" src=5000 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:commplex_main_port_t:s0 tclass=tcp_socket
type=AVC msg=audit(1420694028.935:15621): avc:  denied  { open } for  pid=1363 comm="httpd" path="/var/log/keystone/keystone.log" dev="vda1" ino=25700381 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:keystone_log_t:s0 tclass=file

We still need the changes I recommended in comment#11 to allow keystone to work in httpd, plus we need to correct the label for /usr/share/keystone to allow keystone to run in eventlet.  Basically, both supported deployment methods for keystone are broken in enforcing mode, so this is important to get addressed for RHEL 7.1.

Comment 21 Nathan Kinder 2015-01-08 06:28:28 UTC
I forgot to mention that the testing in comment#20 was using the selinux-policy-3.13.1-14.el7.noarch package.

Comment 23 Ryan Hallisey 2015-01-08 14:56:11 UTC
Adding the below rule to fix httpd_t.

allow httpd_t keystone_log_t:file open;

Comment 24 Ryan Hallisey 2015-01-08 16:23:39 UTC
^ to openstack-selinux

Comment 25 Nathan Kinder 2015-01-09 02:51:41 UTC
After using Ryan's latest openstack-selinux package, there is still the problem around the keystone_cgi_script_exec_t on /usr/share/keystone when deploying in eventlet.  To work around that, I'm defining the following fcontext before running packstack:

  semanage fcontext -a -t usr_t /usr/share/keystone\(/.\*\)\?

There are still some other unrelated AVCs that prevent packstack from completing in enforcing mode, but I'll file a separate bug for those.

All that remains here is to correct the label used for /usr/share/keystone(/.*)? to usr_t in selinux-policy.

Comment 26 Miroslav Grepl 2015-01-09 14:15:26 UTC
If we change it to usr_t we don't get a transition. What is a path to CGI scripts?

Comment 27 Nathan Kinder 2015-01-09 18:38:23 UTC
The Keystone CGI scripts are placed in /var/www/cgi-bin/keystone:

[rhosuser@rhos ~]$ sudo ls -laZ /var/www/cgi-bin/keystone
drwxr-xr-x. keystone keystone system_u:object_r:httpd_sys_script_exec_t:s0 .
drwxr-xr-x. root     root     system_u:object_r:httpd_sys_script_exec_t:s0 ..
-rw-r--r--. keystone keystone system_u:object_r:httpd_sys_script_exec_t:s0 admin
-rw-r--r--. keystone keystone system_u:object_r:httpd_sys_script_exec_t:s0 main

Comment 28 Nathan Kinder 2015-01-09 20:48:52 UTC
I don't see the transition occurring, regardless of the label that is used on the CGI scripts.  As a test, I set the following fcontext rules before installing keystone:

  semanage fcontext -a -t usr_t /usr/share/keystone\(/.\*\)\?
  semanage fcontext -a -t keystone_cgi_script_exec_t /var/www/cgi-bin/keystone\(/.\*\)\?

This results in the keystone (mod_wsgi) processes running as httpd_t:

[rhosuser@rhos ~]$ ps -efZ | grep keystone
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 rhosuser 990 7879  0 15:45 pts/0 00:00:00 grep --color=auto keystone
system_u:system_r:httpd_t:s0    keystone 28858 28857  0 14:59 ?        00:00:07 keystone-admin  -DFOREGROUND
system_u:system_r:httpd_t:s0    keystone 28859 28857  0 14:59 ?        00:00:00 keystone-main   -DFOREGROUND

I will note that keystone seems to be working fine in enforcing mode as httpd_t though after setting the above fcontext rules using these packages:


Comment 29 Miroslav Grepl 2015-01-12 11:11:01 UTC
commit 420fe90411dd264136d08f084273e019a35ff03e
Author: Miroslav Grepl <mgrepl@redhat.com>
Date:   Mon Jan 12 12:10:08 2015 +0100

    Fix labeling for keystone CGI scripts.

Comment 30 Nathan Kinder 2015-01-14 15:13:49 UTC
I no longer see the AVCs described in this bug when deploying keystone in httpd with selinux-policy-3.13.1-16.el7, so the fixes appear to be working.

I do see an unrelated glance AVC during installation of RHEL OSP that seems to be newly introduced as of selinux-policy-3.13.1-15.el7.  I've opened bug 1181818 to track that issue.

Comment 33 errata-xmlrpc 2015-03-05 10:44:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.