OpenStack's L2 Network Service (Quantum) doesn't currently have any SELinux policy defined for it so it runs unconfined in the initrc_t domain
I changed the version to Fedora 17 and the component to selinux-policy-targeted. There is no need to address the Fedora 16 version of quantum, as that was basically a preview. In Fedora 17, the quantum-server and quantum-*-agent daemon processes currently run unconfined, and together perform network configuration for cloud tenants. They need to run confined with appropriate SELinux policies. The following F17 packages are involved: openstack-quantum python-quantum openstack-quantum-cisco openstack-quantum-linuxbridge openstack-quantum-openvswitch openstack-quantum-nicira openstack-quantum-ryu The quantum-server daemon runs on a controller node and is configured to use a plugin from one of the openstack-quantum-* packages. These plugins typically access a DB and/or use sockets (http) to talk to an external network controller. The linuxbridge, openvswitch, and ryu plugins require that a respective quantum-linuxbridge-agent, quantum-openvswitch-agent, or quantum-ryu-agent daemon process run on all nova-compute nodes. These agent daemons read from a DB and issue network configuration commands such as "sudo quantum-rootwrap ip ..." and "sudo quantum-rootwrap ovs-vsctl ...".
I just added an initial quantum policy for F17, which should get us going on generating a more formal policy for it. Currently I combined all of the quauntum services with the same type. I would prefer that quantum not use /tmp but should move to /run/quantum directory for temporary content.
Should show up in selinux-policy-3.10.0-106.fc17
Dan, Thanks! We'll add a comment here when we've tested with selinux-policy-3.10.0-106.fc17. I'm not aware of any quantum services using /tmp (directly). Have you seen evidence of this? -Bob
type=PATH msg=audit(03/21/2012 13:18:56.756:3143) : item=1 name=/tmp/ffiJsbyQ2 inode=237886 dev=00:20 mode=file,600 ouid=quantum ogid=quantum rdev=00:00 obj=system_u:object_r:tmp_t:s0 type=PATH msg=audit(03/21/2012 13:18:56.756:3143) : item=0 name=/tmp/ inode=11288 dev=00:20 mode=dir,sticky,777 ouid=root ogid=root rdev=00:00 obj=system_u:object_r:tmp_t:s0 type=CWD msg=audit(03/21/2012 13:18:56.756:3143) : cwd=/ type=SYSCALL msg=audit(03/21/2012 13:18:56.756:3143) : arch=x86_64 syscall=unlink success=yes exit=0 a0=7fff0016e8f0 a1=c2 a2=2 a3=b8e8c2b480 items=2 ppid=1 pid=31583 auid=unset uid=quantum gid=quantum euid=quantum suid=quantum fsuid=quantum egid=quantum sgid=quantum fsgid=quantum tty=(none) ses=unset comm=python exe=/usr/bin/python subj=system_u:system_r:quantum_t:s0 key=(null) type=AVC msg=audit(03/21/2012 13:18:56.756:3143) : avc: denied { unlink } for pid=31583 comm=python name=ffiJsbyQ2 dev="tmpfs" ino=237886 scontext=system_u:system_r:quantum_t:s0 tcontext=system_u:object_r:tmp_t:s0 tclass=file type=AVC msg=audit(03/21/2012 13:18:56.756:3143) : avc: denied { remove_name } for pid=31583 comm=python name=ffiJsbyQ2 dev="tmpfs" ino=237886 scontext=system_u:system_r:quantum_t:s0 tcontext=system_u:object_r:tmp_t:s0 tclass=dir
type=MMAP msg=audit(03/21/2012 13:18:56.756:3144) : fd=9 flags=follow type=SYSCALL msg=audit(03/21/2012 13:18:56.756:3144) : arch=x86_64 syscall=mmap success=yes exit=-197877760(Unknown error 197877760) a0=0 a1=1000 a2=5 a3=1 items=0 ppid=1 pid=31583 auid=unset uid=quantum gid=quantum euid=quantum suid=quantum fsuid=quantum egid=quantum sgid=quantum fsgid=quantum tty=(none) ses=unset comm=python exe=/usr/bin/python subj=system_u:system_r:quantum_t:s0 key=(null) type=AVC msg=audit(03/21/2012 13:18:56.756:3144) : avc: denied { execute } for pid=31583 comm=python path=/tmp/ffiJsbyQ2 (deleted) dev="tmpfs" ino=237886 scontext=system_u:system_r:quantum_t:s0 tcontext=system_u:object_r:tmp_t:s0 tclass=file
Thanks. This must be happening inside some python library that quantum is using. Any suggestions on how to track it down? Is this something that can be dealt with in the systemd unit? I'm checking whether other OpenStack components have had similar issues.
I think it is doing some kind of JIT stuff. But anyways we can confine this with SELinux, as long as it was not intentional, we might want to run these apps with PrivateTmp in systemd to make sure they don't do dumb things. But they will not be able to share content in /tmp if they do this.
Dan, would this be the python runtime doing JIT? That would seem to be a known situation with known solutions. The quantum server and agents should not be doing anything at all unusual here. Should these services use PrivateTmp? Is that the norm for python daemons?
I would like to make it the norm for all root daemons, especially if they do not use /tmp for communication. http://danwalsh.livejournal.com/51459.html
selinux-policy-3.10.0-106.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/selinux-policy-3.10.0-106.fc17
Package selinux-policy-3.10.0-106.fc17: * should fix your issue, * was pushed to the Fedora 17 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing selinux-policy-3.10.0-106.fc17' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2012-4694/selinux-policy-3.10.0-106.fc17 then log in and leave karma (feedback).
Created attachment 576580 [details] AVC denials using linuxbridge plugin.
I've updated the quantum server and agent systemd units to enable PrivateTmp and done a bit of testing with selinux-policy-3.10.0-110.fc17.noarch and openstack-quantum[-linuxbridge]-2012.1-1.fc17.noarch. I am seeing a number of AVC denials, and have attached an excerpt from audit.log above. Note that this does not exhaustively test the commands executed by the linuxbridge agent, and doesn't cover the openvswitch agent at all. Commands executed by the quantum-linuxbridge-agent are listed in /usr/lib/python2.7/site-packages/quantum/rootwrap/linuxbridge-agent.py: filterlist = [ # quantum/plugins/linuxbridge/agent/linuxbridge_quantum_agent.py: # 'brctl', 'addbr', bridge_name # 'brctl', 'addif', bridge_name, interface # 'brctl', 'addif', bridge_name, tap_device_name # 'brctl', 'delbr', bridge_name # 'brctl', 'delif', bridge_name, interface_name # 'brctl', 'delif', current_bridge_name, ... # 'brctl', 'setfd', bridge_name, ... # 'brctl', 'stp', bridge_name, 'off' filters.CommandFilter("/usr/sbin/brctl", "root"), filters.CommandFilter("/sbin/brctl", "root"), # quantum/plugins/linuxbridge/agent/linuxbridge_quantum_agent.py: # 'ip', 'link', 'add', 'link', ... # 'ip', 'link', 'delete', interface # 'ip', 'link', 'set', bridge_name, 'down' # 'ip', 'link', 'set', bridge_name, 'up' # 'ip', 'link', 'set', interface, 'down' # 'ip', 'link', 'set', interface, 'up' # 'ip', 'link', 'show', 'dev', device # 'ip', 'tuntap' # 'ip', 'tuntap' filters.CommandFilter("/usr/sbin/ip", "root"), filters.CommandFilter("/sbin/ip", "root"), ] Commands executed by the quantum-openvswitch-agent are listed in /usr/lib/python2.7/site-packages/quantum/rootwrap/openvswitch-agent.py: filterlist = [ # quantum/plugins/openvswitch/agent/ovs_quantum_agent.py: # "ovs-vsctl", "--timeout=2", ... filters.CommandFilter("/usr/bin/ovs-vsctl", "root"), filters.CommandFilter("/bin/ovs-vsctl", "root"), # quantum/plugins/openvswitch/agent/ovs_quantum_agent.py: # "ovs-ofctl", cmd, self.br_name, args filters.CommandFilter("/usr/bin/ovs-ofctl", "root"), filters.CommandFilter("/bin/ovs-ofctl", "root"), # quantum/plugins/openvswitch/agent/ovs_quantum_agent.py: # "xe", "vif-param-get", ... filters.CommandFilter("/usr/bin/xe", "root"), filters.CommandFilter("/usr/sbin/xe", "root"), ] Do you need actual AVC denials generated from all of these commands before you can update the policy to allow them? If so, I will do some more exhaustive testing with both plugins and attach the audit.log.
Added rules to allow quantum to transition to ifconfig_t domain when executing /usr/sbin/ip and brctl_t domain when executing brctl. We do not define policy for these so I am not sure what access these are going to require. # "ovs-vsctl", "--timeout=2", ... filters.CommandFilter("/usr/bin/ovs-vsctl", "root"), filters.CommandFilter("/bin/ovs-vsctl", "root"), # quantum/plugins/openvswitch/agent/ovs_quantum_agent.py: # "ovs-ofctl", cmd, self.br_name, args filters.CommandFilter("/usr/bin/ovs-ofctl", "root"), filters.CommandFilter("/bin/ovs-ofctl", "root"), # quantum/plugins/openvswitch/agent/ovs_quantum_agent.py: # "xe", "vif-param-get", ... filters.CommandFilter("/usr/bin/xe", "root"), filters.CommandFilter("/usr/sbin/xe", "root"),
Created attachment 576589 [details] Latest policy for quantum.pp # semodule -i quantum.pp To test the latest policy package out.
Created attachment 576817 [details] AVC denials using linuxbridge plugin. Here's the audit.log for the same test case after installing attachment 576589 [details]. Most denials were eliminated but a couple remain, especially regarding sudo. I will run additional test cases as soon as possible and attach logs of any additional denials I encounter.
selinux-policy-3.10.0-106.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
*** Bug 877378 has been marked as a duplicate of this bug. ***
In #877378 Matthias Runge writes: using openvswitch in connection with the quantum openvswitch plugin results in the following SELinux deny: Additional Information: Source Context system_u:system_r:quantum_t:s0 Target Context system_u:system_r:initrc_t:s0 Target Objects /run/openvswitch/db.sock [ unix_stream_socket ] Source ovs-ofctl Source Path /usr/bin/ovs-ofctl Port <Unknown> Host turing.berg.ol Source RPM Packages openvswitch-1.4.2-5.fc17.x86_64 Target RPM Packages Policy RPM selinux-policy-3.10.0-159.fc17.noarch Selinux Enabled True Policy Type targeted Enforcing Mode Permissive Host Name turing.berg.ol Platform Linux turing.berg.ol 3.6.6-1.fc17.x86_64 #1 SMP Mon Nov 5 21:59:35 UTC 2012 x86_64 x86_64 Alert Count 4 First Seen 2012-11-16 20:20:20 CET Last Seen 2012-11-16 20:23:34 CET Local ID 21262aa1-741c-422d-af24-2157c8bed7ea Raw Audit Messages type=AVC msg=audit(1353093814.733:1368): avc: denied { connectto } for pid=5967 comm="ovs-vsctl" path="/run/openvswitch/db.sock" scontext=system_u:system_r:quantum_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=unix_stream_socket type=SYSCALL msg=audit(1353093814.733:1368): arch=x86_64 syscall=connect success=yes exit=0 a0=3 a1=7fff1e51aad0 a2=1f a3=ffffffff items=0 ppid=5958 pid=5967 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=ovs-vsctl exe=/usr/bin/ovs-vsctl subj=system_u:system_r:quantum_t:s0 key=(null) Hash: ovs-ofctl,quantum_t,initrc_t,unix_stream_socket,connectto [mrunge@turing horizon (master)]$ rpm -q openvswitch selinux-policy-targeted openvswitch-1.4.2-5.fc17.x86_64 selinux-policy-targeted-3.10.0-159.fc17.noarch
ps -eZ | grep initrc_t Some procss that is listening on /run/openvswitch/db.sock is running with the wrong label.
(In reply to comment #21) > ps -eZ | grep initrc_t > > Some procss that is listening on /run/openvswitch/db.sock > > is running with the wrong label. [root@turing ~]# ps -eZ | grep initrc_t system_u:system_r:initrc_t:s0 730 ? 00:00:00 cinder-api system_u:system_r:initrc_t:s0 734 ? 00:00:00 swift-account-s system_u:system_r:initrc_t:s0 738 ? 00:00:00 swift-object-se system_u:system_r:initrc_t:s0 758 ? 00:00:00 swift-proxy-ser system_u:system_r:initrc_t:s0 762 ? 00:00:04 python system_u:system_r:initrc_t:s0 764 ? 00:00:51 python system_u:system_r:initrc_t:s0 765 ? 00:00:00 swift-container system_u:system_r:initrc_t:s0 767 ? 00:00:18 cinder-schedule system_u:system_r:initrc_t:s0 872 ? 00:00:04 ovsdb-server system_u:system_r:initrc_t:s0 873 ? 00:00:16 ovsdb-server system_u:system_r:initrc_t:s0 1123 ? 00:00:04 ovs-vswitchd system_u:system_r:initrc_t:s0 1124 ? 00:00:17 ovs-vswitchd system_u:system_r:initrc_t:s0 1312 ? 00:00:00 swift-object-se system_u:system_r:initrc_t:s0 1313 ? 00:00:00 swift-object-se system_u:system_r:initrc_t:s0 1314 ? 00:00:00 swift-object-se system_u:system_r:initrc_t:s0 1315 ? 00:00:00 swift-account-s system_u:system_r:initrc_t:s0 1316 ? 00:00:00 swift-account-s system_u:system_r:initrc_t:s0 1341 ? 00:00:00 swift-container system_u:system_r:initrc_t:s0 1342 ? 00:00:00 swift-container system_u:system_r:initrc_t:s0 1343 ? 00:00:00 swift-proxy-ser system_u:system_r:initrc_t:s0 1344 ? 00:00:00 swift-proxy-ser system_u:system_r:initrc_t:s0 1345 ? 00:00:00 swift-proxy-ser system_u:system_r:initrc_t:s0 1346 ? 00:00:00 swift-proxy-ser system_u:system_r:initrc_t:s0 1347 ? 00:00:00 swift-proxy-ser system_u:system_r:initrc_t:s0 1348 ? 00:00:00 swift-proxy-ser system_u:system_r:initrc_t:s0 1351 ? 00:00:00 swift-proxy-ser system_u:system_r:initrc_t:s0 1354 ? 00:00:00 swift-proxy-ser
We have policy for this in F18, I guess we need to back port to F17.
Yes, there is also another bug which requires to have backported policy.
Added. commit 79176052323c4f68168adcff72dbcbc10613832e Author: Miroslav Grepl <mgrepl> Date: Tue Dec 4 16:28:53 2012 +0100 Backport openvswitch policy from F18
*** Bug 872974 has been marked as a duplicate of this bug. ***
selinux-policy-3.10.0-165.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/selinux-policy-3.10.0-165.fc17
Package selinux-policy-3.10.0-165.fc17: * should fix your issue, * was pushed to the Fedora 17 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing selinux-policy-3.10.0-165.fc17' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2012-20544/selinux-policy-3.10.0-165.fc17 then log in and leave karma (feedback).
selinux-policy-3.10.0-166.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.