nova has several daemons which run as the nova user - the api, objectstore, compute, network, volume and scheduler services We need SELinux policy to constrain these two daemons Basic instructions on getting set up are here: https://fedoraproject.org/wiki/Getting_started_with_OpenStack_Nova Note also the sudo and polkit configuration - the daemons run a whole bunch of privileged commands See also bug #732699 for glance
Note, as of openstack-nova-2011.3-0.5.d4.fc16 we have switched to systemd
Okay, even when running in the initrc_t domain, I'm seeing these when doing euca-add-keypair: avc: denied { write } for pid=25605 comm="ssh-keygen" name="tmpuao9tX" dev=dm-1 ino=2627585 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=dir avc: denied { add_name } for pid=25605 comm="ssh-keygen" name="temp" scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=dir avc: denied { create } for pid=25605 comm="ssh-keygen" name="temp" scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=file avc: denied { write open } for pid=25605 comm="ssh-keygen" name="temp" dev=dm-1 ino=2627586 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=file avc: denied { getattr } for pid=25605 comm="ssh-keygen" path="/tmp/tmpuao9tX/temp.pub" dev=dm-1 ino=2627587 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=file avc: denied { read } for pid=25606 comm="ssh-keygen" name="temp.pub" dev=dm-1 ino=2627587 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=file From the log: 2011-09-06 13:19:33,511 ERROR nova.api [2a149440-f70b-42a5-8a17-01a0c0e19f6d markmc markmc] Unexpected error raised: Unexpected error while running command. Command: ssh-keygen -q -b 1024 -N -f /tmp/tmpJjIaQz/temp Exit code: 1 Stdout: 'Saving the key failed: /tmp/tmpJjIaQz/temp.\n' Stderr: 'open /tmp/tmpJjIaQz/temp failed: Permission denied.\r\n' And this when starting an instance (which causes dnsmasq to be run): Sep 6 13:23:49 zig kernel: [75573.729918] type=1400 audit(1315311829.674:734): avc: denied { read } for pid=25669 comm="dnsmasq" name="sh" dev=dm-1 ino=131075 scontext=system_u:system_r:dnsmasq_t:s0 tcontext=system_u:object_r:bin_t:s0 tclass=lnk_file Sep 6 13:23:49 zig dnsmasq[25668]: cannot run lease-init script /usr/bin/nova-dhcpbridge: No such file or directory Sep 6 13:23:49 zig dnsmasq[25668]: FAILED to start up log for this one: (nova): TRACE: Command: sudo FLAGFILE=/etc/nova/nova.conf NETWORK_ID=1 dnsmasq --strict-order --bind-interfaces --interface=br0 --conf-file= --domain=novalocal --pid-file=/var/lib/nova/networks/nova-br0.pid --listen-address=10.0.0.1 --except-interface=lo --dhcp-range=10.0.0.2,static,120s --dhcp-lease-max=256 --dhcp-hostsfile=/var/lib/nova/networks/nova-br0.conf --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro (nova): TRACE: Exit code: 3 (nova): TRACE: Stdout: '' (nova): TRACE: Stderr: '\ndnsmasq: cannot run lease-init script /usr/bin/nova-dhcpbridge: No such file or directory\n'
Shouldn't this be filed against selinux-policy component?
Fair point, thanks Alan
I am looking on it. What is your output of # ps -eZ |grep initrc
I am getting # ps -eZ |grep initrc staff_u:system_r:initrc_t:s0 10220 ? 00:00:00 epmd staff_u:system_r:initrc_t:s0 10234 ? 00:00:00 beam.smp staff_u:system_r:initrc_t:s0 10304 ? 00:00:00 cpu_sup staff_u:system_r:initrc_t:s0 10308 ? 00:00:00 inet_gethost staff_u:system_r:initrc_t:s0 10309 ? 00:00:00 inet_gethost
I am creating policy for these daemons and also for all openstack-nova-* daemons. I will send it to rkukura for testing since it is quite a lot of work.
(In reply to comment #7) > I am creating policy for these daemons and also for all openstack-nova-* > daemons. I will send it to rkukura for testing since it is quite a lot of work. Awesome, thanks Miroslav
I created initial policies, but I overlooked part of your comment > Note also the sudo and polkit configuration - the daemons run a whole bunch of > privileged commands which mean you run iptables, lvm, ifconfing and so on using sudo and i see AVC like type=AVC msg=audit(1318426149.510:2939): avc: denied { execute } for pid=7317 comm="sudo" name="xtables-multi" dev=dm-2 ino=2883601 scontext=system_u:system_r:nova_api_t:s0 tcontext=system_u:object_r:iptables_exec_t:s0 tclass=file # grep sudo /var/log/audit/audit.log | wc -l 245 which means you do almost "everything" from SELinux point of view. So you can not expect SElinux could lock it down.
nova_api_t, nova_network_t, nova_volume_t run sudo. Well, we could make this domain as unconfined but still you run "sudo" directly in the code.
The use of sudo is recognized by upstream as "a really bad idea" and there are efforts to remedy the problem in the forthcoming Essex development.
Great.
Created attachment 528051 [details] Policy for nova daemons
Created attachment 528054 [details] Policy for rabbitmq
How to test: Just unpack archives and run # sh nova.sh # sh rabbitmq.sh # echo "-w /etc/shadow -p wa" >> /etc/audit/audit.rules # service auditd restart and restart nova and rabbitmq daemons and test it. Then I would like to get /var/log/audit/audit.log.
I'll give this a try this afternoon.
Created attachment 528426 [details] commands and audit.log output I've added an attachment containing commands starting rabbitmq, glance, and nova, and running through an openstack test scenario after having applied the policies. Output from audit.log is interspersed.
I am adding fixes but I am seeing allow nova_compute_t svirt_image_t:file { read write getattr open setattr }; could you explain me how virt machine are started?
Fixed in selinux-policy-3.10.0-43.fc16
selinux-policy-3.10.0-43.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/selinux-policy-3.10.0-43.fc16
Package selinux-policy-3.10.0-43.fc16: * should fix your issue, * was pushed to the Fedora 16 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing selinux-policy-3.10.0-43.fc16' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2011-14618 then log in and leave karma (feedback).
Package selinux-policy-3.10.0-45.1.fc16: * should fix your issue, * was pushed to the Fedora 16 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing selinux-policy-3.10.0-45.1.fc16' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2011-14618 then log in and leave karma (feedback).
selinux-policy-3.10.0-45.1.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report.