Bug 734346 - SELinux policy for nova
Summary: SELinux policy for nova
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: selinux-policy
Version: 16
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miroslav Grepl
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-08-30 07:05 UTC by Mark McLoughlin
Modified: 2011-10-25 03:34 UTC (History)
9 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2011-10-25 03:34:16 UTC


Attachments (Terms of Use)
Policy for nova daemons (20.00 KB, application/x-tar)
2011-10-13 15:12 UTC, Miroslav Grepl
no flags Details
Policy for rabbitmq (10.00 KB, application/x-tar)
2011-10-13 15:13 UTC, Miroslav Grepl
no flags Details
commands and audit.log output (225.85 KB, text/plain)
2011-10-17 03:30 UTC, Bob Kukura
no flags Details

Description Mark McLoughlin 2011-08-30 07:05:40 UTC
nova has several daemons which run as the nova user - the api, objectstore, compute, network, volume and scheduler services

We need SELinux policy to constrain these two daemons

Basic instructions on getting set up are here:

  https://fedoraproject.org/wiki/Getting_started_with_OpenStack_Nova

Note also the sudo and polkit configuration - the daemons run a whole bunch of privileged commands

See also bug #732699 for glance

Comment 1 Mark McLoughlin 2011-08-30 20:54:50 UTC
Note, as of openstack-nova-2011.3-0.5.d4.fc16 we have switched to systemd

Comment 2 Mark McLoughlin 2011-09-06 12:44:51 UTC
Okay, even when running in the initrc_t domain, I'm seeing these when doing euca-add-keypair:

avc:  denied  { write } for  pid=25605 comm="ssh-keygen" name="tmpuao9tX" dev=dm-1 ino=2627585 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=dir
avc:  denied  { add_name } for  pid=25605 comm="ssh-keygen" name="temp" scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=dir
avc:  denied  { create } for  pid=25605 comm="ssh-keygen" name="temp" scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=file
avc:  denied  { write open } for  pid=25605 comm="ssh-keygen" name="temp" dev=dm-1 ino=2627586 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=file
avc:  denied  { getattr } for  pid=25605 comm="ssh-keygen" path="/tmp/tmpuao9tX/temp.pub" dev=dm-1 ino=2627587 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=file
avc:  denied  { read } for  pid=25606 comm="ssh-keygen" name="temp.pub" dev=dm-1 ino=2627587 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:initrc_tmp_t:s0 tclass=file

From the log:

2011-09-06 13:19:33,511 ERROR nova.api [2a149440-f70b-42a5-8a17-01a0c0e19f6d markmc markmc] Unexpected error raised: Unexpected error while running command.
Command: ssh-keygen -q -b 1024 -N  -f /tmp/tmpJjIaQz/temp
Exit code: 1
Stdout: 'Saving the key failed: /tmp/tmpJjIaQz/temp.\n'
Stderr: 'open /tmp/tmpJjIaQz/temp failed: Permission denied.\r\n'




And this when starting an instance (which causes dnsmasq to be run):

Sep  6 13:23:49 zig kernel: [75573.729918] type=1400 audit(1315311829.674:734): avc:  denied  { read } for  pid=25669 comm="dnsmasq" name="sh" dev=dm-1 ino=131075 scontext=system_u:system_r:dnsmasq_t:s0 tcontext=system_u:object_r:bin_t:s0 tclass=lnk_file
Sep  6 13:23:49 zig dnsmasq[25668]: cannot run lease-init script /usr/bin/nova-dhcpbridge: No such file or directory
Sep  6 13:23:49 zig dnsmasq[25668]: FAILED to start up

log for this one:

(nova): TRACE: Command: sudo FLAGFILE=/etc/nova/nova.conf NETWORK_ID=1 dnsmasq --strict-order --bind-interfaces --interface=br0 --conf-file= --domain=novalocal --pid-file=/var/lib/nova/networks/nova-br0.pid --listen-address=10.0.0.1 --except-interface=lo --dhcp-range=10.0.0.2,static,120s --dhcp-lease-max=256 --dhcp-hostsfile=/var/lib/nova/networks/nova-br0.conf --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro
(nova): TRACE: Exit code: 3
(nova): TRACE: Stdout: ''
(nova): TRACE: Stderr: '\ndnsmasq: cannot run lease-init script /usr/bin/nova-dhcpbridge: No such file or directory\n'

Comment 3 Alan Pevec 2011-10-10 19:48:55 UTC
Shouldn't this be filed against selinux-policy component?

Comment 4 Mark McLoughlin 2011-10-11 05:57:38 UTC
Fair point, thanks Alan

Comment 5 Miroslav Grepl 2011-10-11 06:59:19 UTC
I am looking on it. What is your output of

# ps -eZ |grep initrc

Comment 6 Miroslav Grepl 2011-10-11 13:56:38 UTC
I am getting

# ps -eZ |grep initrc
staff_u:system_r:initrc_t:s0    10220 ?        00:00:00 epmd
staff_u:system_r:initrc_t:s0    10234 ?        00:00:00 beam.smp
staff_u:system_r:initrc_t:s0    10304 ?        00:00:00 cpu_sup
staff_u:system_r:initrc_t:s0    10308 ?        00:00:00 inet_gethost
staff_u:system_r:initrc_t:s0    10309 ?        00:00:00 inet_gethost

Comment 7 Miroslav Grepl 2011-10-12 12:06:02 UTC
I am creating policy for these daemons and also for all openstack-nova-* daemons. I will send it to rkukura for testing since it is quite a lot of work.

Comment 8 Mark McLoughlin 2011-10-12 12:40:23 UTC
(In reply to comment #7)
> I am creating policy for these daemons and also for all openstack-nova-*
> daemons. I will send it to rkukura for testing since it is quite a lot of work.

Awesome, thanks Miroslav

Comment 9 Miroslav Grepl 2011-10-12 14:56:26 UTC
I created initial policies, but I overlooked part of your comment

> Note also the sudo and polkit configuration - the daemons run a whole bunch of
> privileged commands

which mean you run iptables, lvm, ifconfing and so on using sudo and i see AVC like

type=AVC msg=audit(1318426149.510:2939): avc:  denied  { execute } for  pid=7317 comm="sudo" name="xtables-multi" dev=dm-2 ino=2883601
scontext=system_u:system_r:nova_api_t:s0 tcontext=system_u:object_r:iptables_exec_t:s0  tclass=file

# grep sudo /var/log/audit/audit.log | wc -l
245

which means you do almost "everything" from SELinux point of view. So you can not expect SElinux could lock it down.

Comment 10 Miroslav Grepl 2011-10-12 14:59:48 UTC
nova_api_t, nova_network_t, nova_volume_t run sudo. Well, we could make this domain as unconfined but still you run "sudo" directly in the code.

Comment 11 Matt Domsch 2011-10-12 17:42:09 UTC
The use of sudo is recognized by upstream as "a really bad idea" and there are efforts to remedy the problem in the forthcoming Essex development.

Comment 12 Miroslav Grepl 2011-10-12 18:28:26 UTC
Great.

Comment 13 Miroslav Grepl 2011-10-13 15:12:56 UTC
Created attachment 528051 [details]
Policy for nova daemons

Comment 14 Miroslav Grepl 2011-10-13 15:13:41 UTC
Created attachment 528054 [details]
Policy for rabbitmq

Comment 15 Miroslav Grepl 2011-10-13 15:16:21 UTC
How to test:

Just unpack archives and run

# sh nova.sh
# sh rabbitmq.sh
# echo "-w /etc/shadow -p wa" >> /etc/audit/audit.rules 
# service auditd restart

and restart nova and rabbitmq daemons and test it.

Then I would like to get /var/log/audit/audit.log.

Comment 16 Bob Kukura 2011-10-13 17:16:53 UTC
I'll give this a try this afternoon.

Comment 17 Bob Kukura 2011-10-17 03:30:43 UTC
Created attachment 528426 [details]
commands and audit.log output

I've added an attachment containing commands starting rabbitmq, glance, and nova, and running through an openstack test scenario after having applied the policies. Output from audit.log is interspersed.

Comment 18 Miroslav Grepl 2011-10-18 18:10:40 UTC
I am adding fixes but I am seeing

allow nova_compute_t svirt_image_t:file { read write getattr open setattr };


could you explain me how virt machine are started?

Comment 19 Miroslav Grepl 2011-10-19 09:00:12 UTC
Fixed in selinux-policy-3.10.0-43.fc16

Comment 20 Fedora Update System 2011-10-19 10:23:39 UTC
selinux-policy-3.10.0-43.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/selinux-policy-3.10.0-43.fc16

Comment 21 Fedora Update System 2011-10-20 02:24:13 UTC
Package selinux-policy-3.10.0-43.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing selinux-policy-3.10.0-43.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-14618
then log in and leave karma (feedback).

Comment 22 Fedora Update System 2011-10-22 00:11:05 UTC
Package selinux-policy-3.10.0-45.1.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing selinux-policy-3.10.0-45.1.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-14618
then log in and leave karma (feedback).

Comment 23 Fedora Update System 2011-10-25 03:34:16 UTC
selinux-policy-3.10.0-45.1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.