Bug 462702 - xenstored fails to start because selinux prevents access to files on tmpfs
xenstored fails to start because selinux prevents access to files on tmpfs
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: xen (Show other bugs)
5.3
i386 Linux
medium Severity high
: beta
: ---
Assigned To: Richard W.M. Jones
Virtualization Bugs
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-09-18 09:54 EDT by Richard W.M. Jones
Modified: 2009-12-14 16:12 EST (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-01-20 16:14:52 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
/var/log/messages (37.18 KB, text/plain)
2008-09-18 10:16 EDT, Richard W.M. Jones
no flags Details
/var/log/audit/audit.log (155.30 KB, text/plain)
2008-09-18 10:18 EDT, Richard W.M. Jones
no flags Details
Patch to /etc/init.d/xend (564 bytes, patch)
2008-09-18 13:57 EDT, Richard W.M. Jones
no flags Details | Diff
Patch to /etc/init.d/xend (562 bytes, patch)
2008-09-18 17:23 EDT, Richard W.M. Jones
no flags Details | Diff
xend-restorecon-xenstored-tmpfs.patch (634 bytes, patch)
2008-09-19 05:48 EDT, Richard W.M. Jones
no flags Details | Diff
xend-restorecon-xenstored-tmpfs.patch (631 bytes, patch)
2008-09-19 06:43 EDT, Richard W.M. Jones
no flags Details | Diff

  None (edit)
Description Richard W.M. Jones 2008-09-18 09:54:30 EDT
Description of problem:

When a machine running a 5.3 nightly build is booted into Xen,
xenstored fails to start.  (xend is OK though).

The appears to be because SELinux is preventing access to
xenstored's temporary files which are now stored on a tmpfs
partition.  The error message is:

type=1400 audit(1221744582.491:6): avc:  denied  { search } for  pid=3596 comm="xenstored" name="/" dev=tmpfs ino=11386 scontext=system_u:system_r:xenstored_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=dir
type=1400 audit(1221744582.491:7): avc:  denied  { search } for  pid=3596 comm="xenstored" name="/" dev=tmpfs ino=11386 scontext=system_u:system_r:xenstored_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=dir

$ mount
/dev/mapper/VolGroup-RHEL52i386 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/mapper/VolGroup-Share on /mnt/share type ext3 (rw)
/dev/sda2 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
none on /var/lib/xenstored type tmpfs (rw)

Version-Release number of selected component (if applicable):

$ rpm -qa | grep policy
policycoreutils-gui-1.33.12-14.el5
selinux-policy-targeted-2.4.6-154.el5
checkpolicy-1.33.1-4.el5
selinux-policy-2.4.6-154.el5
policycoreutils-1.33.12-14.el5

$ rpm -qa | grep xen
xen-devel-3.0.3-72.el5
xen-libs-3.0.3-72.el5
kernel-xen-2.6.18-115.el5
xen-3.0.3-72.el5

How reproducible:

Always, if SELinux is enabled at boot time and the machine
is booted into the Xen HV / kernel-xen kernel.

Works fine if I add selinux=0 on the kernel line.

Steps to Reproduce:
1. Install RHEL 5.3 nightly (I'm using 20080918)
2. Boot into Xen kernel.
  
Actual results:

This is what you see with SELinux enabled:

# ps ax | grep xen
   21 ?        S<     0:00 [xenwatch]
   22 ?        S<     0:00 [xenbus]
10021 ?        S      0:00 python /usr/sbin/xend start
10023 ?        Sl     0:00 python /usr/sbin/xend start
10029 pts/2    S+     0:00 grep xen

Expected results:

This is what you see without SELinux:

# ps ax | grep xen
   21 ?        S<     0:00 [xenwatch]
   22 ?        S<     0:00 [xenbus]
 3583 ?        S      0:00 xenstored --pid-file /var/run/xenstore.pid
 3588 ?        S      0:00 python /usr/sbin/xend start
 3589 ?        Sl     0:00 python /usr/sbin/xend start
 3591 ?        Sl     0:00 xenconsoled --log none --timestamp none --log-dir /var/log/xen/console
 4258 pts/1    S+     0:00 grep xen

Additional info:

I think there might be the same problem with xenconsoled ...
I'm going to check that and add any additional information
to this BZ.
Comment 1 Richard W.M. Jones 2008-09-18 10:16:46 EDT
Created attachment 317084 [details]
/var/log/messages

I booted the machine with selinux permissive and there
seem to be further xenstored permissions problems.

Doesn't seem to be any selinux-related problem with
xenconsoled.  Probably xenconsoled fails because it needs
to talk to xenstored.

Attached is the cut down /var/log/messages from after a
permissive boot.
Comment 2 Richard W.M. Jones 2008-09-18 10:18:11 EDT
Created attachment 317085 [details]
/var/log/audit/audit.log

This is the complete audit.log from the machine, so only the last part 
is relevant to the permissive boot.
Comment 3 Daniel Walsh 2008-09-18 13:22:25 EDT
xenstored seems to be creating files/directories in a tmpfs_t directory?
Comment 4 Daniel Walsh 2008-09-18 13:25:02 EDT
/var/lib/xenstored/tdb is a tmpfs_t file system?
Comment 5 Daniel Walsh 2008-09-18 13:27:26 EDT
none on /var/lib/xenstored type tmpfs (rw)
Is the problem.

If you want to do this you need to mount this file system with the correct context


# matchpathcon /var/lib/xenstored
/var/lib/xenstored	system_u:object_r:xenstored_var_lib_t:s0

So you would need to use -o context="system_u:object_r:xenstored_var_lib_t:s0"
Comment 6 Richard W.M. Jones 2008-09-18 13:46:47 EDT
Just to be clear here, "I" (as in me, the administrator of the machine)
didn't mount this directory or change anything about the way it
is mounted.  This is a plain 5.2 install upgraded with 5.3 packages
from our nightly builds, and it fails to start Xen correctly when SELinux
is enabled.

It turns out that the tmpfs directory is created by /etc/init.d/xend,
so I'm reopening the bug and changing the component to xen which
seems more likely.
Comment 7 Richard W.M. Jones 2008-09-18 13:57:05 EDT
Created attachment 317102 [details]
Patch to /etc/init.d/xend

Suggested patch to /etc/init.d/xend based on dwalsh's comment.
Comment 8 Bill Burns 2008-09-18 15:48:05 EDT
So are those args silently ignored when selinux is disabled?
Comment 10 Daniel Walsh 2008-09-18 16:20:57 EDT
Ok add 

restorecon -R -v /var/lib/xenstored

To init script and that will fix it also, and will be ignored when selinux is disabled.
Comment 11 Richard W.M. Jones 2008-09-18 16:46:59 EDT
Sorry, I'm well out of my depth with these SELinux commands.

Do I need to add both the restorecon command _and_ the mount
option?  Or just the restorecon command?  Do I run restorecon
after the mount (I assume so)?
Comment 12 Daniel Walsh 2008-09-18 17:04:13 EDT
Yes no mount option necessary.

restorecon after the mount.

All this is doing is placing the correct labels on the file system you are creating/mounting.
Comment 13 Richard W.M. Jones 2008-09-18 17:23:58 EDT
Created attachment 317127 [details]
Patch to /etc/init.d/xend

Second attempt at a patch.

Not tested yet - I'll test it tomorrow morning.
Comment 14 Don Dutile 2008-09-18 18:03:23 EDT
I upgraded my rhel5.2 system to the same nightly rhel5.3,
edited the /etc/selinux/config to default to enforcing,
and xenstored was running as expected:

# ps -ef | grep xen
root        21    19  0 17:44 ?        00:00:00 [xenwatch]
root        22    19  0 17:44 ?        00:00:00 [xenbus]
root      3611     1  0 17:44 ?        00:00:00 xenstored --pid-file /var/run/xenstore.pid
root      3626     1  0 17:44 ?        00:00:00 python /usr/sbin/xend start
root      3627  3626  0 17:44 ?        00:00:00 python /usr/sbin/xend start
root      3629     1  0 17:44 ?        00:00:00 xenconsoled --log none --log-dir /var/log/xen/console
root      4666  4625  0 17:51 pts/2    00:00:00 grep xen


Now there is another config option in the /etc/selinux/config:
# SELINUXTYPE= type of policy in use. Possible values are:
#       targeted - Only targeted network daemons are protected.
#       strict - Full SELinux protection.

my test system was set to 'targeted'.

If I changed this setting to 'strict', the system generated an SELINUX
failure message during boot & forced a reboot; had to use the selinux=0
param to the kernel boot to get it to reboot & change everything back.

btw: here's the rpm status of selinux on the system after the update,
in case it's differnt than an install:
# rpm -qa | grep selinux
selinux-policy-2.4.6-154.el5
libselinux-1.33.4-5.el5
libselinux-1.33.4-5.el5             <== pizza, pizza ???
libselinux-python-1.33.4-5.el5
libselinux-devel-1.33.4-5.el5
libselinux-devel-1.33.4-5.el5       <== hmm... maybe one x86_64, one i386 ?
selinux-policy-targeted-2.4.6-154.el5


So, I couldn't repro the stated failure, but maybe there's
some other subtle diff btwn an updated/upgraded 5.3 & an cleanly
installed 5.3.
Comment 15 Chris Lalancette 2008-09-19 02:31:40 EDT
Don,
     Hm, that's odd.  Two things to check:

1.  Do you have /var/lib/xenstored mounted as a tmpfs?
2.  What does the variable XENSTORED_TMPFS say in /etc/init.d/xend?

Chris Lalancette
Comment 16 Richard W.M. Jones 2008-09-19 04:55:42 EDT
Don,

The only significant difference seems to be that I've
installed i386 RHEL.  I can't see how that would change
anything though.  Can you run 'mount' and see if /var/lib/
xenstored is mounted separated as tmpfs, ie:

  # mount | grep xenstored
  none on /var/lib/xenstored type tmpfs (rw)

- - -

In reply to my comment 13, I have tested this patch and
it appears to fix the problem, ie. xenstored starts
up OK.
Comment 17 Richard W.M. Jones 2008-09-19 05:48:30 EDT
Created attachment 317172 [details]
xend-restorecon-xenstored-tmpfs.patch

This is the rebased, POSTed patch.
Comment 18 Richard W.M. Jones 2008-09-19 06:43:05 EDT
Created attachment 317180 [details]
xend-restorecon-xenstored-tmpfs.patch

This removes the -v option (thus suppressing a debug message
at boot time).
Comment 19 Don Dutile 2008-09-19 09:30:16 EDT
re: from comment #15:

1.  Do you have /var/lib/xenstored mounted as a tmpfs?
     -- NOPE!.... that would explain the lack of failure

2.  What does the variable XENSTORED_TMPFS say in /etc/init.d/xend?
     -- it doesn't exist in the file.

- Don
Comment 21 Daniel Berrange 2008-09-19 09:58:58 EDT
Fix built into xen-3.0.3-73.el5
Comment 23 Jay Turner 2008-09-19 10:42:28 EDT
QE ack for 5.3.
Comment 27 errata-xmlrpc 2009-01-20 16:14:52 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0118.html

Note You need to log in before you can comment on or make changes to this bug.