Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 5 product line. The current stable release is 5.10. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 462702

Summary: xenstored fails to start because selinux prevents access to files on tmpfs
Product: Red Hat Enterprise Linux 5 Reporter: Richard W.M. Jones <rjones>
Component: xenAssignee: Richard W.M. Jones <rjones>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: medium    
Version: 5.3CC: berrange, clalance, ddutile, dwalsh, gozen, herrold, jturner, riek, sputhenp, syeghiay, xen-maint
Target Milestone: betaKeywords: Reopened
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-01-20 21:14:52 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/var/log/messages
none
/var/log/audit/audit.log
none
Patch to /etc/init.d/xend
none
Patch to /etc/init.d/xend
none
xend-restorecon-xenstored-tmpfs.patch
none
xend-restorecon-xenstored-tmpfs.patch none

Description Richard W.M. Jones 2008-09-18 13:54:30 UTC
Description of problem:

When a machine running a 5.3 nightly build is booted into Xen,
xenstored fails to start.  (xend is OK though).

The appears to be because SELinux is preventing access to
xenstored's temporary files which are now stored on a tmpfs
partition.  The error message is:

type=1400 audit(1221744582.491:6): avc:  denied  { search } for  pid=3596 comm="xenstored" name="/" dev=tmpfs ino=11386 scontext=system_u:system_r:xenstored_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=dir
type=1400 audit(1221744582.491:7): avc:  denied  { search } for  pid=3596 comm="xenstored" name="/" dev=tmpfs ino=11386 scontext=system_u:system_r:xenstored_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=dir

$ mount
/dev/mapper/VolGroup-RHEL52i386 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/mapper/VolGroup-Share on /mnt/share type ext3 (rw)
/dev/sda2 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
none on /var/lib/xenstored type tmpfs (rw)

Version-Release number of selected component (if applicable):

$ rpm -qa | grep policy
policycoreutils-gui-1.33.12-14.el5
selinux-policy-targeted-2.4.6-154.el5
checkpolicy-1.33.1-4.el5
selinux-policy-2.4.6-154.el5
policycoreutils-1.33.12-14.el5

$ rpm -qa | grep xen
xen-devel-3.0.3-72.el5
xen-libs-3.0.3-72.el5
kernel-xen-2.6.18-115.el5
xen-3.0.3-72.el5

How reproducible:

Always, if SELinux is enabled at boot time and the machine
is booted into the Xen HV / kernel-xen kernel.

Works fine if I add selinux=0 on the kernel line.

Steps to Reproduce:
1. Install RHEL 5.3 nightly (I'm using 20080918)
2. Boot into Xen kernel.
  
Actual results:

This is what you see with SELinux enabled:

# ps ax | grep xen
   21 ?        S<     0:00 [xenwatch]
   22 ?        S<     0:00 [xenbus]
10021 ?        S      0:00 python /usr/sbin/xend start
10023 ?        Sl     0:00 python /usr/sbin/xend start
10029 pts/2    S+     0:00 grep xen

Expected results:

This is what you see without SELinux:

# ps ax | grep xen
   21 ?        S<     0:00 [xenwatch]
   22 ?        S<     0:00 [xenbus]
 3583 ?        S      0:00 xenstored --pid-file /var/run/xenstore.pid
 3588 ?        S      0:00 python /usr/sbin/xend start
 3589 ?        Sl     0:00 python /usr/sbin/xend start
 3591 ?        Sl     0:00 xenconsoled --log none --timestamp none --log-dir /var/log/xen/console
 4258 pts/1    S+     0:00 grep xen

Additional info:

I think there might be the same problem with xenconsoled ...
I'm going to check that and add any additional information
to this BZ.

Comment 1 Richard W.M. Jones 2008-09-18 14:16:46 UTC
Created attachment 317084 [details]
/var/log/messages

I booted the machine with selinux permissive and there
seem to be further xenstored permissions problems.

Doesn't seem to be any selinux-related problem with
xenconsoled.  Probably xenconsoled fails because it needs
to talk to xenstored.

Attached is the cut down /var/log/messages from after a
permissive boot.

Comment 2 Richard W.M. Jones 2008-09-18 14:18:11 UTC
Created attachment 317085 [details]
/var/log/audit/audit.log

This is the complete audit.log from the machine, so only the last part 
is relevant to the permissive boot.

Comment 3 Daniel Walsh 2008-09-18 17:22:25 UTC
xenstored seems to be creating files/directories in a tmpfs_t directory?

Comment 4 Daniel Walsh 2008-09-18 17:25:02 UTC
/var/lib/xenstored/tdb is a tmpfs_t file system?

Comment 5 Daniel Walsh 2008-09-18 17:27:26 UTC
none on /var/lib/xenstored type tmpfs (rw)
Is the problem.

If you want to do this you need to mount this file system with the correct context


# matchpathcon /var/lib/xenstored
/var/lib/xenstored	system_u:object_r:xenstored_var_lib_t:s0

So you would need to use -o context="system_u:object_r:xenstored_var_lib_t:s0"

Comment 6 Richard W.M. Jones 2008-09-18 17:46:47 UTC
Just to be clear here, "I" (as in me, the administrator of the machine)
didn't mount this directory or change anything about the way it
is mounted.  This is a plain 5.2 install upgraded with 5.3 packages
from our nightly builds, and it fails to start Xen correctly when SELinux
is enabled.

It turns out that the tmpfs directory is created by /etc/init.d/xend,
so I'm reopening the bug and changing the component to xen which
seems more likely.

Comment 7 Richard W.M. Jones 2008-09-18 17:57:05 UTC
Created attachment 317102 [details]
Patch to /etc/init.d/xend

Suggested patch to /etc/init.d/xend based on dwalsh's comment.

Comment 8 Bill Burns 2008-09-18 19:48:05 UTC
So are those args silently ignored when selinux is disabled?

Comment 10 Daniel Walsh 2008-09-18 20:20:57 UTC
Ok add 

restorecon -R -v /var/lib/xenstored

To init script and that will fix it also, and will be ignored when selinux is disabled.

Comment 11 Richard W.M. Jones 2008-09-18 20:46:59 UTC
Sorry, I'm well out of my depth with these SELinux commands.

Do I need to add both the restorecon command _and_ the mount
option?  Or just the restorecon command?  Do I run restorecon
after the mount (I assume so)?

Comment 12 Daniel Walsh 2008-09-18 21:04:13 UTC
Yes no mount option necessary.

restorecon after the mount.

All this is doing is placing the correct labels on the file system you are creating/mounting.

Comment 13 Richard W.M. Jones 2008-09-18 21:23:58 UTC
Created attachment 317127 [details]
Patch to /etc/init.d/xend

Second attempt at a patch.

Not tested yet - I'll test it tomorrow morning.

Comment 14 Don Dutile (Red Hat) 2008-09-18 22:03:23 UTC
I upgraded my rhel5.2 system to the same nightly rhel5.3,
edited the /etc/selinux/config to default to enforcing,
and xenstored was running as expected:

# ps -ef | grep xen
root        21    19  0 17:44 ?        00:00:00 [xenwatch]
root        22    19  0 17:44 ?        00:00:00 [xenbus]
root      3611     1  0 17:44 ?        00:00:00 xenstored --pid-file /var/run/xenstore.pid
root      3626     1  0 17:44 ?        00:00:00 python /usr/sbin/xend start
root      3627  3626  0 17:44 ?        00:00:00 python /usr/sbin/xend start
root      3629     1  0 17:44 ?        00:00:00 xenconsoled --log none --log-dir /var/log/xen/console
root      4666  4625  0 17:51 pts/2    00:00:00 grep xen


Now there is another config option in the /etc/selinux/config:
# SELINUXTYPE= type of policy in use. Possible values are:
#       targeted - Only targeted network daemons are protected.
#       strict - Full SELinux protection.

my test system was set to 'targeted'.

If I changed this setting to 'strict', the system generated an SELINUX
failure message during boot & forced a reboot; had to use the selinux=0
param to the kernel boot to get it to reboot & change everything back.

btw: here's the rpm status of selinux on the system after the update,
in case it's differnt than an install:
# rpm -qa | grep selinux
selinux-policy-2.4.6-154.el5
libselinux-1.33.4-5.el5
libselinux-1.33.4-5.el5             <== pizza, pizza ???
libselinux-python-1.33.4-5.el5
libselinux-devel-1.33.4-5.el5
libselinux-devel-1.33.4-5.el5       <== hmm... maybe one x86_64, one i386 ?
selinux-policy-targeted-2.4.6-154.el5


So, I couldn't repro the stated failure, but maybe there's
some other subtle diff btwn an updated/upgraded 5.3 & an cleanly
installed 5.3.

Comment 15 Chris Lalancette 2008-09-19 06:31:40 UTC
Don,
     Hm, that's odd.  Two things to check:

1.  Do you have /var/lib/xenstored mounted as a tmpfs?
2.  What does the variable XENSTORED_TMPFS say in /etc/init.d/xend?

Chris Lalancette

Comment 16 Richard W.M. Jones 2008-09-19 08:55:42 UTC
Don,

The only significant difference seems to be that I've
installed i386 RHEL.  I can't see how that would change
anything though.  Can you run 'mount' and see if /var/lib/
xenstored is mounted separated as tmpfs, ie:

  # mount | grep xenstored
  none on /var/lib/xenstored type tmpfs (rw)

- - -

In reply to my comment 13, I have tested this patch and
it appears to fix the problem, ie. xenstored starts
up OK.

Comment 17 Richard W.M. Jones 2008-09-19 09:48:30 UTC
Created attachment 317172 [details]
xend-restorecon-xenstored-tmpfs.patch

This is the rebased, POSTed patch.

Comment 18 Richard W.M. Jones 2008-09-19 10:43:05 UTC
Created attachment 317180 [details]
xend-restorecon-xenstored-tmpfs.patch

This removes the -v option (thus suppressing a debug message
at boot time).

Comment 19 Don Dutile (Red Hat) 2008-09-19 13:30:16 UTC
re: from comment #15:

1.  Do you have /var/lib/xenstored mounted as a tmpfs?
     -- NOPE!.... that would explain the lack of failure

2.  What does the variable XENSTORED_TMPFS say in /etc/init.d/xend?
     -- it doesn't exist in the file.

- Don

Comment 21 Daniel Berrangé 2008-09-19 13:58:58 UTC
Fix built into xen-3.0.3-73.el5

Comment 23 Jay Turner 2008-09-19 14:42:28 UTC
QE ack for 5.3.

Comment 27 errata-xmlrpc 2009-01-20 21:14:52 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0118.html