Bug 504238 - kvm guest installations are failing with selinux-policy-targeted < 2.4.6-245.el5
kvm guest installations are failing with selinux-policy-targeted < 2.4.6-245.el5
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: selinux-policy-targeted (Show other bugs)
5.4
All Linux
high Severity high
: beta
: ---
Assigned To: Daniel Walsh
BaseOS QE
: Regression, TestBlocker
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-06-04 18:20 EDT by Gurhan Ozen
Modified: 2011-09-20 05:40 EDT (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-09-02 03:58:05 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Gurhan Ozen 2009-06-04 18:20:59 EDT
Description of problem:
Not sure what's going on but all kvm guests are failing with the same reason on rhts with 20090604.nightly tree. I have manually run an installation with --debug option and here is what I got:

# virt-install --name x86_64_hvm_guest_3 --cdrom /var/lib/libvirt/images/x86_64_hvm_guest.iso --nonsparse --hvm --file /var/lib/libvirt/images/x86_64_hvm_guest_3 -s 10 --extra-args ks=http://lab.rhts.bos.redhat.com/kickstarts/hosts/guest-81-16.rhts.bos.redhat.com/ks.cfg --prompt --accelerate --os-variant=virtio26  --noreboot --debug
Thu, 04 Jun 2009 16:57:17 DEBUG    Using libvirt URI 'qemu:///system'
Thu, 04 Jun 2009 16:57:17 DEBUG    Requesting virt method 'hvm'
Thu, 04 Jun 2009 16:57:17 DEBUG    Received virt method 'hvm'
Thu, 04 Jun 2009 16:57:17 DEBUG    Hypervisor name is 'kvm'
How much RAM should be allocated (in megabytes)? 512
Thu, 04 Jun 2009 16:57:23 DEBUG    DISPLAY is set: graphics defaulting to VNC.
Thu, 04 Jun 2009 16:57:23 DEBUG    Setting os type to 'linux' for variant 'virtio26'
Thu, 04 Jun 2009 16:57:23 DEBUG    DistroInstaller location is a local file/path: /var/lib/libvirt/images/x86_64_hvm_guest.iso
Thu, 04 Jun 2009 16:57:23 DEBUG    Setting size for existing storage to '0.00880241394043'
Thu, 04 Jun 2009 16:57:23 DEBUG    Detected storage as type 'file'
 

Starting install...
Thu, 04 Jun 2009 16:57:24 DEBUG    Setting size for existing storage to '0.00880241394043'
Thu, 04 Jun 2009 16:57:24 DEBUG    Detected storage as type 'file'
Creating storage file...                                                                                      |  10 GB     01:08     
Thu, 04 Jun 2009 16:58:33 DEBUG    Creating guest from:
<domain type='kvm'>
  <name>x86_64_hvm_guest_3</name>
  <currentMemory>524288</currentMemory>
  <memory>524288</memory>
  <uuid>6b25e622-84e8-679e-fe74-7a4ce3e69239</uuid>
  <os>
    <type arch='x86_64'>hvm</type>
    <boot dev='cdrom'/>
  </os>
  <features>
    <acpi/><apic/><pae/>
  </features>
  <clock offset="utc"/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <vcpu>1</vcpu>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <console type='pty'/>
    <disk type='file' device='disk'>
      <source file='/var/lib/libvirt/images/x86_64_hvm_guest_3'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <disk type='file' device='cdrom'>
      <source file='/var/lib/libvirt/images/x86_64_hvm_guest.iso'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
    </disk>
    <interface type='network'>
      <source network='default'/>
      <mac address='54:52:00:63:d9:d9'/>
      <model type='virtio'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' keymap='en-us'/>
  </devices>
</domain>

internal error Domain x86_64_hvm_guest_3 didn't show up

Domain installation may not have been
 successful.  If it was, you can restart your domain
 by running 'virsh start x86_64_hvm_guest_3'; otherwise, please
 restart your installation.
Thu, 04 Jun 2009 16:58:43 ERROR    internal error Domain x86_64_hvm_guest_3 didn't show up
Traceback (most recent call last):
  File "/usr/sbin/virt-install", line 865, in ?
    main()
  File "/usr/sbin/virt-install", line 763, in main
    start_time, guest.start_install)
  File "/usr/sbin/virt-install", line 818, in do_install
    dom = install_func(conscb, progresscb, wait=(not wait))
  File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 541, in start_install
    return self._do_install(consolecb, meter, removeOld, wait)
  File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 633, in _do_install
    self.domain = self.conn.createLinux(install_xml, 0)
  File "/usr/lib64/python2.4/site-packages/libvirt.py", line 974, in createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self)
libvirtError: internal error Domain x86_64_hvm_guest_3 didn't show up


This is happening for both x86_64 and i386 guests. We have also got the following AVC messages popping up:

time->Thu Jun  4 13:01:34 2009
type=SYSCALL msg=audit(1244134894.840:15): arch=c000003e syscall=2 success=no exit=-13 a0=7fff23ef2e11 a1=42 a2=180 a3=0 items=0 ppid=1 pid=5229 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/bin/qemu-kvm" subj=system_u:system_r:qemu_t:s0-s0:c0.c1023 key=(null)
type=AVC msg=audit(1244134894.840:15): avc:  denied  { write } for  pid=5229 comm="qemu-kvm" name="qemu" dev=dm-0 ino=10715432 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:virt_var_run_t:s0 tclass=dir

Just as FYI, I have tried the same tree with a -152 kernel but got the same failure as well. Installations were happening successfully for me, as recent as 20090602.nightly tree. 

Version-Release number of selected component (if applicable):
[root@dell-pe2900-03 ~]# uname -a 
Linux dell-pe2900-03.rhts.bos.redhat.com 2.6.18-151.el5 #1 SMP Wed May 27 16:14:57 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
#  rpm -qa | egrep "kvm|libvirt"
kmod-kvm-83-59.el5
libvirt-0.6.3-4.el5
libvirt-0.6.3-4.el5
kvm-83-59.el5
libvirt-python-0.6.3-4.el5
rh-tests-virt-libvirt-migrate-2.0-6
etherboot-zroms-kvm-5.4.4-10.el5


How reproducible:
Very

Steps to Reproduce:
1. Install 20090604.nightly tree and try to install kvm guests. 
2.
3.
  
Actual results:
Fails.

Expected results:
Should install.

Additional info:
Comment 1 Chris Lalancette 2009-06-05 05:05:56 EDT
This seems to have started happening because of the change in selinux policy between the 20090602 and 20090604 trees.  Gurhan downgraded the selinux policy to the policy from 20090602, and things started working again.  Moving over to selinux-policy to have the selinux people look at it.

Chris Lalancette
Comment 5 Daniel Walsh 2009-06-05 11:14:33 EDT
Fixed in selinux-policy-2.4.6-242.el5
Comment 8 Gurhan Ozen 2009-06-05 15:34:07 EDT
(In reply to comment #5)
> Fixed in selinux-policy-2.4.6-242.el5  

Fails....

[root@tyan-gt24-05 ~]# uname -a ;  rpm -qa | grep selinux-policy
Linux tyan-gt24-05.rhts.bos.redhat.com 2.6.18-152.el5 #1 SMP Wed Jun 3 18:57:00 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
selinux-policy-2.4.6-242.el5
selinux-policy-targeted-2.4.6-242.el5
[root@tyan-gt24-05 ~]#  /sbin/ausearch -sv no -m AVC -m USER_AVC -m SELINUX_ERR -ts 6/5/2009 14:2:15
----
time->Fri Jun  5 14:04:09 2009
type=SYSCALL msg=audit(1244225049.921:9): arch=c000003e syscall=2 success=no exit=-13 a0=541cab a1=202 a2=0 a3=0 items=0 ppid=1 pid=4906 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/bin/qemu-kvm" subj=system_u:system_r:qemu_t:s0-s0:c0.c1023 key=(null)
type=AVC msg=audit(1244225049.921:9): avc:  denied  { read write } for  pid=4906 comm="qemu-kvm" name="ksm" dev=tmpfs ino=6132 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:device_t:s0 tclass=chr_file
----
time->Fri Jun  5 14:10:10 2009
type=SYSCALL msg=audit(1244225410.032:14): arch=c000003e syscall=2 success=no exit=-13 a0=541cab a1=202 a2=0 a3=0 items=0 ppid=1 pid=6573 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/bin/qemu-kvm" subj=system_u:system_r:qemu_t:s0-s0:c0.c1023 key=(null)
type=AVC msg=audit(1244225410.032:14): avc:  denied  { read write } for  pid=6573 comm="qemu-kvm" name="ksm" dev=tmpfs ino=6132 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:device_t:s0 tclass=chr_file
[root@tyan-gt24-05 ~]#
Comment 9 Daniel Walsh 2009-06-06 06:49:22 EDT
What is a /dev/ksm?  Never seen one before.
Comment 10 Daniel Walsh 2009-06-06 06:54:33 EDT
More importantly what is the security properties of this device?  Should eveyone be allowed to read/write it?  All qemu processes?
Comment 11 Chris Lalancette 2009-06-08 02:26:27 EDT
/dev/ksm is a character device used (via ioctl's, I believe) to control the ksm module.  The ksm module is responsible for doing page sharing among processes.  To be honest, I'm not sure what the security properties should be; from my understanding, there should be one daemon in the system that talks to the /dev/ksm device and controls how often the scanning is done, etc.  But I'm not intimately involved with it, so I could be wrong.  Izik, can you answer Comment #10?

Chris Lalancette
Comment 12 Izik Eidus 2009-06-08 06:41:43 EDT
/dev/ksm should be accessed only by the kvm-user or the root user.

every qemu process should have access to /dev/ksm, but not normal users.

Thanks.


(letting normal users control ksm, may lead to situation of allocating a big amount of unswappable memory)
Comment 13 Chris Lalancette 2009-06-08 07:33:43 EDT
(In reply to comment #12)
> /dev/ksm should be accessed only by the kvm-user or the root user.
> 
> every qemu process should have access to /dev/ksm, but not normal users.

Hm, I'm a bit confused though.  Wasn't the idea of ksm is that it is transparent to processes?  That is, why should qemu touch /dev/ksm?  Shouldn't qemu just do the normal things that it does, and ksm will transparently share the pages behind it's back?  Or am I missing something with how ksm works (certainly possible)?

Chris Lalancette
Comment 14 Izik Eidus 2009-06-08 07:51:20 EDT
ksm is transparent to the process, but ksm need what process to scan...
so all qemu is doing is just open /dev/ksm and say "scan me"...

in the mainline version we can do it with ksmctl -pid....

but qemu need access to /dev/ksm in order to tell it to scan the memory.
Comment 15 Chris Lalancette 2009-06-08 08:04:30 EDT
Ah, OK, gotcha.  Thanks for clarifying.  So, it looks like this is a RHEL specific change to the policy.  Dan Walsh, does this clear things up for you?  Can you fix up the policy for RHEL-5 now?

Chris Lalancette
Comment 16 Daniel Walsh 2009-06-08 09:10:12 EDT
I need to define a new type for /dev/ksm but ut from an SELinux point of view qemu processes should ot be trusted.  Think of a hacker figuring out a way to compromize your guest OS (windows)  then we have a bug in qemu/kvm that allows him to break out of the qemu process and take it over.   What kind of attack can he mount on the host system through the /dev/ksm?  What kind of attack can he run against other qemu processes running guest os?
Comment 17 Izik Eidus 2009-06-08 09:39:38 EDT
he can make hugh amount of unswappable memory allocated. he can run the whole system into condion of out of memory.


the only thing that /dev/ksm is need is: the kvm user and the root user will have access to /dev/ksm

in our virtualziation soultion we have kvm user right?
Comment 18 Daniel Walsh 2009-06-08 15:02:00 EDT
Ok as long  as he can only crash the machine, he can not subvirt other processes or take over the machine.



Fixed in selinux-policy-2.4.6-243.el5
Comment 19 Gurhan Ozen 2009-06-08 17:25:37 EDT
(In reply to comment #18)
> Ok as long  as he can only crash the machine, he can not subvirt other
> processes or take over the machine.
> 
> 
> 
> Fixed in selinux-policy-2.4.6-243.el5  

Fails..

type=AVC msg=audit(1244492216.620:8): avc:  denied  { append } for  pid=5666 comm="qemu-kvm" path="/var/log/libvirt/qemu/rhel5Latest_x86_64_hvm_guest_kvm.log" dev=dm-0 ino=20578746 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:virt_log_t:s0 tclass=file
type=AVC msg=audit(1244492216.620:8): avc:  denied  { append } for  pid=5666 comm="qemu-kvm" path="/var/log/libvirt/qemu/rhel5Latest_x86_64_hvm_guest_kvm.log" dev=dm-0 ino=20578746 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:virt_log_t:s0 tclass=file

# rpm -qa | grep selinux-policy
selinux-policy-2.4.6-243.el5
selinux-policy-targeted-2.4.6-243.el5
Comment 20 Alan Pevec 2009-06-09 09:11:10 EDT
workaround to get past this AVC: chcon -R -t qemu_var_run_t /var/log/libvirt/qemu/
Comment 21 Gurhan Ozen 2009-06-09 10:56:35 EDT
(In reply to comment #20)
> workaround to get past this AVC: chcon -R -t qemu_var_run_t
> /var/log/libvirt/qemu/  

We need this to be fixed asap, workarounds aren't acceptable since this does prevent guests from being installed.
Comment 23 Daniel Walsh 2009-06-09 16:01:52 EDT
Fixed in selinux-policy-2.4.6-244.el5
Comment 25 Gurhan Ozen 2009-06-10 17:50:09 EDT
(In reply to comment #23)
> Fixed in selinux-policy-2.4.6-244.el5  

Fails..
type=AVC msg=audit(1244668864.288:28): avc:  denied  { name_bind } for  pid=25366 comm="qemu-kvm" src=5901 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:port_t:s0 tclass=tcp_socket

The error from virt-install:
internal error unable to start guest: inet_listen: bind(ipv4,127.0.0.1,5901): Permission denied
inet_listen: FAILED

Domain installation may not have been
 successful.  If it was, you can restart your domain
 by running 'virsh start rhel5Latest_x86_64_hvm_guest_kvm2'; otherwise, please
 restart your installation.
ERROR    internal error unable to start guest: inet_listen: bind(ipv4,127.0.0.1,5901): Permission denied
inet_listen: FAILED
Traceback (most recent call last):
  File "/usr/sbin/virt-install", line 870, in ?
    main()
  File "/usr/sbin/virt-install", line 768, in main
    start_time, guest.start_install)
  File "/usr/sbin/virt-install", line 823, in do_install
    dom = install_func(conscb, progresscb, wait=(not wait))
  File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 541, in start_install
    return self._do_install(consolecb, meter, removeOld, wait)
  File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 633, in _do_install
    self.domain = self.conn.createLinux(install_xml, 0)
  File "/usr/lib64/python2.4/site-packages/libvirt.py", line 974, in createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self)
libvirtError: internal error unable to start guest: inet_listen: bind(ipv4,127.0.0.1,5901): Permission denied
inet_listen: FAILED

# rpm -qa | grep selinux-policy 
selinux-policy-2.4.6-244.el5
selinux-policy-targeted-2.4.6-244.el5
Comment 28 Daniel Walsh 2009-06-11 14:08:12 EDT
Updating selinux-policy-targeted-2.4.6-245.el5  

Will turn the default booleanin setting for qemu_full_network to true

If you want to retest


setsebool -P qemu_full_network=1

Will allow your qemu processes to use the network.

If you want to run in permissive mode, you can either execute 

#setenforce 0 

This will put the machine in permissive mode

Or edit /etc/selinux/config and set the SELINUX flag to permissive
Comment 31 Gurhan Ozen 2009-06-11 17:38:44 EDT
Ok, the installations are still failing, but this time it doesn't seem to be due to selinux policy. The error I get is:

"ERROR    Could not find usable default libvirt connection."

~/.virtinst/virt-install.log has the same error message with no other messages.

audit.log has no messages.

The same error happens in both selinux enforcing and permissive modes. 

To make this more interesting, when i launch an installation from virt-manager, it does boot the CD however it seems to stall there and not install.

Changing the component.
Comment 32 Alan Pevec 2009-06-11 18:18:33 EDT
Which kvm is it now? If it's the latest kvm-83-74 this could due to move of qemu-kvm to /usr/libexec/
Comment 33 Gurhan Ozen 2009-06-11 18:26:58 EDT
(In reply to comment #32)
> Which kvm is it now? If it's the latest kvm-83-74 this could due to move of
> qemu-kvm to /usr/libexec/  

# rpm -q kvm 
kvm-83-74.el5
Comment 34 Gurhan Ozen 2009-06-11 18:31:07 EDT
(In reply to comment #33)
> (In reply to comment #32)
> > Which kvm is it now? If it's the latest kvm-83-74 this could due to move of
> > qemu-kvm to /usr/libexec/  
> 
> # rpm -q kvm 
> kvm-83-74.el5  

Sorry, I think i sent it too early:

# rpm -ql kvm-83-74.el5 | grep qemu-kvm 
/usr/libexec/qemu-kvm
/usr/share/man/man1/qemu-kvm.1.gz

Is this what you were trying to figure out?
Comment 35 Cole Robinson 2009-06-11 18:33:00 EDT
Alan is probably right: in virtinst, we look for
/usr/bin/{qemu,qemu-kvm,kvm,xenner}, and if any is found, we use qemu:///system
as the default connection. This can be worked around in the interim by using
--connect qemu:///system.

virt-manager will also need a similar fix to try and detect a default
connection, though it is less urgent there.

Let's track these issues in separate bugs though so as not to pollute this one.
Comment 36 Alan Pevec 2009-06-11 18:33:32 EDT
And I'm sorry, forgot to ask which libvirt you have :)
You need latest:
* Wed Jun 10 2009 Daniel Veillard <veillard@redhat.com> - 0.6.3-8.el5 - the libexec patch wasn't working due to a remaining check in /usr/bin - Resolves: rhbz#504046
Comment 37 Alan Pevec 2009-06-11 18:35:45 EDT
(In reply to comment #35)
> This can be worked around in the interim by using
> --connect qemu:///system.

this won't help without the latest libvirt fix
Comment 38 Gurhan Ozen 2009-06-11 18:50:09 EDT
(In reply to comment #37)
> (In reply to comment #35)
> > This can be worked around in the interim by using
> > --connect qemu:///system.
> 
> this won't help without the latest libvirt fix  

Yes, it does have libvirt-0.6.3-8.el5 and yes the workaround works. 
I'll open a different followup bug.
Comment 39 Gurhan Ozen 2009-06-11 18:59:25 EDT
See BZ #505450
Comment 40 Suzanne Yeghiayan 2009-06-12 15:06:29 EDT
This bug represents the selinux-policy-targeted issue resolved in -245 and is included in that advisory.
Please open new bugs for additional issues.
Restored component from python-virtinst back to selinux-policy-targeted.
Comment 42 errata-xmlrpc 2009-09-02 03:58:05 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-1242.html

Note You need to log in before you can comment on or make changes to this bug.