Description of problem: Not sure what's going on but all kvm guests are failing with the same reason on rhts with 20090604.nightly tree. I have manually run an installation with --debug option and here is what I got: # virt-install --name x86_64_hvm_guest_3 --cdrom /var/lib/libvirt/images/x86_64_hvm_guest.iso --nonsparse --hvm --file /var/lib/libvirt/images/x86_64_hvm_guest_3 -s 10 --extra-args ks=http://lab.rhts.bos.redhat.com/kickstarts/hosts/guest-81-16.rhts.bos.redhat.com/ks.cfg --prompt --accelerate --os-variant=virtio26 --noreboot --debug Thu, 04 Jun 2009 16:57:17 DEBUG Using libvirt URI 'qemu:///system' Thu, 04 Jun 2009 16:57:17 DEBUG Requesting virt method 'hvm' Thu, 04 Jun 2009 16:57:17 DEBUG Received virt method 'hvm' Thu, 04 Jun 2009 16:57:17 DEBUG Hypervisor name is 'kvm' How much RAM should be allocated (in megabytes)? 512 Thu, 04 Jun 2009 16:57:23 DEBUG DISPLAY is set: graphics defaulting to VNC. Thu, 04 Jun 2009 16:57:23 DEBUG Setting os type to 'linux' for variant 'virtio26' Thu, 04 Jun 2009 16:57:23 DEBUG DistroInstaller location is a local file/path: /var/lib/libvirt/images/x86_64_hvm_guest.iso Thu, 04 Jun 2009 16:57:23 DEBUG Setting size for existing storage to '0.00880241394043' Thu, 04 Jun 2009 16:57:23 DEBUG Detected storage as type 'file' Starting install... Thu, 04 Jun 2009 16:57:24 DEBUG Setting size for existing storage to '0.00880241394043' Thu, 04 Jun 2009 16:57:24 DEBUG Detected storage as type 'file' Creating storage file... | 10 GB 01:08 Thu, 04 Jun 2009 16:58:33 DEBUG Creating guest from: <domain type='kvm'> <name>x86_64_hvm_guest_3</name> <currentMemory>524288</currentMemory> <memory>524288</memory> <uuid>6b25e622-84e8-679e-fe74-7a4ce3e69239</uuid> <os> <type arch='x86_64'>hvm</type> <boot dev='cdrom'/> </os> <features> <acpi/><apic/><pae/> </features> <clock offset="utc"/> <on_poweroff>destroy</on_poweroff> <on_reboot>destroy</on_reboot> <on_crash>destroy</on_crash> <vcpu>1</vcpu> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <console type='pty'/> <disk type='file' device='disk'> <source file='/var/lib/libvirt/images/x86_64_hvm_guest_3'/> <target dev='vda' bus='virtio'/> </disk> <disk type='file' device='cdrom'> <source file='/var/lib/libvirt/images/x86_64_hvm_guest.iso'/> <target dev='hdc' bus='ide'/> <readonly/> </disk> <interface type='network'> <source network='default'/> <mac address='54:52:00:63:d9:d9'/> <model type='virtio'/> </interface> <input type='mouse' bus='ps2'/> <graphics type='vnc' port='-1' keymap='en-us'/> </devices> </domain> internal error Domain x86_64_hvm_guest_3 didn't show up Domain installation may not have been successful. If it was, you can restart your domain by running 'virsh start x86_64_hvm_guest_3'; otherwise, please restart your installation. Thu, 04 Jun 2009 16:58:43 ERROR internal error Domain x86_64_hvm_guest_3 didn't show up Traceback (most recent call last): File "/usr/sbin/virt-install", line 865, in ? main() File "/usr/sbin/virt-install", line 763, in main start_time, guest.start_install) File "/usr/sbin/virt-install", line 818, in do_install dom = install_func(conscb, progresscb, wait=(not wait)) File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 541, in start_install return self._do_install(consolecb, meter, removeOld, wait) File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 633, in _do_install self.domain = self.conn.createLinux(install_xml, 0) File "/usr/lib64/python2.4/site-packages/libvirt.py", line 974, in createLinux if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self) libvirtError: internal error Domain x86_64_hvm_guest_3 didn't show up This is happening for both x86_64 and i386 guests. We have also got the following AVC messages popping up: time->Thu Jun 4 13:01:34 2009 type=SYSCALL msg=audit(1244134894.840:15): arch=c000003e syscall=2 success=no exit=-13 a0=7fff23ef2e11 a1=42 a2=180 a3=0 items=0 ppid=1 pid=5229 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/bin/qemu-kvm" subj=system_u:system_r:qemu_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(1244134894.840:15): avc: denied { write } for pid=5229 comm="qemu-kvm" name="qemu" dev=dm-0 ino=10715432 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:virt_var_run_t:s0 tclass=dir Just as FYI, I have tried the same tree with a -152 kernel but got the same failure as well. Installations were happening successfully for me, as recent as 20090602.nightly tree. Version-Release number of selected component (if applicable): [root@dell-pe2900-03 ~]# uname -a Linux dell-pe2900-03.rhts.bos.redhat.com 2.6.18-151.el5 #1 SMP Wed May 27 16:14:57 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux # rpm -qa | egrep "kvm|libvirt" kmod-kvm-83-59.el5 libvirt-0.6.3-4.el5 libvirt-0.6.3-4.el5 kvm-83-59.el5 libvirt-python-0.6.3-4.el5 rh-tests-virt-libvirt-migrate-2.0-6 etherboot-zroms-kvm-5.4.4-10.el5 How reproducible: Very Steps to Reproduce: 1. Install 20090604.nightly tree and try to install kvm guests. 2. 3. Actual results: Fails. Expected results: Should install. Additional info:
This seems to have started happening because of the change in selinux policy between the 20090602 and 20090604 trees. Gurhan downgraded the selinux policy to the policy from 20090602, and things started working again. Moving over to selinux-policy to have the selinux people look at it. Chris Lalancette
Fixed in selinux-policy-2.4.6-242.el5
(In reply to comment #5) > Fixed in selinux-policy-2.4.6-242.el5 Fails.... [root@tyan-gt24-05 ~]# uname -a ; rpm -qa | grep selinux-policy Linux tyan-gt24-05.rhts.bos.redhat.com 2.6.18-152.el5 #1 SMP Wed Jun 3 18:57:00 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux selinux-policy-2.4.6-242.el5 selinux-policy-targeted-2.4.6-242.el5 [root@tyan-gt24-05 ~]# /sbin/ausearch -sv no -m AVC -m USER_AVC -m SELINUX_ERR -ts 6/5/2009 14:2:15 ---- time->Fri Jun 5 14:04:09 2009 type=SYSCALL msg=audit(1244225049.921:9): arch=c000003e syscall=2 success=no exit=-13 a0=541cab a1=202 a2=0 a3=0 items=0 ppid=1 pid=4906 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/bin/qemu-kvm" subj=system_u:system_r:qemu_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(1244225049.921:9): avc: denied { read write } for pid=4906 comm="qemu-kvm" name="ksm" dev=tmpfs ino=6132 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:device_t:s0 tclass=chr_file ---- time->Fri Jun 5 14:10:10 2009 type=SYSCALL msg=audit(1244225410.032:14): arch=c000003e syscall=2 success=no exit=-13 a0=541cab a1=202 a2=0 a3=0 items=0 ppid=1 pid=6573 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/bin/qemu-kvm" subj=system_u:system_r:qemu_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(1244225410.032:14): avc: denied { read write } for pid=6573 comm="qemu-kvm" name="ksm" dev=tmpfs ino=6132 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:device_t:s0 tclass=chr_file [root@tyan-gt24-05 ~]#
What is a /dev/ksm? Never seen one before.
More importantly what is the security properties of this device? Should eveyone be allowed to read/write it? All qemu processes?
/dev/ksm is a character device used (via ioctl's, I believe) to control the ksm module. The ksm module is responsible for doing page sharing among processes. To be honest, I'm not sure what the security properties should be; from my understanding, there should be one daemon in the system that talks to the /dev/ksm device and controls how often the scanning is done, etc. But I'm not intimately involved with it, so I could be wrong. Izik, can you answer Comment #10? Chris Lalancette
/dev/ksm should be accessed only by the kvm-user or the root user. every qemu process should have access to /dev/ksm, but not normal users. Thanks. (letting normal users control ksm, may lead to situation of allocating a big amount of unswappable memory)
(In reply to comment #12) > /dev/ksm should be accessed only by the kvm-user or the root user. > > every qemu process should have access to /dev/ksm, but not normal users. Hm, I'm a bit confused though. Wasn't the idea of ksm is that it is transparent to processes? That is, why should qemu touch /dev/ksm? Shouldn't qemu just do the normal things that it does, and ksm will transparently share the pages behind it's back? Or am I missing something with how ksm works (certainly possible)? Chris Lalancette
ksm is transparent to the process, but ksm need what process to scan... so all qemu is doing is just open /dev/ksm and say "scan me"... in the mainline version we can do it with ksmctl -pid.... but qemu need access to /dev/ksm in order to tell it to scan the memory.
Ah, OK, gotcha. Thanks for clarifying. So, it looks like this is a RHEL specific change to the policy. Dan Walsh, does this clear things up for you? Can you fix up the policy for RHEL-5 now? Chris Lalancette
I need to define a new type for /dev/ksm but ut from an SELinux point of view qemu processes should ot be trusted. Think of a hacker figuring out a way to compromize your guest OS (windows) then we have a bug in qemu/kvm that allows him to break out of the qemu process and take it over. What kind of attack can he mount on the host system through the /dev/ksm? What kind of attack can he run against other qemu processes running guest os?
he can make hugh amount of unswappable memory allocated. he can run the whole system into condion of out of memory. the only thing that /dev/ksm is need is: the kvm user and the root user will have access to /dev/ksm in our virtualziation soultion we have kvm user right?
Ok as long as he can only crash the machine, he can not subvirt other processes or take over the machine. Fixed in selinux-policy-2.4.6-243.el5
(In reply to comment #18) > Ok as long as he can only crash the machine, he can not subvirt other > processes or take over the machine. > > > > Fixed in selinux-policy-2.4.6-243.el5 Fails.. type=AVC msg=audit(1244492216.620:8): avc: denied { append } for pid=5666 comm="qemu-kvm" path="/var/log/libvirt/qemu/rhel5Latest_x86_64_hvm_guest_kvm.log" dev=dm-0 ino=20578746 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:virt_log_t:s0 tclass=file type=AVC msg=audit(1244492216.620:8): avc: denied { append } for pid=5666 comm="qemu-kvm" path="/var/log/libvirt/qemu/rhel5Latest_x86_64_hvm_guest_kvm.log" dev=dm-0 ino=20578746 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:virt_log_t:s0 tclass=file # rpm -qa | grep selinux-policy selinux-policy-2.4.6-243.el5 selinux-policy-targeted-2.4.6-243.el5
workaround to get past this AVC: chcon -R -t qemu_var_run_t /var/log/libvirt/qemu/
(In reply to comment #20) > workaround to get past this AVC: chcon -R -t qemu_var_run_t > /var/log/libvirt/qemu/ We need this to be fixed asap, workarounds aren't acceptable since this does prevent guests from being installed.
Fixed in selinux-policy-2.4.6-244.el5
(In reply to comment #23) > Fixed in selinux-policy-2.4.6-244.el5 Fails.. type=AVC msg=audit(1244668864.288:28): avc: denied { name_bind } for pid=25366 comm="qemu-kvm" src=5901 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 tcontext=system_u:object_r:port_t:s0 tclass=tcp_socket The error from virt-install: internal error unable to start guest: inet_listen: bind(ipv4,127.0.0.1,5901): Permission denied inet_listen: FAILED Domain installation may not have been successful. If it was, you can restart your domain by running 'virsh start rhel5Latest_x86_64_hvm_guest_kvm2'; otherwise, please restart your installation. ERROR internal error unable to start guest: inet_listen: bind(ipv4,127.0.0.1,5901): Permission denied inet_listen: FAILED Traceback (most recent call last): File "/usr/sbin/virt-install", line 870, in ? main() File "/usr/sbin/virt-install", line 768, in main start_time, guest.start_install) File "/usr/sbin/virt-install", line 823, in do_install dom = install_func(conscb, progresscb, wait=(not wait)) File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 541, in start_install return self._do_install(consolecb, meter, removeOld, wait) File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 633, in _do_install self.domain = self.conn.createLinux(install_xml, 0) File "/usr/lib64/python2.4/site-packages/libvirt.py", line 974, in createLinux if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self) libvirtError: internal error unable to start guest: inet_listen: bind(ipv4,127.0.0.1,5901): Permission denied inet_listen: FAILED # rpm -qa | grep selinux-policy selinux-policy-2.4.6-244.el5 selinux-policy-targeted-2.4.6-244.el5
Updating selinux-policy-targeted-2.4.6-245.el5 Will turn the default booleanin setting for qemu_full_network to true If you want to retest setsebool -P qemu_full_network=1 Will allow your qemu processes to use the network. If you want to run in permissive mode, you can either execute #setenforce 0 This will put the machine in permissive mode Or edit /etc/selinux/config and set the SELINUX flag to permissive
Ok, the installations are still failing, but this time it doesn't seem to be due to selinux policy. The error I get is: "ERROR Could not find usable default libvirt connection." ~/.virtinst/virt-install.log has the same error message with no other messages. audit.log has no messages. The same error happens in both selinux enforcing and permissive modes. To make this more interesting, when i launch an installation from virt-manager, it does boot the CD however it seems to stall there and not install. Changing the component.
Which kvm is it now? If it's the latest kvm-83-74 this could due to move of qemu-kvm to /usr/libexec/
(In reply to comment #32) > Which kvm is it now? If it's the latest kvm-83-74 this could due to move of > qemu-kvm to /usr/libexec/ # rpm -q kvm kvm-83-74.el5
(In reply to comment #33) > (In reply to comment #32) > > Which kvm is it now? If it's the latest kvm-83-74 this could due to move of > > qemu-kvm to /usr/libexec/ > > # rpm -q kvm > kvm-83-74.el5 Sorry, I think i sent it too early: # rpm -ql kvm-83-74.el5 | grep qemu-kvm /usr/libexec/qemu-kvm /usr/share/man/man1/qemu-kvm.1.gz Is this what you were trying to figure out?
Alan is probably right: in virtinst, we look for /usr/bin/{qemu,qemu-kvm,kvm,xenner}, and if any is found, we use qemu:///system as the default connection. This can be worked around in the interim by using --connect qemu:///system. virt-manager will also need a similar fix to try and detect a default connection, though it is less urgent there. Let's track these issues in separate bugs though so as not to pollute this one.
And I'm sorry, forgot to ask which libvirt you have :) You need latest: * Wed Jun 10 2009 Daniel Veillard <veillard> - 0.6.3-8.el5 - the libexec patch wasn't working due to a remaining check in /usr/bin - Resolves: rhbz#504046
(In reply to comment #35) > This can be worked around in the interim by using > --connect qemu:///system. this won't help without the latest libvirt fix
(In reply to comment #37) > (In reply to comment #35) > > This can be worked around in the interim by using > > --connect qemu:///system. > > this won't help without the latest libvirt fix Yes, it does have libvirt-0.6.3-8.el5 and yes the workaround works. I'll open a different followup bug.
See BZ #505450
This bug represents the selinux-policy-targeted issue resolved in -245 and is included in that advisory. Please open new bugs for additional issues. Restored component from python-virtinst back to selinux-policy-targeted.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2009-1242.html