Description of problem: I've finally managed to reproduce a problem using virt-install to install fully-virtualized guests. You must have SELinux enabled to reproduce this. Here's the steps. 1) Mount remote server to serve out the boot.iso (e.g. mount server:/path/to/RHEL/ /mnt/tmp 2) virt-install -n rhel4fvtest -r 1024 -f /var/lib/xen/images/rhel4fvtest.dsk -s 6 --nonsparse -c /mnt/tmp/images/boot.iso -v --vnc 3) It will say "Starting install...", and after some time (because of the --nonsparse), it will spew a traceback like the following: libvir: Xen Daemon error : POST operation failed: (xend.err 'destroyDevice() takes exactly 3 arguments (2 given)') Failed to get devices for domain rhel4fvtest Traceback (most recent call last): File "/usr/sbin/virt-install", line 447, in ? main() File "/usr/sbin/virt-install", line 411, in main dom = guest.start_install(conscb) File "/usr/lib/python2.4/site-packages/virtinst/XenGuest.py", line 367, in start_install self.domain = self.conn.createLinux(cxml, 0) File "/usr/lib64/python2.4/site-packages/libvirt.py", line 249, in createLinux if ret is None:raise libvirtError('virDomainCreateLinux() failed') libvirt.libvirtError: virDomainCreateLinux() failed The interesting thing here is that although virt-install failed, it still *created* the fully virt domain. However, it never unpaused the domain. Interestingly, you can actually go ahead and unpause the domain (xm unpause <dom>), but because virt-install failed it will never write out a valid configuration. I'm not sure what the solution is (since I understand that we don't necessarily want to run around destroying domains), but this behavior is confusing to end users.
I can't reproduce this: running enforcing, with an iso on NFS via autofs, it runs fine. What selinux avc denials are you getting? Thanks.
Here are the denials I am getting: type=ANOM_PROMISCUOUS msg=audit(1170094440.915:25): dev=vif1.0 prom=256 old_prom=0 auid=4294967295 type=SYSCALL msg=audit(1170094440.915:25): arch=c000003e syscall=16 success=yes exit=0 a0=3 a1=89a2 a2=7fffa5ef2a30 a3=21000 items=0 ppid=3597 pid=3690 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) comm="brctl" exe="/usr/sbin/brctl" subj=system_u:system_r:udev_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(1170094441.095:26): avc: denied { read write } for pid=3742 comm="ifconfig" name="rhel4fvtest2.dsk" dev=sda3 ino=9493705 scontext=system_u:system_r:ifconfig_t:s0 tcontext=root:object_r:xen_image_t:s0 tclass=file type=AVC msg=audit(1170094441.095:26): avc: denied { read } for pid=3742 comm="ifconfig" name="boot.iso" dev=0:14 ino=28361949 scontext=system_u:system_r:ifconfig_t:s0 tcontext=system_u:object_r:nfs_t:s0 tclass=file type=SYSCALL msg=audit(1170094441.095:26): arch=c000003e syscall=59 success=yes exit=0 a0=1a68d070 a1=1a68bb80 a2=1a68d440 a3=0 items=0 ppid=3696 pid=3742 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) comm="ifconfig" exe="/sbin/ifconfig" subj=system_u:system_r:ifconfig_t:s0 key=(null) type=AVC_PATH msg=audit(1170094441.095:26): path="/mnt/bigpapi/released/RHEL-4/U4/AS/x86_64/tree/images/boot.iso" type=AVC_PATH msg=audit(1170094441.095:26): path="/var/lib/xen/images/rhel4fvtest2.dsk" type=ANOM_PROMISCUOUS msg=audit(1170094441.103:27): dev=tap0 prom=256 old_prom=0 auid=4294967295 type=SYSCALL msg=audit(1170094441.103:27): arch=c000003e syscall=16 success=yes exit=0 a0=5 a1=89a2 a2=7fff9ac4b850 a3=21000 items=0 ppid=3696 pid=3747 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) comm="brctl" exe="/usr/sbin/brctl" subj=system_u:system_r:xend_t:s0 key=(null) Some of these seem very odd to me, so to verify, I tried the exact same virt-install command as posted above, but I copied the boot.iso to /var/lib/xen/images, and pointed the -c there. The install started successfully. Chris Lalancette
As another data point: I found that mounting an NFS share read-only allows the install to start successfully. It's only when mounting the very same NFS share read-write that I run into the failure. Chris Lalancette
Can you try mounting the nfs share with the context=system_u:object_r:xen_image_t option.
Karl, I got the exact same failure when mounting the share with the above context line. I used "mount -o context=system_u:object_r:xen_image_t,bg,intr,hard server:/vol/redhat /mnt/point"; that is, I got the same error on the console (about destroyDevice), and I seemed to have the same errors in /var/log/audit/audit.log Chris Lalancette
Can you try a relabel, please? I cannot reproduce this at all. Also, is the nfs iso dir exported with no_root_squash or any other non-standard options?
Hi This should help reproduce, it is the original from a thread that I logged on the xen-user list yesterday. http://lists.xensource.com/archives/html/xen-users/2007-01/msg01305.html So you know I am NOT running selinux and still rec. the same destroy device error. From my investigation it would appear that a 3rd input parameter named "force" was added to the destroyDevice func. in DevController.py recently. Hope this helps G ---x--- Hello all The chaps at XenServer directed me to this mailing list and to my surprise found 2 very recent hits on the error I am receiving. Is it a bug perhaps? Jordi logged this: http://lists.xensource.com/archives/html/xen-users/2007-01/msg01270.html and Shawn logged this one: http://lists.xensource.com/archives/html/xen-users/2007-01/msg00892.html Here is more detail on my problem, please let me know if you have found anything to date. On following the "Concise instructions for installing a CentOS VM on Xen" http://mark.foster.cc/wiki/index.php/Centos-4_on_Xen using a FC6 host and Oracle EL4 guest I am able to create the LVM partition, bootstrap etc. without any errors however when I attempt to create the VM a destroyDevice error is given. So you know I have attempted to boot with the FC6 xen kernel first (2.6.19-1.2895.fc6xen) and then installed the Red Hat xen kernel vmlinuz-2.6.16.33-xen_3.0.4.1 by chroot and then rpm -ivh kernel-xen... I wonder if the kernel is correct!?! # xm create -c el4-1 gives... "Using config file "/etc/xen/el4-1". Error: destroyDevice() takes exactly 3 arguments (2 given)" "xm list" while "xm create" is running -------------------------------------- # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 1892 2 r----- 2100.2 el4-1 24 96 1 --p--- 0.0 /etc/xen/el4-1 config file: ------------------ [root@localhost mnt]# cat /etc/xen/el4-1 kernel = "/root/boot/vmlinuz-2.6.16.33-xen_3.0.4.1" # also used FC6: kernel = "/boot/vmlinuz-2.6.19-1.2895.fc6xen" memory = 96 name = "el4-1" disk = [ 'phy:/dev/VolGroup01/LogVol01,sdb1,w','phy:/dev/VolGroup01/LogVol02,sdb2,w' ] dhcp="dhcp" root = "/dev/sdb1 ro" guest fstab: ------------ [root@localhost mnt]# cat fstab /dev/sdb1 / ext3 defaults 1 1 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 proc /proc proc defaults 0 0 /dev/sdb2 swap swap defaults 0 0
Let me clarify *yet again* WRT to comment #8, because people seem to have a hard time understanding this 'Error: destroyDevice() takes exactly 3 arguments (2 given)" Has absolutely *nothing* todo with the causes of device failure. Yes, there is a bug in the usage of destroyDevice - this bug, however, is merely in the cleanup path *after* hotplug has already failed. ie, the guest was booted. hotplug of the network/disk device failed, and thus XenD went to tear down the domain - only at this last stage (after the real failure) does it hit the 'destroyDevice' bug. So, please ignore comment #8 in this ticket
OK, so it seems like the SELinux denials are just a red herring. I tried again with a read-write NFS share with SELinux in permissive mode, and it still failed. Then I went back and remounted the NFS share in read-only mode, and I was able to successfully start the install. Finally, I turned SELinux back to Enforcing, and tried again with the read-only share, and I was able to successfully start the install again. So I'm changing the summary here; while the denials are odd (and may represent weird things that QEMU is doing), they aren't a problem with SELinux or the policy (I don't think). Chris Lalancette
OK, more information from Dan Berrange: <danpb> clalance: aahhhhh, easy test case <danpb> losetup /dev/loop0 /mnt/tmp/images/boot.iso <danpb> that will work for a readonly NFS mount, fail for a read-write one <clalance> danpb: Oh, I see. That is a good test case. <clalance> I wonder if that is expected behavior. <danpb> clalance: losetup is trying to open the file RW always <danpb> now the actual ISO on NFS is read only <danpb> so when NFS is mounted RW, the open call gets -1 EACCESS <danpb> but when NFS is mounted RO, the open call gets -1 EROFS <danpb> and in the EROFS case, losetup will automatically fallback to trying to create a read only looop device <danpb> but it doesn't do this fallback for the EACCESS case <danpb> clalance: ultimately this is a bug in Xen hotplug scripts <danpb> because we told Xen that we wanted a readonly CDROM device <danpb> so losetup should be given the -r flag to force it to do a read only loopdevice regardless Chris Lalancette
Created attachment 147118 [details] Do a readonly loop device setup for readonly disks The attached patch changes the block hotplug script so that if the virtual disk is exported to the guest in read only mode, we explicitly ask for a readonly loop device. eg, using losetup -r. This should solve problems in exporting a readonly file, from a read-write NFS mount
Hi Daniel Sorry about comment 8 and thanks for the explanation. Take care G
Merged upstream at: http://xenbits2.xensource.com/xen-unstable.hg?rev/0c17f16f9f03 But the patch was later reverted in http://xenbits2.xensource.com/xen-unstable.hg?rev/67faf6c59bfa It was reverted because older versions of losetup do not support -r. Since we don't care about old version of losetup, so recommend we keep this patch in our tree. There isn't any other easy way to solve this problem without the -r option that I can think of.
If you look at the file context of the destination file you will see that it is user_home_t, You probably created the image file in your homedir and then used the mv command to place it in /var/llib/xen/images. The mv command maintains the MAC label so you end up with a file labeled user_home_t instead of xend_lib_t. restorecon /var/lib/xen/images/boot.iso will fix the labeleing. If you installed setroubleshoot, it would have translated this error message for you. If you use install instead of mv it will do the right thing, or use mv followed by restorecon. Think of the MAC privs in this case the same way you would of the ownership. If you moved a file from your home dir to this directory it would still be owned by you, and you would have to do a chown.
Reopening this since this is not an selinux problem .... that was just one update from IBM.
The patch I attached in comment #13 and also clarified in comment #15 is the solution we need to track for this ticket. SELinux is just a diversion - its nothing todo with the core problem here - the core problem is that loop devices must be setup readonly if the image ISO itself is readonly. Hence my patch to call losetup with the -r flag.
(In reply to comment #19) > The patch I attached in comment #13 and also clarified in comment #15 is the > solution we need to track for this ticket. SELinux is just a diversion - its > nothing todo with the core problem here - the core problem is that loop devices > must be setup readonly if the image ISO itself is readonly. Hence my patch to > call losetup with the -r flag. > Daniel, Will this patch (comment #13) make it in RHE5 U1?
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux maintenance release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Update release for currently deployed products. This request is not yet committed for inclusion in an Update release.
Built in xen-3.0.3-31.el5 into dist-5E-qu-candidate
I just tested this out with the latest 5.1 Beta packages, and all seems well; with the same NFS mount that was causing problems before, I am now able to start installs properly. Chris Lalancette
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHEA-2007-0635.html