Description of problem: trying out libguestfs for the first time. Trying to debug a virtual machine that won't boot. # virt-ls /dev/rootvg/xenfdev32 /boot new guestfs handle 0x14b5e760 looking for supermin appliance in /usr/lib64/guestfs launch: external command failed: PATH='/usr/lib64/guestfs':$PATH libguestfs-supermin-helper '/usr/lib64/guestfs' /tmp/libguestfsGF6gd6/kernel /tmp/libguestfsGF6gd6/initrd at /usr/bin/virt-ls line 173. closing guestfs handle 0x14b5e760 (state 0) # guestfish -i /dev/rootvg/xenfdev32 new guestfs handle 0x210a550 new guestfs handle 0x1ac6db40 looking for supermin appliance in /usr/lib64/guestfs launch: external command failed: PATH='/usr/lib64/guestfs':$PATH libguestfs-supermin-helper '/usr/lib64/guestfs' /tmp/libguestfsOtQT18/kernel /tmp/libguestfsOtQT18/initrd at /usr/bin/virt-inspector line 217. closing guestfs handle 0x1ac6db40 (state 0) closing guestfs handle 0x210a550 (state 0) Version-Release number of selected component (if applicable): libguestfs-1.0.76-1.el5.4
Please can you run: libguestfs-test-tool and attach the complete output here.
And also check that multilib hasn't messed up your installation: rpm -V libguestfs
# libguestfs-test-tool ===== Test starts here ===== LIBGUESTFS_DEBUG=1 new guestfs handle 0x10803150 library version: 1.0.76 guestfs_get_append: (null) guestfs_get_autosync: 0 guestfs_get_memsize: 500 guestfs_get_path: /usr/lib64/guestfs guestfs_get_qemu: /usr/bin/qemu-system-x86_64 guestfs_get_verbose: 1 Launching appliance, timeout set to 120 seconds. looking for supermin appliance in /usr/lib64/guestfs libguestfs: error: external command failed: PATH='/usr/lib64/guestfs':$PATH libguestfs-supermin-helper '/usr/lib64/guestfs' /tmp/libguestfsNxWETA/kernel /tmp/libguestfsNxWETA/initrd libguestfs-test-tool: failed to launch appliance closing guestfs handle 0x10803150 (state 0) # ls /usr/lib64/guestfs/ initramfs.epel-5.x86_64.supermin.hostfiles kmod.whitelist initramfs.epel-5.x86_64.supermin.img No multi-lib, rpm -V checks out fine.
Orion, I wonder if you have limited disk space on /tmp, although it _should_ print an error message. In any case is it possible you could do this (you don't need to be root): libguestfs-supermin-helper /usr/lib64/guestfs /tmp/kernel /tmp/initrd echo $? That should work without any error, and the status code should be 0. After running that you should get two files which will be similar (but not exactly the same) to this: ls -lh /tmp/kernel /tmp/initrd -rw-rw-r-- 1 rjones rjones 69M 2009-11-24 16:51 /tmp/initrd lrwxrwxrwx 1 rjones rjones 46 2009-11-24 16:51 /tmp/kernel -> /boot/vmlinuz-2.6.32-0.33.rc5.git1.fc13.x86_64
FYI, this is from a RHEL 5.4 64 bit machine with the same version of libguestfs as you reported: $ libguestfs-supermin-helper /usr/lib64/guestfs /tmp/kernel /tmp/initrd $ echo $? 0 $ ls -lh /tmp/kernel /tmp/initrd -rw-r--r-- 1 rjones rjones 96M Nov 24 17:12 /tmp/initrd lrwxrwxrwx 1 rjones rjones 28 Nov 24 17:11 /tmp/kernel -> /boot/vmlinuz-2.6.18-164.el5 $ rpm -q libguestfs libguestfs-1.0.76-1.el5.4 You wouldn't expect the /tmp/kernel and /tmp/initrd files to be exactly the same, but they should be broadly similar.
[root@hammer ~]# libguestfs-supermin-helper /usr/lib64/guestfs /tmp/kernel /tmp/initrd [root@hammer ~]# ls -l /tmp total 0 drwx------ 2 root root 60 Nov 24 10:22 ssh-wyWkq20901 drwx------ 2 root root 60 Nov 24 09:25 ssh-ynlri20346
[root@hammer ~]# libguestfs-supermin-helper /usr/lib64/guestfs /tmp/kernel /tmp/initrd [root@hammer ~]# echo $? 1 [root@hammer ~]# df /tmp Filesystem Size Used Avail Use% Mounted on tmpfs 1.8G 0 1.8G 0% /tmp
I think I might have it: 21088 stat("/boot/vmlinuz-*.x86_64*", 0xbf0e578) = -1 ENOENT (No such file or directory) 21088 lstat("/boot/vmlinuz-*.x86_64*", 0xbf0e578) = -1 ENOENT (No such file or directory) ls /boot/vmlinuz* /boot/vmlinuz-2.6.18-164.2.1.el5xen /boot/vmlinuz-2.6.18-164.el5virttest17xen /boot/vmlinuz-2.6.18-164.6.1.el5xen I need some guest kernels installed?
Ah ha ... you're running this inside Xen, right? This is actually a proper bug in the script in that case, or perhaps more accurately it's a missing dependency. It's looking for a bootable kernel (one that can be booted in qemu, which Xen kernels can't be). And so that script expects to find a non-Xen kernel in /boot which it can use. So it does: kernels=$(ls -1vr /boot/vmlinuz-*.$arch* 2>/dev/null | grep -v xen; ls -1vr /boot/vmlinuz-* 2>/dev/null | grep -v xen) If you install a non-Xen kernel in your Xen guest, it should work, although libguestfs will likely be quite slow because it won't be able to use hardware virtualization[1]. (BTW, /usr/bin/libguestfs-supermin-helper is just a shell script) [1] http://libguestfs.org/FAQ.html#slow
This code is supposed to print an error to stderr. I wonder why it didn't do that? if [ -z "$modpath" ]; then echo "$0: failed to find a suitable kernel" >&2 exit 1 fi
Never gets there because kernels is empty: # sh -x libguestfs-supermin-helper /usr/lib64/guestfs /tmp/kernel /tmp/initrd + unset CDPATH + set -e ++ cd /usr/lib64/guestfs ++ pwd + sourcedir=/usr/lib64/guestfs + kernel=/tmp/kernel + initrd=/tmp/initrd ++ echo x86_64 ++ sed 's/^i.86$/i?86/' + arch=x86_64 ++ ls -1vr '/boot/vmlinuz-*.x86_64*' ++ grep -v xen + kernels=
(In reply to comment #11) > Never gets there because kernels is empty: Yup, that'll be a problem :-) I pushed this patch upstream: https://www.redhat.com/archives/libguestfs/2009-November/msg00225.html and the same to febootstrap: http://git.annexia.org/?p=febootstrap.git;a=commitdiff;h=a1c6fbdf00f249ddaadc733afaa5ac6e8fbcc30e I think the best thing here is if you could install the kernel package and try again. *Note* that I don't mean you have to boot your Xen guest with a fullvirt kernel (which probably wouldn't work). It just needs to use that kernel in order to boot the libguestfs appliance using qemu.
Installed "kernel" and getting much farther: # guestfish -i /dev/rootvg/xenfdev32 Could not open '/dev/kqemu' - QEMU acceleration layer not activated: No such file or directory get_e2uuid: get_e2uuid: tune2fs -l: /sbin/tune2fs: Filesystem has unsupported feature(s) while trying to open /dev/hda1 Couldn't find valid filesystem superblock. at /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/Sys/Guestfs/Lib.pm line 693 I'm trying to probe the filesystem types for the filesystems in /dev/rootvg/xenfdev32 because I suspect that anaconda in F-13 is forcing /boot to be ext4 even though I specify ext3. I was hoping that libguestfs could help determine that, but I really have no idea how to use it. Thanks for all the help.
(In reply to comment #13) > Installed "kernel" and getting much farther: > > # guestfish -i /dev/rootvg/xenfdev32 > Could not open '/dev/kqemu' - QEMU acceleration layer not activated: No such > file or directory > get_e2uuid: get_e2uuid: tune2fs -l: /sbin/tune2fs: Filesystem has unsupported > feature(s) while trying to open /dev/hda1 > Couldn't find valid filesystem superblock. at > /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/Sys/Guestfs/Lib.pm > line 693 > > I'm trying to probe the filesystem types for the filesystems in > /dev/rootvg/xenfdev32 because I suspect that anaconda in F-13 is forcing /boot > to be ext4 even though I specify ext3. I was hoping that libguestfs could help > determine that, but I really have no idea how to use it. This is a separate bug in virt-inspector (which is a sub- package of libguestfs). Can you open another bug about it? For testing this, I'd start simple: guestfish -a /dev/rootvg/xenfdev32 Type "run" at the prompt. Then try out some guestfish commands, eg: list-devices list-partitions lvs file /dev/hda1 vfs_type /dev/hda1 mount /dev/hda1 / There's much more about guestfish in the man page: http://libguestfs.org/guestfish.1.html and in the recipes page: http://libguestfs.org/recipes.html > Thanks for all the help. And thanks for your persistence!
(In reply to comment #14) > guestfish -a /dev/rootvg/xenfdev32 If you don't want to modify the image, do: guestfish --ro -a /dev/rootvg/xenfdev32
I think now I need access to the "e4" versions of the "e2" utilities in guestfish. I think I can hack up a version for that.
(In reply to comment #16) > I think now I need access to the "e4" versions of the "e2" utilities in > guestfish. I think I can hack up a version for that. Right ... RHEL 5.4 has the non-standard ext4dev type and e4fsprogs. So that'll be the second area where you are exploring new territory :-)
Updating summary to more accurately reflect the bug.
This is a Fedora EPEL packaging bug. Changing Product field to reflect that.
I talked to Dan Berrange and he things the best way to solve this is to add: Requires: kernel into the spec file. However this will have the unfortunate side-effect that users will have a non-Xen kernel added to their grub.conf (if they are using this in a Xen guest -- it won't affect anyone else).
On second thoughts, kernel is a virtual provides of kernel-xen, so we can't just do that.
It at least isn't installed as the default: default=1 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz # hiddenmenu title CentOS (2.6.18-164.6.1.el5) root (hd0,0) kernel /vmlinuz-2.6.18-164.6.1.el5 ro root=/dev/rootvg/root crashkernel=128M@16M initrd /initrd-2.6.18-164.6.1.el5.img title CentOS (2.6.18-164.6.1.el5xen) root (hd0,0) kernel /xen.gz-2.6.18-164.6.1.el5 module /vmlinuz-2.6.18-164.6.1.el5xen ro root=/dev/rootvg/root crashkernel=128M@16M module /initrd-2.6.18-164.6.1.el5xen.img
Moving Product back to Virtualization. This isn't just a simple packaging bug ...
BTW, for recovering broken VMs you might want to look at virt-rescue which gives you a shell: http://libguestfs.org/virt-rescue.1.html
I can confirm that for fully updated CentOS 5 host installing kernel package on this host is sufficient to make libguestfs work.