Red Hat Bugzilla – Full Text Bug Listing
|Summary:||guestfish not able to mount freebsd ufs2 partitions automatically|
|Product:||[Community] Virtualization Tools||Reporter:||Praveen Arimbrathodiyil <parimbra>|
|Component:||libguestfs||Assignee:||Richard W.M. Jones <rjones>|
|Status:||ASSIGNED ---||QA Contact:|
|Version:||unspecified||CC:||eblake, kzak, mbooth, qzhang, rjones, tools-bugs, virt-maint|
|Fixed In Version:||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
Description Praveen Arimbrathodiyil 2009-11-26 09:18:40 EST
Description of problem: mount command inside guestfish is failing to mount ufs2 partitions from qemu image of debian gnu/kfreesd. Version-Release number of selected component (if applicable): How reproducible: always Steps to Reproduce: 1. create a qemu disk image and install debian gnu/kfreebsd (I can provide the 126MB lzma compressed image for testing) 2. open the disk image with guestfish 3. mount the partitions Actual results: ><fs> sfdisk-l /dev/sda Disk /dev/sda: 1044 cylinders, 255 heads, 63 sectors/track Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0 Device Boot Start End #cyls #blocks Id System /dev/sda1 * 0+ 304 305- 2449881 83 Linux /dev/sda2 305 1043 739 5936017+ 5 Extended /dev/sda3 0 - 0 0 0 Empty /dev/sda4 0 - 0 0 0 Empty /dev/sda5 305+ 336 32- 257008+ 82 Linux swap / Solaris /dev/sda6 337+ 1043 707- 5678946 83 Linux ><fs> mount /dev/sda1 / libguestfs: error: mount: mount: /dev/sda1 on /: mount: you must specify the filesystem type ><fs> mount -r -t ufs -o ufstype=ufs2 /dev/sda1 / mount should have 2 parameter(s) type 'help mount' for help on mount ><fs> Expected results: be able to mount the partitions and access data Additional info: ufs kernel module was loaded
Comment 1 Richard W.M. Jones 2009-11-26 10:22:30 EST
Could you provide the test image?
Comment 2 Praveen Arimbrathodiyil 2009-11-26 12:12:39 EST
The image is available from http://j4v4m4n.in/share/debian-gnu-kfreebsd.img.lzma
Comment 3 Richard W.M. Jones 2009-11-26 12:41:48 EST
Can you try this command in guestfish: mount-vfs ro,ufstype=ufs2 ufs /dev/sda1 / That works for me on the image you supplied. If it doesn't work, let me know which version of libguestfs / guestfish you are using. ---------------------------- $ guestfish -h mount-vfs NAME mount-vfs - mount a guest disk with mount options and vfstype SYNOPSIS mount-vfs options vfstype device mountpoint DESCRIPTION This is the same as the "mount" command, but it allows you to set both the mount options and the vfstype as for the mount(8) *-o* and *-t* flags.
Comment 4 Praveen Arimbrathodiyil 2009-11-26 13:09:46 EST
yes. It works in read-only mode. Thanks for the quick help. UFS write support is disabled in kernel. I will need to rebuild the kernel for write support. But why is sfdisk-l showing it as Linux File System? Should this be filed as a bug against sfdisk?
Comment 5 Richard W.M. Jones 2009-11-26 14:08:32 EST
(In reply to comment #4) > yes. It works in read-only mode. Thanks for the quick help. > > UFS write support is disabled in kernel. I will need to rebuild the kernel for > write support. Right. libguestfs uses your host kernel to access the filesystem. In fact it chooses the most recent suitable kernel in the /boot directory, and at the moment there is no way to specify that it should pick a particular kernel. This may or may not be a problem if you recompile your kernel - it may end up choosing the wrong one (but if the version number of your custom kernel is higher than any of the others, it should pick that one). > But why is sfdisk-l showing it as Linux File System? Should this be filed as a > bug against sfdisk? sfdisk-l displays what's in the partition table. MBR partition tables contain a single byte which tells the BIOS what operating system is using that partition (in reality nothing much uses this byte). If you want to quickly determine what is really on the partition, I suggest using the 'file' command, like this: ><fs> file /dev/sda1 Unix Fast File system [v2] (little-endian) last mounted on /, last written at Thu Nov 26 13:11:38 2009, clean flag 0, readonly flag 0, number of blocks 1224940, number of data blocks 1183459, number of cylinder groups 14, block size 16384, fragment size 2048, average file size 16384, average number of files in dir 64, pending blocks to free 0, pending inodes to free 0, system-wide uuid 0, minimum percentage of free blocks 8, TIME optimization We also have a command 'vfs-type' which is *supposed* to return something like 'ufs'. Unfortunately it doesn't seem to work for these UFS partitions, which is possibly a bug in Fedora libblkid.
Comment 6 Richard W.M. Jones 2009-11-26 14:19:22 EST
Moving to Virtualization Tools product.
Comment 7 Richard W.M. Jones 2009-11-26 14:20:56 EST
Also of note: virt-inspector completely fails on the supplied disk image. The reason is that the mount commands all fail to find anything mountable: $ LIBGUESTFS_TRACE=1 virt-inspector debian-gnu-kfreebsd.img add_drive_ro "debian-gnu-kfreebsd.img" launch list_partitions pvs lvs file "/dev/sda1" mount_ro "/dev/sda1" "/" umount_all file "/dev/sda2" mount_ro "/dev/sda2" "/" umount_all file "/dev/sda5" umount_all file "/dev/sda6" mount_ro "/dev/sda6" "/" umount_all kill_subprocess
Comment 8 Praveen Arimbrathodiyil 2009-11-26 14:22:40 EST
I will try compiling a newer kernel with write support. Meanwhile I requested ufs2 write support be enabled in default kernel https://bugzilla.redhat.com/show_bug.cgi?id=541703 it would be good to have sfdisk-l use the same method that file command uses to display partition information. Thanks for the explanations.
Comment 9 Praveen Arimbrathodiyil 2009-11-26 14:33:13 EST
Created https://bugzilla.redhat.com/show_bug.cgi?id=541708 for tracking vfs-type/libblkid bug.
Comment 10 Richard W.M. Jones 2009-11-26 14:50:40 EST
(In reply to comment #8) > it would be good to have sfdisk-l use the same method that file command uses to > display partition information. sfdisk-l just displays the output of the sfdisk command. Don't try to parse the output from a program. You should use 'file', or ideally 'vfs-type' (when it's fixed) to determine the filesystem type.
Comment 11 Richard W.M. Jones 2009-11-27 04:51:08 EST
With the patched libblkid from bug 541708: ><fs> vfs-type /dev/sda1 ufs However you still can't automatically mount ufs partitions without specifying the ufstype=ufs2 option. This seems to be a limitation of the ufs kernel driver itself.
Comment 12 Praveen Arimbrathodiyil 2009-11-27 11:35:17 EST
Even with a ufs write support enabled kernel, guestfish cannot upload a file. I don't know if it is a problem with the kernel module itself. ><fs> mount-vfs rw,ufstype=ufs2 ufs /dev/sda1 / ><fs> upload inittab /etc/inittab libguestfs: error: upload: /etc/inittab: Read-only file system I have built a kernel with ufs write support enabled, which may be used for testing this http://j4v4m4n.in/share/kernel-184.108.40.206-127.bz541703.fc12.src.rpm http://j4v4m4n.in/share/kernel-220.127.116.11-127.bz541703.fc12.x86_64.rpm (as you mentioned I had to remove the other kernel as it was trying to take the latest kernel which did not have write support)
Comment 13 Richard W.M. Jones 2009-11-27 12:31:05 EST
First thing to do is to check that libguestfs has picked up your kernel. At the guestfish prompt, do: dmesg | less and take a look at the kernel version. The algorithm it uses is not really designed for people who want to run a specific kernel :-( You can see what it's doing by looking at this line in /usr/bin/libguestfs-supermin-helper: arch=$(echo "x86_64" | sed 's/^i.86$/i?86/') kernels=$(ls -1vr /boot/vmlinuz-*.$arch* 2>/dev/null | grep -v xen; ls -1vr /boot/vmlinuz-* 2>/dev/null | grep -v xen) I will try out your kernel and see if I can get it to work here.
Comment 14 Richard W.M. Jones 2009-11-27 12:38:26 EST
So I removed all my kernels > 18.104.22.168 and installed your RPM, and it does pick up the correct kernel. As you say, the kernel ufs driver is *claiming* that the disk is mounted rw, *but* any attempt to write files results in "Read-only file system" errors: ><fs> mount-vfs ufstype=ufs2 ufs /dev/sda1 / ><fs> upload /etc/inittab /TEST libguestfs: error: upload: /TEST: Read-only file system ><fs> debug sh mount /proc on /proc type proc (rw) /sys on /sys type sysfs (rw) /dev/sda1 on /sysroot type ufs (rw,ufstype=ufs2) ><fs> touch /TEST libguestfs: error: touch: open: /TEST: Read-only file system Therefore my opinion is this is a bug in the ufs driver. It's claiming full read-write support, but not actually delivering it. BTW, if you want to send interactive shell commands, you may prefer to use one of two techniques: (a) in guestfish, use: ><fs> debug sh "cmd arg1 arg2 arg3 ..." (b) or instead of guestfish, use virt-rescue
Comment 15 Richard W.M. Jones 2009-11-30 09:36:15 EST
Quoth: http://ghantoos.org/2009/04/04/mounting-ufs-in-readwrite-under-linux/#comment-1601 If you get the error “Read-only file system” when trying to write to the drive you need to do a fsck on a solaris machine before mounting under linux. Looking at the code there are quire a few states that remark the file system as readonly. Run: dmesg | tail if you see any of the following then it will be marked as readonly ufs_read_super: fs is active ufs_read_super: fs is bad ufs_read_super: can’t grok fs_clean 0xFFFFFFF ufs_read_super: fs needs fsck if you see any of the following then it will be left as readwrite fs is clean fs is stable fs is DEC OSF
Comment 16 Praveen Arimbrathodiyil 2009-11-30 10:05:37 EST
booted a second debian gnu/kfreebsd system as hdb and ran fsck on /dev/ad0s1 (/dev/sda1 in guestfish). Now write is working. ><fs> mount-vfs rw,ufstype=ufs2 ufs /dev/sda1 / ><fs> upload inittab /root/inittab ><fs> ls /root .Xauthority ... inittab ><fs> cat /root/inittab # /etc/inittab: init(8) configuration. # $Id: inittab,v 1.91 2002/01/25 13:35:21 miquels Exp $ # The default runlevel. id:1:initdefault: ... ata2.01: NODEV after polling detection ata2.00: ATAPI: QEMU DVD-ROM, 0.11.0, max UDMA/100 ata2.00: configured for MWDMA2 ata1.01: NODEV after polling detection ata1.00: ATA-7: QEMU HARDDISK, 0.11.0, max UDMA/100 ata1.00: 16777216 sectors, multi 16: LBA48 ata1.00: configured for MWDMA2 scsi 0:0:0:0: Direct-Access ATA QEMU HARDDISK 0.11 PQ: 0 ANSI: 5 sd 0:0:0:0: [sda] 16777216 512-byte logical blocks: (8.58 GB/8.00 GiB) sd 0:0:0:0: [sda] Write Protect is off sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00 sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA sda: sd 0:0:0:0: Attached scsi generic sg0 type 0 ... sda1 sda2 < sda5 sda6 > Thanks for all the help.
Comment 17 Praveen Arimbrathodiyil 2009-11-30 11:05:22 EST
with a corrupt file system (closed qemu window while the guest was extracting emacs) ><rescue> mount -o rw,ufstype=ufs2 -t ufs /dev/sda1 / ufs_read_super: fs is active ufsutils is not packaged in Fedora, so can't do fsck.ufs, right now. Added ufsutils to package maintainers wishlist http://fedoraproject.org/wiki/PackageMaintainers/WishList#T-W will try to compile and run fsck.ufs to fix it. Previously it was fixed by running another debian gnu/kfreebsd as hdb and running fsck.ufs on /dev/ad0s1.
Comment 18 Praveen Arimbrathodiyil 2009-11-30 12:10:20 EST
I copied fsck.ufs and libbsd.so.* from debian. I can see it running from command line guestfish and virt-rescue is unable to find it [pravi@savannah debian]$ ls /sbin/fsck.* /sbin/fsck.cramfs /sbin/fsck.ext4 /sbin/fsck.ufs /sbin/fsck.ext2 /sbin/fsck.ext4dev /sbin/fsck.vfat /sbin/fsck.ext3 /sbin/fsck.msdos /sbin/fsck.xfs [pravi@savannah debian]$ fsck.ufs usage: fsck.ufs [-BCFpfny] [-b block] [-c level] [-m mode] filesystem ... ><rescue> /sbin/fsck. fsck.cramfs fsck.ext3 fsck.ext4dev fsck.vfat fsck.ext2 fsck.ext4 fsck.msdos
Comment 19 Richard W.M. Jones 2009-11-30 14:49:14 EST
(In reply to comment #18) > I copied fsck.ufs and libbsd.so.* from debian. I can see it running from > command line > > guestfish and virt-rescue is unable to find it > > [pravi@savannah debian]$ ls /sbin/fsck.* > /sbin/fsck.cramfs /sbin/fsck.ext4 /sbin/fsck.ufs > /sbin/fsck.ext2 /sbin/fsck.ext4dev /sbin/fsck.vfat > /sbin/fsck.ext3 /sbin/fsck.msdos /sbin/fsck.xfs > [pravi@savannah debian]$ fsck.ufs > usage: fsck.ufs [-BCFpfny] [-b block] [-c level] [-m mode] filesystem ... > > ><rescue> /sbin/fsck. > fsck.cramfs fsck.ext3 fsck.ext4dev fsck.vfat > fsck.ext2 fsck.ext4 fsck.msdos I'm not quite clear where you copied it to. * If it's still on the host system, nothing will be able to see it in the virtual machine unless you 'upload' it. * If it's in the guest VM, then virt-rescue will be able to see it when you mount the VM's disks: mount /dev/sda1 /sysroot ls /sysroot/sbin/fsck.ufs (Of course you may not be able to mount the VM's disks before doing an fsck, but that's another matter). * You can't just add files to the appliance. You'd have to rebuild the appliance, which means rebuilding libguestfs from source, editing appliance/packagelist.in to add any extra packages you need. * You *can* however put the files you need into a separate disk image and attach that. Roughly speaking you'd do this: $ mkisofs -JR -o myutils.img mkfs.ufs ..... $ guestfish -a guest.img -a myutils.img # your utilities are available on /dev/sdb Note that your utilities have to be statically linked, else you may need to include any libraries (libbsd.so* etc) they need.
Comment 20 Praveen Arimbrathodiyil 2009-12-01 02:35:28 EST
I copied it to the host system, assuming it was taking tools from the host (especially since you mentioned it takes kernel modules from the host). * Where does guestfish/virt-rescue, finds fsck programs? Does it ship a copy or does it use the host copy? * OK. second point answers this question, the appliance ships its own copy. * There is no ufsutils package in Fedora, I hope some one picks it from the wishlist. * My own disk image would mean statically linking fsck.ufs. I can't see any configure scripts in ufsutils tarball, and the Makefile does not have any targets. * Can libguestfs edit itself? Can I open libguestfs image from libguestfs? Or at least a copy of it?
Comment 21 Richard W.M. Jones 2009-12-07 04:17:44 EST
(In reply to comment #20) > I copied it to the host system, assuming it was taking tools from the host > (especially since you mentioned it takes kernel modules from the host). > > * Where does guestfish/virt-rescue, finds fsck programs? Does it ship a copy or > does it use the host copy? It ships an appliance which contains the tools: http://libguestfs.org/guestfs.3.html#architecture You can't just add stuff to the appliance, you'd have to rebuild the appliance. > * OK. second point answers this question, the appliance ships its own copy. > > * There is no ufsutils package in Fedora, I hope some one picks it from the > wishlist. So we won't be able to add fsck.ufs to the Fedora version of libguestfs until someone packages this. > * My own disk image would mean statically linking fsck.ufs. I can't see any > configure scripts in ufsutils tarball, and the Makefile does not have any > targets. > > * Can libguestfs edit itself? Can I open libguestfs image from libguestfs? Or > at least a copy of it? No, because the appliance that libguestfs uses isn't a real disk image. http://rwmj.wordpress.com/2009/10/22/supermin-appliance-now-in-febootstrap/#content or: http://libguestfs.org/README.txt (section "Supermin appliance")
Comment 22 Richard W.M. Jones 2009-12-07 04:19:10 EST
(In reply to comment #21) > (In reply to comment #20) > > * There is no ufsutils package in Fedora, I hope some one picks it from the > > wishlist. > > So we won't be able to add fsck.ufs to the Fedora version > of libguestfs until someone packages this. I should add that once someone does package it, it's a single line change in this file: http://git.annexia.org/?p=libguestfs.git;a=blob;f=appliance/packagelist.in;hb=HEAD to add fsck.ufs / ufsutils to libguestfs.
Comment 23 Eric Blake 2010-09-24 14:29:29 EDT
I'm seeing the same issues with a MirBSD disk image, during the Fedora 14 virtualization test day: libguestfs-1.5.18-1.fc14.2.i686 libguestfs-tools-1.5.18-1.fc14.2.i686 # virt-inspector /mnt/backup/opt/libvirt/images/mirbsd.img No operating system could be detected inside this disk image. This may be because the file is not a disk image, or is not a virtual machine image, or because the OS type is not understood by virt-inspector. If you feel this is an error, please file a bug report including as much information about the disk image as possible. # guestfish -a /mnt/backup/opt/libvirt/images/mirbsd.img ><fs> run ><fs> sfdisk_l /dev/sda Disk /dev/vda: 20805 cylinders, 16 heads, 63 sectors/track Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0 Device Boot Start End #cyls #blocks Id System /dev/vda1 0 - 0 0 0 Empty /dev/vda2 0 - 0 0 0 Empty /dev/vda3 0 - 0 0 0 Empty /dev/vda4 * 0+ 1304 1305- 10482381 27 Unknown ><fs> file /dev/sda4 Unix Fast File system [v1] (little-endian), last mounted on /, last written at Thu Sep 23 05:49:57 2010, clean flag 1, number of blocks 4718684, number of data blocks 4643043, number of cylinder groups 58, block size 16384, fragment size 2048, minimum percentage of free blocks 5, rotational delay 0ms, disk rotational speed 60rps, TIME optimization ><fs> vfs-type /dev/sda4 ufs ><fs> mount /dev/sda4 / libguestfs: error: mount: /dev/vda4 on /: mount: wrong fs type, bad option, bad superblock on /dev/vda4, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so ><fs> dmesg | tail [ 1.321915] virtio-pci 0000:00:04.0: irq 44 for MSI/MSI-X [ 1.323886] virtio-pci 0000:00:04.0: irq 45 for MSI/MSI-X [ 1.345811] vdb: unknown partition table [ 1.348465] insmod.static used greatest stack depth: 5868 bytes left [ 1.519660] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input1 [ 1.820673] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0 [ 1.858762] sed used greatest stack depth: 5696 bytes left [ 1.899358] ip used greatest stack depth: 5644 bytes left [ 319.928883] ufs was compiled with read-only support, can't be mounted as read-write ><fs> mount-options ro,ufstype=ufs2 /dev/sda4 / libguestfs: error: mount_options: /dev/vda4 on /: mount: wrong fs type, bad option, bad superblock on /dev/vda4, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so ><fs> mount-options ro,ufstype=44bsd /dev/sda4 / ><fs> ls /home eblake ><fs> umount / ><fs> mount-options ro /dev/sda4 / ><fs> ls /home libguestfs: error: ls: opendir: /home: No such file or directory ><fs> dmesg | tail [ 1.509665] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input1 [ 1.801441] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0 [ 1.941518] ip used greatest stack depth: 5644 bytes left [ 43.110905] You didn't specify the type of your ufs filesystem [ 43.110908] [ 43.110909] mount -t ufs -o ufstype=sun|sunx86|44bsd|ufs2|5xbsd|old|hp|nextstep|nextstep-cd|openstep ... [ 43.110912] [ 43.110913] >>>WARNING<<< Wrong ufstype may corrupt your filesystem, default is ufstype=old [ 45.314476] UFS-fs error (device vda4): ufs_check_page: bad entry in directory #2: rec_len is too small for name_len - offset=0, rec_len=12, name_len=260 Should I file a separate bug about virt-inspector not trying all the possible ufstype=??? options before giving up on reading a ufs disk? Then again, given that mounting with ufstype=old 'succeeded' but could not read the filesystem, how do we know which order the various ufstypes should safely be tried?
Comment 24 Richard W.M. Jones 2010-09-24 16:54:45 EDT
I'm more convinced this is a UFS or kernel bug. AFAIK UFS has all sorts of variations, depending on what BSD kernel they were created under, and the variations are not reflected in a detectable way in the superblock. Therefore if you mount the wrong one you could read corrupted data, and since we can't read it to find out what BSD it is, we can't know correctly how to mount it. Maybe there is some workaround for the above ... Or maybe because we are only reading, trying all variations is good enough. Note in the C inspection code, which shadows the Perl inspection code in virt-inspector (and obsoletes it eventually) we have a hack for FreeBSD: http://git.annexia.org/?p=libguestfs.git;a=blob;f=src/inspect.c;h=1f4096d1529f8a39339d10094eb7dddaaa9bf16c;hb=HEAD#l544 Therefore (eg) guestfish -i might be able to see your guests.
Comment 25 Eric Blake 2010-09-24 17:38:20 EDT
That particular commit of C code will probalby work for FreeBSD (where ufstype=ufs2 is needed) but still fail on MirBSD (where ufstype=44bsd is needed). But I agree with you that since we are mounting things read-only until we know for sure _which_ option works, that it would be rather easy to set up a for-loop that tries _all_ of the options one after the other until one of them results in a readable directory - it's only dangerous if you attempt a read-write mount without getting ufstype correct. Per the dmesg output, that's up to 10 mount attempts before declaring the partition unmountable: ufstype=sun|sunx86|44bsd|ufs2|5xbsd|old|hp|nextstep|nextstep-cd|openstep Alas, it is not as simple as whether the mount succeeds (on my MirBSD setup, ufstype=ufs2 failed to mount because of a bad magic number, while ufstype=old completed mount and it wasn't until readdir failed to see anything in the mounted directory that I concluded that old was the wrong type; it wasn't until I tried ufstype44bsd that I got in), but the combination of a successful read-only mount plus successful readdir ought to be enough to know which option to then output for guestfish -i to use for read-write mounting, all without knowing which BSD created the image and without maintaining a map of BSD flavors to ufs styles. It's annoying that the kernel can't automate this loop for you, and insists on you passing the ufstype argument; it's also annoying that file and/or libblkid doesn't know how to inspect the ufs partition well enough to tell you the correct ufstype= mount argument to use in the first place. But I think libguestfs can still get the right behavior whether or not these other tools are improved in their automated ufs handling.
Comment 26 Richard W.M. Jones 2012-04-16 03:35:02 EDT
*** Bug 812706 has been marked as a duplicate of this bug. ***
Comment 28 Richard W.M. Jones 2013-11-22 10:24:02 EST
Note that ufs.ko is now in the kernel-modules-extra package in Fedora. If you don't explicitly install this package then nothing UFS-related will work at all.