Bug 541618

Could you provide the test image?

The image is available from http://j4v4m4n.in/share/debian-gnu-kfreebsd.img.lzma

Can you try this command in guestfish:

  mount-vfs ro,ufstype=ufs2 ufs /dev/sda1 /

That works for me on the image you supplied.

If it doesn't work, let me know which version of
libguestfs / guestfish you are using.

----------------------------

$ guestfish -h mount-vfs
NAME
    mount-vfs - mount a guest disk with mount options and vfstype

SYNOPSIS
     mount-vfs options vfstype device mountpoint

DESCRIPTION
    This is the same as the "mount" command, but it allows you to set both
    the mount options and the vfstype as for the mount(8) *-o* and *-t*
    flags.

yes. It works in read-only mode. Thanks for the quick help.

UFS write support is disabled in kernel. I will need to rebuild the kernel for write support.

But why is sfdisk-l showing it as Linux File System? Should this be filed as a bug against sfdisk?

(In reply to comment #4)
> yes. It works in read-only mode. Thanks for the quick help.
> 
> UFS write support is disabled in kernel. I will need to rebuild the kernel for
> write support.

Right.  libguestfs uses your host kernel to access the filesystem.
In fact it chooses the most recent suitable kernel in the /boot
directory, and at the moment there is no way to specify that it
should pick a particular kernel.  This may or may not be a problem
if you recompile your kernel - it may end up choosing the wrong one
(but if the version number of your custom kernel is higher than any
of the others, it should pick that one).

> But why is sfdisk-l showing it as Linux File System? Should this be filed as a
> bug against sfdisk?  

sfdisk-l displays what's in the partition table.  MBR partition
tables contain a single byte which tells the BIOS what operating
system is using that partition (in reality nothing much uses
this byte).

If you want to quickly determine what is really on the partition,
I suggest using the 'file' command, like this:

><fs> file /dev/sda1
Unix Fast File system [v2] (little-endian) last mounted on /, last written at Thu Nov 26 13:11:38 2009, clean flag 0, readonly flag 0, number of blocks 1224940, number of data blocks 1183459, number of cylinder groups 14, block size 16384, fragment size 2048, average file size 16384, average number of files in dir 64, pending blocks to free 0, pending inodes to free 0, system-wide uuid 0, minimum percentage of free blocks 8, TIME optimization

We also have a command 'vfs-type' which is *supposed* to return
something like 'ufs'.  Unfortunately it doesn't seem to work for
these UFS partitions, which is possibly a bug in Fedora libblkid.

Moving to Virtualization Tools product.

Also of note: virt-inspector completely fails on the
supplied disk image.  The reason is that the mount commands
all fail to find anything mountable:

$ LIBGUESTFS_TRACE=1 virt-inspector debian-gnu-kfreebsd.img 
add_drive_ro "debian-gnu-kfreebsd.img"
launch
list_partitions
pvs
lvs
file "/dev/sda1"
mount_ro "/dev/sda1" "/"
umount_all
file "/dev/sda2"
mount_ro "/dev/sda2" "/"
umount_all
file "/dev/sda5"
umount_all
file "/dev/sda6"
mount_ro "/dev/sda6" "/"
umount_all
kill_subprocess

I will try compiling a newer kernel with write support. Meanwhile I requested
ufs2 write support be enabled in default kernel
https://bugzilla.redhat.com/show_bug.cgi?id=541703

it would be good to have sfdisk-l use the same method that file command uses to
display partition information.

Thanks for the explanations.

Created https://bugzilla.redhat.com/show_bug.cgi?id=541708 for tracking vfs-type/libblkid bug.

(In reply to comment #8)
> it would be good to have sfdisk-l use the same method that file command uses to
> display partition information.

sfdisk-l just displays the output of the sfdisk command.
Don't try to parse the output from a program.  You
should use 'file', or ideally 'vfs-type' (when it's fixed)
to determine the filesystem type.

With the patched libblkid from bug 541708:

 ><fs> vfs-type /dev/sda1
 ufs

However you still can't automatically mount ufs
partitions without specifying the ufstype=ufs2 option.
This seems to be a limitation of the ufs kernel
driver itself.

Even with a ufs write support enabled kernel, guestfish cannot upload a file. I don't know if it is a problem with the kernel module itself.

><fs> mount-vfs rw,ufstype=ufs2 ufs /dev/sda1 /
><fs> upload inittab /etc/inittab
libguestfs: error: upload: /etc/inittab: Read-only file system

I have built a kernel with ufs write support enabled, which may be used for testing this

http://j4v4m4n.in/share/kernel-2.6.31.5-127.bz541703.fc12.src.rpm
http://j4v4m4n.in/share/kernel-2.6.31.5-127.bz541703.fc12.x86_64.rpm

(as you mentioned I had to remove the other kernel as it was trying to take the latest kernel which did not have write support)

First thing to do is to check that libguestfs has picked
up your kernel.  At the guestfish prompt, do:

  dmesg | less

and take a look at the kernel version.

The algorithm it uses is not really designed for people
who want to run a specific kernel :-(  You can see what
it's doing by looking at this line in /usr/bin/libguestfs-supermin-helper:

arch=$(echo "x86_64" | sed 's/^i.86$/i?86/')
kernels=$(ls -1vr /boot/vmlinuz-*.$arch* 2>/dev/null | grep -v xen; ls -1vr /boot/vmlinuz-* 2>/dev/null | grep -v xen)

I will try out your kernel and see if I can get it to
work here.

So I removed all my kernels > 2.6.31.5 and installed
your RPM, and it does pick up the correct kernel.

As you say, the kernel ufs driver is *claiming* that
the disk is mounted rw, *but* any attempt to write
files results in "Read-only file system" errors:

><fs> mount-vfs ufstype=ufs2 ufs /dev/sda1 /
><fs> upload /etc/inittab /TEST
libguestfs: error: upload: /TEST: Read-only file system
><fs> debug sh mount
/proc on /proc type proc (rw)
/sys on /sys type sysfs (rw)
/dev/sda1 on /sysroot type ufs (rw,ufstype=ufs2)
><fs> touch /TEST
libguestfs: error: touch: open: /TEST: Read-only file system

Therefore my opinion is this is a bug in the ufs
driver.  It's claiming full read-write support, but
not actually delivering it.

BTW, if you want to send interactive shell commands,
you may prefer to use one of two techniques:

(a) in guestfish, use:

><fs> debug sh "cmd arg1 arg2 arg3 ..."

(b) or instead of guestfish, use virt-rescue

Quoth:
http://ghantoos.org/2009/04/04/mounting-ufs-in-readwrite-under-linux/#comment-1601

If you get the error “Read-only file system” when trying to write to the drive you need to do a fsck on
a solaris machine before mounting under linux. Looking at the code there are quire a few states that
remark the file system as readonly.

Run: dmesg | tail
if you see any of the following then it will be marked as readonly
ufs_read_super: fs is active
ufs_read_super: fs is bad
ufs_read_super: can’t grok fs_clean 0xFFFFFFF
ufs_read_super: fs needs fsck

if you see any of the following then it will be left as readwrite
fs is clean
fs is stable
fs is DEC OSF

booted a second debian gnu/kfreebsd system as hdb and ran fsck on /dev/ad0s1 (/dev/sda1 in guestfish).

Now write is working.

><fs> mount-vfs rw,ufstype=ufs2 ufs /dev/sda1 /
><fs> upload inittab /root/inittab
><fs> ls /root
.Xauthority

...

inittab
><fs> cat /root/inittab
# /etc/inittab: init(8) configuration.
# $Id: inittab,v 1.91 2002/01/25 13:35:21 miquels Exp $

# The default runlevel.
id:1:initdefault:

...

ata2.01: NODEV after polling detection
ata2.00: ATAPI: QEMU DVD-ROM, 0.11.0, max UDMA/100
ata2.00: configured for MWDMA2
ata1.01: NODEV after polling detection
ata1.00: ATA-7: QEMU HARDDISK, 0.11.0, max UDMA/100
ata1.00: 16777216 sectors, multi 16: LBA48 
ata1.00: configured for MWDMA2
scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK    0.11 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 16777216 512-byte logical blocks: (8.58 GB/8.00 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sda:
sd 0:0:0:0: Attached scsi generic sg0 type 0


...
 sda1 sda2 < sda5 sda6 >

Thanks for all the help.

with a corrupt file system (closed qemu window while the guest was extracting emacs)

><rescue> mount -o rw,ufstype=ufs2 -t ufs /dev/sda1 /
ufs_read_super: fs is active

ufsutils is not packaged in Fedora, so can't do fsck.ufs, right now. Added ufsutils to package maintainers wishlist http://fedoraproject.org/wiki/PackageMaintainers/WishList#T-W

will try to compile and run fsck.ufs to fix it. Previously it was fixed by running another debian gnu/kfreebsd as hdb and running fsck.ufs on /dev/ad0s1.

I copied fsck.ufs and libbsd.so.* from debian. I can see it running from command line

guestfish and virt-rescue is unable to find it

[pravi@savannah debian]$ ls /sbin/fsck.*
/sbin/fsck.cramfs  /sbin/fsck.ext4     /sbin/fsck.ufs
/sbin/fsck.ext2    /sbin/fsck.ext4dev  /sbin/fsck.vfat
/sbin/fsck.ext3    /sbin/fsck.msdos    /sbin/fsck.xfs
[pravi@savannah debian]$ fsck.ufs 
usage: fsck.ufs [-BCFpfny] [-b block] [-c level] [-m mode] filesystem ...

><rescue> /sbin/fsck. 
fsck.cramfs   fsck.ext3     fsck.ext4dev  fsck.vfat     
fsck.ext2     fsck.ext4     fsck.msdos

(In reply to comment #18)
> I copied fsck.ufs and libbsd.so.* from debian. I can see it running from
> command line
> 
> guestfish and virt-rescue is unable to find it
> 
> [pravi@savannah debian]$ ls /sbin/fsck.*
> /sbin/fsck.cramfs  /sbin/fsck.ext4     /sbin/fsck.ufs
> /sbin/fsck.ext2    /sbin/fsck.ext4dev  /sbin/fsck.vfat
> /sbin/fsck.ext3    /sbin/fsck.msdos    /sbin/fsck.xfs
> [pravi@savannah debian]$ fsck.ufs 
> usage: fsck.ufs [-BCFpfny] [-b block] [-c level] [-m mode] filesystem ...
> 
> ><rescue> /sbin/fsck. 
> fsck.cramfs   fsck.ext3     fsck.ext4dev  fsck.vfat     
> fsck.ext2     fsck.ext4     fsck.msdos  

I'm not quite clear where you copied it to.

* If it's still on the host system, nothing will be able to see
it in the virtual machine unless you 'upload' it.

* If it's in the guest VM, then virt-rescue will be able to see
it when you mount the VM's disks:

mount /dev/sda1 /sysroot
ls /sysroot/sbin/fsck.ufs

(Of course you may not be able to mount the VM's disks before
doing an fsck, but that's another matter).

* You can't just add files to the appliance.  You'd have to
rebuild the appliance, which means rebuilding libguestfs
from source, editing appliance/packagelist.in to add any
extra packages you need.

* You *can* however put the files you need into a separate
disk image and attach that.  Roughly speaking you'd do this:

$ mkisofs -JR -o myutils.img mkfs.ufs .....

$ guestfish -a guest.img -a myutils.img
# your utilities are available on /dev/sdb

Note that your utilities have to be statically linked, else
you may need to include any libraries (libbsd.so* etc) they need.

I copied it to the host system, assuming it was taking tools from the host (especially since you mentioned it takes kernel modules from the host).

* Where does guestfish/virt-rescue, finds fsck programs? Does it ship a copy or does it use the host copy?

* OK. second point answers this question, the appliance ships its own copy. 

* There is no ufsutils package in Fedora, I hope some one picks it from the wishlist.

* My own disk image would mean statically linking fsck.ufs. I can't see any configure scripts in ufsutils tarball, and the Makefile does not have any targets.

* Can libguestfs edit itself? Can I open libguestfs image from libguestfs? Or at least a copy of it?

(In reply to comment #20)
> I copied it to the host system, assuming it was taking tools from the host
> (especially since you mentioned it takes kernel modules from the host).
> 
> * Where does guestfish/virt-rescue, finds fsck programs? Does it ship a copy or
> does it use the host copy?

It ships an appliance which contains the tools:

http://libguestfs.org/guestfs.3.html#architecture

You can't just add stuff to the appliance, you'd have to rebuild
the appliance.

> * OK. second point answers this question, the appliance ships its own copy. 
> 
> * There is no ufsutils package in Fedora, I hope some one picks it from the
> wishlist.

So we won't be able to add fsck.ufs to the Fedora version
of libguestfs until someone packages this.

> * My own disk image would mean statically linking fsck.ufs. I can't see any
> configure scripts in ufsutils tarball, and the Makefile does not have any
> targets.
> 
> * Can libguestfs edit itself? Can I open libguestfs image from libguestfs? Or
> at least a copy of it?  

No, because the appliance that libguestfs uses isn't a
real disk image.

http://rwmj.wordpress.com/2009/10/22/supermin-appliance-now-in-febootstrap/#content
or: http://libguestfs.org/README.txt (section "Supermin appliance")

(In reply to comment #21)
> (In reply to comment #20)
> > * There is no ufsutils package in Fedora, I hope some one picks it from the
> > wishlist.
> 
> So we won't be able to add fsck.ufs to the Fedora version
> of libguestfs until someone packages this.

I should add that once someone does package it, it's a single
line change in this file:

http://git.annexia.org/?p=libguestfs.git;a=blob;f=appliance/packagelist.in;hb=HEAD

to add fsck.ufs / ufsutils to libguestfs.

I'm seeing the same issues with a MirBSD disk image, during the Fedora 14 virtualization test day:

libguestfs-1.5.18-1.fc14.2.i686
libguestfs-tools-1.5.18-1.fc14.2.i686


# virt-inspector /mnt/backup/opt/libvirt/images/mirbsd.img 
No operating system could be detected inside this disk image.

This may be because the file is not a disk image, or is not a virtual machine
image, or because the OS type is not understood by virt-inspector.

If you feel this is an error, please file a bug report including as much
information about the disk image as possible.

# guestfish -a /mnt/backup/opt/libvirt/images/mirbsd.img 
><fs> run
><fs> sfdisk_l /dev/sda

Disk /dev/vda: 20805 cylinders, 16 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/vda1          0       -       0          0    0  Empty
/dev/vda2          0       -       0          0    0  Empty
/dev/vda3          0       -       0          0    0  Empty
/dev/vda4   *      0+   1304    1305-  10482381   27  Unknown

><fs> file /dev/sda4
Unix Fast File system [v1] (little-endian), last mounted on /, last written at Thu Sep 23 05:49:57 2010, clean flag 1, number of blocks 4718684, number of data blocks 4643043, number of cylinder groups 58, block size 16384, fragment size 2048, minimum percentage of free blocks 5, rotational delay 0ms, disk rotational speed 60rps, TIME optimization
><fs> vfs-type /dev/sda4
ufs
><fs> mount /dev/sda4 /
libguestfs: error: mount: /dev/vda4 on /: mount: wrong fs type, bad option, bad superblock on /dev/vda4,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
><fs> dmesg | tail
[    1.321915] virtio-pci 0000:00:04.0: irq 44 for MSI/MSI-X
[    1.323886] virtio-pci 0000:00:04.0: irq 45 for MSI/MSI-X
[    1.345811]  vdb: unknown partition table
[    1.348465] insmod.static used greatest stack depth: 5868 bytes left
[    1.519660] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input1
[    1.820673] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0
[    1.858762] sed used greatest stack depth: 5696 bytes left
[    1.899358] ip used greatest stack depth: 5644 bytes left
[  319.928883] ufs was compiled with read-only support, can't be mounted as read-write

><fs> mount-options ro,ufstype=ufs2 /dev/sda4 /
libguestfs: error: mount_options: /dev/vda4 on /: mount: wrong fs type, bad option, bad superblock on /dev/vda4,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
><fs> mount-options ro,ufstype=44bsd /dev/sda4 /
><fs> ls /home
eblake
><fs> umount /
><fs> mount-options ro /dev/sda4 /
><fs> ls /home
libguestfs: error: ls: opendir: /home: No such file or directory
><fs> dmesg | tail
[    1.509665] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input1
[    1.801441] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0
[    1.941518] ip used greatest stack depth: 5644 bytes left
[   43.110905] You didn't specify the type of your ufs filesystem
[   43.110908] 
[   43.110909] mount -t ufs -o ufstype=sun|sunx86|44bsd|ufs2|5xbsd|old|hp|nextstep|nextstep-cd|openstep ...
[   43.110912] 
[   43.110913] >>>WARNING<<< Wrong ufstype may corrupt your filesystem, default is ufstype=old
[   45.314476] UFS-fs error (device vda4): ufs_check_page: bad entry in directory #2: rec_len is too small for name_len - offset=0, rec_len=12, name_len=260



Should I file a separate bug about virt-inspector not trying all the possible ufstype=??? options before giving up on reading a ufs disk?  Then again, given that mounting with ufstype=old 'succeeded' but could not read the filesystem, how do we know which order the various ufstypes should safely be tried?

I'm more convinced this is a UFS or kernel bug.

AFAIK UFS has all sorts of variations, depending on what
BSD kernel they were created under, and the variations are
not reflected in a detectable way in the superblock.
Therefore if you mount the wrong one you could read
corrupted data, and since we can't read it to find out
what BSD it is, we can't know correctly how to mount it.

Maybe there is some workaround for the above ...  Or maybe
because we are only reading, trying all variations is
good enough.

Note in the C inspection code, which shadows the Perl
inspection code in virt-inspector (and obsoletes it
eventually) we have a hack for FreeBSD:

http://git.annexia.org/?p=libguestfs.git;a=blob;f=src/inspect.c;h=1f4096d1529f8a39339d10094eb7dddaaa9bf16c;hb=HEAD#l544

Therefore (eg) guestfish -i might be able to see your
guests.

That particular commit of C code will probalby work for FreeBSD (where ufstype=ufs2 is needed) but still fail on  MirBSD (where ufstype=44bsd is needed).  But I agree with you that since we are mounting things read-only until we know for sure _which_ option works, that it would be rather easy to set up a for-loop that tries _all_ of the options one after the other until one of them results in a readable directory - it's only dangerous if you attempt a read-write mount without getting ufstype correct.

Per the dmesg output, that's up to 10 mount attempts before declaring the partition unmountable:

ufstype=sun|sunx86|44bsd|ufs2|5xbsd|old|hp|nextstep|nextstep-cd|openstep

Alas, it is not as simple as whether the mount succeeds (on my MirBSD setup, ufstype=ufs2 failed to mount because of a bad magic number, while ufstype=old completed mount and it wasn't until readdir failed to see anything in the mounted directory that I concluded that old was the wrong type; it wasn't until I tried ufstype44bsd that I got in), but the combination of a successful read-only mount plus successful readdir ought to be enough to know which option to then output for guestfish -i to use for read-write mounting, all without knowing which BSD created the image and without maintaining a map of BSD flavors to ufs styles.

It's annoying that the kernel can't automate this loop for you, and insists on you passing the ufstype argument; it's also annoying that file and/or libblkid doesn't know how to inspect the ufs partition well enough to tell you the correct ufstype= mount argument to use in the first place.  But I think libguestfs can still get the right behavior whether or not these other tools are improved in their automated ufs handling.

*** Bug 812706 has been marked as a duplicate of this bug. ***

Note that ufs.ko is now in the kernel-modules-extra package
in Fedora.  If you don't explicitly install this package then
nothing UFS-related will work at all.

This cannot be fixed.  See:
https://listman.redhat.com/archives/libguestfs/2021-December/thread.html#00092