Bug 617165

Created attachment 433671 [details]
output info of virt-ls command with an image which has this problem

this issue happened on a RHEL5.5_32 image and a RHEL5.4_64 image

This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

after checked the images (boot it with qemu-kvm), there was some minor error in filesystem:
[...]
Mounting root filesystem.
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access wilee be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
[...]

libguestfs should do some enhancement for such cases (quit after mount failed):

[host]# pstree -p 706
virt-ls(706)─┬─qemu-kvm(713)───{qemu-kvm}(715)
             └─virt-ls(714)
[host]# strace -p 706
Process 706 attached - interrupt to quit
select(7, [4 6], NULL, NULL, NULL^C <unfinished ...>
Process 706 detached
[host]# strace -p 714
Process 714 attached - interrupt to quit
restart_syscall(<... resuming interrupted call ...>) = 0
kill(713, SIG_0)                        = 0
kill(706, SIG_0)                        = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({2, 0}, 0x7fff68c46ac0)       = 0
kill(713, SIG_0)                        = 0
kill(706, SIG_0)                        = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({2, 0}, 0x7fff68c46ac0)       = 0
kill(713, SIG_0)                        = 0
kill(706, SIG_0)                        = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({2, 0}, 0x7fff68c46ac0)       = 0
kill(713, SIG_0)                        = 0
kill(706, SIG_0)                        = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({2, 0}, 0x7fff68c46ac0)       = 0
kill(713, SIG_0)                        = 0
kill(706, SIG_0)                        = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({2, 0}, 0x7fff68c46ac0)       = 0
kill(713, SIG_0)                        = 0
kill(706, SIG_0)                        = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({2, 0}, 0x7fff68c46ac0)       = 0
kill(713, SIG_0)                        = 0
kill(706, SIG_0)                        = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({2, 0}, 0x7fff68c46ac0)       = 0
kill(713, SIG_0)                        = 0
kill(706, SIG_0)                        = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({2, 0}, 0x7fff68c46ac0)       = 0
kill(713, SIG_0)                        = 0
kill(706, SIG_0)                        = 0

Reproducer #1:

$ cat make-image.sh 
#!/bin/sh -
guestfish <<EOF
sparse /tmp/test.img 1G
run
part-disk /dev/sda mbr
mkfs ext3 /dev/sda1
mount /dev/sda1 /
touch /hello
sync
sleep 1
kill-subprocess
EOF

$ ./make-image.sh
[ignore the error from the above command]
$ guestfish --ro -a /tmp/test.img -m /dev/sda1 -v

The error you will see is:

  mount -o ro /dev/vda1 /sysroot/
  EXT3-fs: INFO: recovery required on readonly filesystem.
  EXT3-fs: write access unavailable, cannot proceed.

Reproducer #2:


$ cat make-image.sh 
#!/bin/sh -
guestfish <<EOF
sparse /tmp/test.img 1G
run
part-disk /dev/sda mbr
pvcreate /dev/sda1
vgcreate VG /dev/sda1
lvcreate LV VG 128
mkfs ext3 /dev/VG/LV
mount /dev/VG/LV /
touch /hello
sync
sleep 1
kill-subprocess
EOF

$ ./make-image.sh
[ignore the error from the above command]
$ guestfish --ro -a /tmp/test.img -m /dev/VG/LV -v

The error you will see is different from reproducer #1 but
similar to the errors shown in Linglu's comment 1:

  mount -o ro /dev/VG/LV /sysroot/
  EXT3-fs: INFO: recovery required on readonly filesystem.
  EXT3-fs: write access will be enabled during recovery.
  end_request: I/O error, dev vda, sector 389
  Buffer I/O error on device dm-0, logical block 2
  lost page write due to I/O error on dm-0
  [more of these errors or hangs]

The solution to this is to remove the code which detects if the
readonly=yes|no option is supported by qemu.  Instead we should
just use snapshots as we do on Fedora.

http://git.annexia.org/?p=libguestfs.git;a=blob;f=src/guestfs.c;h=85a042a0a2eaf6b48d068f8e1d4ee0d854018e2f;hb=HEAD#l852

Since this is also an upstream bug, I will clone this and fix it
upstream first.

Upstream patch here:
http://git.annexia.org/?p=libguestfs.git;a=commitdiff;h=799d52be4f08f6c70c0e8ba1aa7367ba4cdd78c4

I will try this on my private RHEL 6 VM first to see
if it resolves the issues in comment 6.

Brew build:
https://brewweb.devel.redhat.com/taskinfo?taskID=2618737

I'm still testing this one.

(In reply to comment #8)
> Upstream patch here:
> http://git.annexia.org/?p=libguestfs.git;a=commitdiff;h=799d52be4f08f6c70c0e8ba1aa7367ba4cdd78c4
> 

Seems this fix will affect bug 612493 as well.

Sorry, took quite a lot longer to test this than I anticipated,
because I had to download and install RHEL 6 beta 2 refresh.

Anyway, I can confirm that the build in comment 9 fixes
the problem.

Non-scratch brew build:
https://brewweb.devel.redhat.com/taskinfo?taskID=2621237

1- Update packages:
guestfish-1.2.7-1.21.el6.x86_64.rpm
libguestfs-mount-1.2.7-1.21.el6.x86_64.rpm
perl-libguestfs-1.2.7-1.21.el6.x86_64.rpm
libguestfs-1.2.7-1.21.el6.x86_64.rpm
libguestfs-tools-1.2.7-1.21.el6.x86_64.rpm

2- Verified with the Reproducer in Comment 6:
Instead of the error message "...write access unavailable, cannot proceed...", the mount operation ran well with following message:
    mount -o ro /dev/vda1 /sysroot/
    [    2.955797] EXT3-fs: INFO: recovery required on readonly filesystem.
    [    2.957494] EXT3-fs: write access will be enabled during recovery.
    [    3.557200] kjournald starting.  Commit interval 5 seconds
    [    3.561454] EXT3-fs: recovery complete.
    [    3.564659] EXT3-fs: mounted filesystem with ordered data mode.
Test passed.

3- Verified with the images mention in Comment 2:
Same situation with PART 2.
Test passed.

Move to VERIFIED according to comment 14.

I made a small modification to the patch -- to fix the documentation.
It probably doesn't affect anything, but I've put this back to MODIFIED
just in case.  The new build is libguestfs-1.2.7-1.23.el6 (snap 11).

Test passed with the same procedures in Comment 14.

*** Bug 612493 has been marked as a duplicate of this bug. ***

Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.