Bug 171983 - ext3 assertion when mounting an lvm read-only snapshot
ext3 assertion when mounting an lvm read-only snapshot
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: kernel (Show other bugs)
3.0
i386 Linux
medium Severity high
: ---
: ---
Assigned To: Heinz Mauelshagen
Brian Brock
:
Depends On:
Blocks: 170417
  Show dependency treegraph
 
Reported: 2005-10-28 12:40 EDT by Christopher Blizzard
Modified: 2007-11-30 17:07 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-05-11 14:27:38 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Christopher Blizzard 2005-10-28 12:40:34 EDT
Description of problem:

kernel-smp-2.4.21-37.EL

Version-Release number of selected component (if applicable):

kernel-smp-2.4.21-37.EL
lvm-1.0.8-14

How reproducible:

Every time.


Steps to Reproduce:
1. Create a snapshot volume of an existing LVM paritition.
2. Attempt to mount it.

Actual results:

Assertion in the kernel.  Unable to remove the snapshot volume and many commants
(such as sync) hang and are unkillable.

Expected results:

Mount is successful.

Additional info:

I have a machine where we have one volume group on a raid partition:

--- Volume group ---
VG Name               Volume00
VG Access             read/write
VG Status             available/resizable
VG #                  0
MAX LV                256
Cur LV                2
Open LV               2
MAX LV Size           255.99 GB
Max PV                256
Cur PV                1
Act PV                1
VG Size               111.68 GB
PE Size               4 MB
Total PE              28591
Alloc PE / Size       28000 / 109.38 GB
Free  PE / Size       591 / 2.31 GB
VG UUID               QwEMPM-dpxY-tusl-3ec8-bsnd-SUHj-1hun8m

Note that it has some free PE space.

That contains two logical volumes:

[root@mav-server4 root]# lvdisplay /dev/Volume00/LogVol00
--- Logical volume ---
LV Name                /dev/Volume00/LogVol00
VG Name                Volume00
LV Write Access        read/write
LV Status              available
LV #                   1
# open                 1
LV Size                107.42 GB
Current LE             27500
Allocated LE           27500
Allocation             next free
Read ahead sectors     1024
Block device           58:0


[root@mav-server4 root]# lvdisplay /dev/Volume00/LogVol01
--- Logical volume ---
LV Name                /dev/Volume00/LogVol01
VG Name                Volume00
LV Write Access        read/write
LV Status              available
LV #                   2
# open                 1
LV Size                1.95 GB
Current LE             500
Allocated LE           500
Allocation             next free
Read ahead sectors     1024
Block device           58:1

One is the main disk partition and the other is used for swap.

Each night I create a snapshot volume and do a backup on that snapshot.  The
command being used to create that snapshot is this:

lvcreate -l 591 -s -n Snapshot00 /dev/Volume00/LogVol00

and then mount...

mount -o ro /dev/Volume00/Snapshot00 /mnt/Snapshot00
mount --bind /boot /mnt/Snapshot00/boot

One of those mount command causes an assertion in the ext3 code.  The logged
assertion is this:

Oct 19 03:17:36 mav-server4 kernel: EXT3-fs: lvm(58,2): orphan cleanup on
readonly fs
Oct 19 03:17:36 mav-server4 kernel: EXT3-fs: lvm(58,2): 5 orphan inodes deleted
Oct 19 03:17:36 mav-server4 kernel: EXT3-fs: recovery complete.
Oct 19 03:17:36 mav-server4 kernel: lvm - lvm_map: ll_rw_blk write for readonly
LV /dev/Volume00/Snapshot00
Oct 19 03:17:36 mav-server4 last message repeated 11 times
Oct 19 03:17:36 mav-server4 kernel: Error (-5) on journal on device 3a:02
Oct 19 03:17:36 mav-server4 kernel: Aborting journal on device lvm(58,2).
Oct 19 03:17:36 mav-server4 kernel: journal commit I/O error
Oct 19 03:17:36 mav-server4 kernel: Can't write to read-only device 3a:02
Oct 19 03:17:36 mav-server4 kernel: Assertion failure in
journal_flush_Rsmp_e2f189ce() at journal.c:1356: "!journal->j_
checkpoint_transactions"
Oct 19 03:17:36 mav-server4 kernel: ------------[ cut here ]------------
Oct 19 03:17:36 mav-server4 kernel: kernel BUG at journal.c:1356!
Oct 19 03:17:36 mav-server4 kernel: invalid operand: 0000
Oct 19 03:17:36 mav-server4 kernel: audit usbserial parport_pc lp parport
autofs4 e1000 iptable_filter ip_tables floppy
 sg sr_mod ide-scsi ide-cd cdrom loop keybdev mousedev hid input usb-uhci u
Oct 19 03:17:36 mav-server4 kernel: CPU:    1
Oct 19 03:17:36 mav-server4 kernel: EIP:    0060:[<f8862d5c>]    Not tainted
Oct 19 03:17:36 mav-server4 kernel: EFLAGS: 00010282
Oct 19 03:17:36 mav-server4 kernel: 
Oct 19 03:17:36 mav-server4 kernel: EIP is at journal_flush_Rsmp_e2f189ce [jbd]
0x1bc (2.4.21-37.ELsmp/i686)
Oct 19 03:17:36 mav-server4 kernel: eax: 0000006f   ebx: d66a4400   ecx:
00000001   edx: c0384e98
Oct 19 03:17:36 mav-server4 kernel: esi: e7191000   edi: 00000000   ebp:
fffffffb   esp: c2bf9e18
Oct 19 03:17:36 mav-server4 kernel: ds: 0068   es: 0068   ss: 0068
Oct 19 03:17:36 mav-server4 kernel: Process mount (pid: 10126, stackpage=c2bf9000)
Oct 19 03:17:36 mav-server4 kernel: Stack: f88652b0 f88641e5 f88640ee 0000054c
f88655c8 db715400 db715400 cce74400 
Oct 19 03:17:36 mav-server4 kernel:        00000001 f8876f29 e7191000 db715448
db715400 db715448 f887667c db715400 
Oct 19 03:17:36 mav-server4 kernel:        cce74400 00000001 00000000 c2bf9e9c
00000000 00000007 00000000 00000000 
Oct 19 03:17:36 mav-server4 kernel: Call Trace:   [<f88652b0>] .rodata.str1.4
[jbd] 0xf70 (0xc2bf9e18)
Oct 19 03:17:36 mav-server4 kernel: [<f88641e5>] .rodata.str1.1 [jbd] 0x675
(0xc2bf9e1c)
Oct 19 03:17:36 mav-server4 kernel: [<f88640ee>] .rodata.str1.1 [jbd] 0x57e
(0xc2bf9e20)
Oct 19 03:17:36 mav-server4 kernel: [<f88655c8>] .rodata.str1.4 [jbd] 0x1288
(0xc2bf9e28)
Oct 19 03:17:36 mav-server4 kernel: [<f8876f29>] ext3_mark_recovery_complete
[ext3] 0x19 (0xc2bf9e3c)
Oct 19 03:17:36 mav-server4 kernel: [<f887667c>] ext3_read_super [ext3] 0x73c
(0xc2bf9e50)
Oct 19 03:17:36 mav-server4 kernel: [<c016c392>] get_sb_bdev [kernel] 0x1b2
(0xc2bf9eb0)
Oct 19 03:17:36 mav-server4 kernel: [<f887fa34>] ext3_fs_type [ext3] 0x0
(0xc2bf9ef4)
Oct 19 03:17:36 mav-server4 kernel: [<c016c741>] do_kern_mount [kernel] 0x121
(0xc2bf9efc)
Oct 19 03:17:36 mav-server4 kernel: [<f887fa34>] ext3_fs_type [ext3] 0x0
(0xc2bf9f00)
Oct 19 03:17:36 mav-server4 kernel: [<c0184c73>] do_add_mount [kernel] 0x93
(0xc2bf9f20)
Oct 19 03:17:36 mav-server4 kernel: [<c0184fa0>] do_mount [kernel] 0x160
(0xc2bf9f40)
Oct 19 03:17:36 mav-server4 kernel: [<c0184df1>] copy_mount_options [kernel]
0x81 (0xc2bf9f70)
Oct 19 03:17:36 mav-server4 kernel: [<c018548f>] sys_mount [kernel] 0xdf
(0xc2bf9f90)
Oct 19 03:17:36 mav-server4 kernel: 
Oct 19 03:17:36 mav-server4 kernel: Code: 0f 0b 4c 05 ee 40 86 f8 e9 0d ff ff ff
8d b4 26 00 00 00 00
Oct 19 03:17:36 mav-server4 kernel: 
Oct 19 03:17:36 mav-server4 kernel: Kernel panic: Fatal exception
Oct 19 03:17:36 mav-server4 kernel: lvm - lvm_map: ll_rw_blk write for readonly
LV /dev/Volume00/Snapshot00
Oct 19 03:17:36 mav-server4 last message repeated 9 times

Impact is that backups can't be done, any sync command hangs and any attempt to
remove the volume group fails until the machine is rebooted.
Comment 1 Christopher Blizzard 2005-10-30 14:44:58 EST
I can verify that this does not happen with this version of the kernel-smp package:

kernel-smp-2.4.21-32.0.1.EL

so this is probably a regression.
Comment 8 Ernie Petrides 2006-05-11 14:27:38 EDT
Closing based on last comment.
Comment 9 mike ledoux 2007-03-28 16:06:07 EDT
I'm seeing this same problem with kernel-smp-2.4.21-47.0.1.EL and
kernel-smp-2.4.21-37.0.1.EL on my production mail servers.  The mount never
returns (stuck in disk-wait), so I can't run backups, remove the lvm snapshot, etc.

Note You need to log in before you can comment on or make changes to this bug.