209582 – lock_dlm Assertion failed

Bug 209582 - lock_dlm Assertion failed

Summary: lock_dlm Assertion failed

Keywords:
Status:	CLOSED DUPLICATE of bug 199673
Alias:	None
Product:	Red Hat Cluster Suite
Classification:	Retired
Component:	gfs
Sub Component:
Version:	4
Hardware:	i686
OS:	Linux
Priority:	medium
Severity:	medium
Target Milestone:	---
Assignee:	David Teigland
QA Contact:	GFS Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2006-10-06 06:34 UTC by Chris Verhoef
Modified:	2010-01-12 03:13 UTC (History)
CC List:	1 user (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2006-11-29 22:25:32 UTC
Embargoed:

Attachments	(Terms of Use)

Description Chris Verhoef 2006-10-06 06:34:34 UTC

Description of problem:

Oct  6 05:37:56 bhnde01 kernel: kernel BUG at
/usr/src/build/765945-i686/BUILD/gfs-kernel-2.6.9-58/smp/src/dlm/lock.c:428!
Oct  6 05:37:56 bhnde01 kernel: invalid operand: 0000 [#1]
Oct  6 05:37:56 bhnde01 kernel: SMP
Oct  6 05:37:56 bhnde01 kernel: Modules linked in: cdi_dr(U) nfsd exportfs lockd
nfs_acl parport_pc lp parport autofs4 lock_dlm(U) gfs(U) lock_harness(U) dlm(U)
cman(U) md5 ipv6 sunrpc dm_mirror dm_mod ohci_hcd cpqphp tg3 bonding(U) floppy
ext3 jbd qla6312(U) qla2300(U) qla2xxx(U) qla2xxx_conf(U) cciss sd_mod scsi_mod
Oct  6 05:37:56 bhnde01 kernel: CPU:    1
Oct  6 05:37:56 bhnde01 kernel: EIP:    0060:[<f8bd0779>]    Tainted: P      VLI
Oct  6 05:37:56 bhnde01 kernel: EFLAGS: 00010246   (2.6.9-42.0.2.ELsmp)
Oct  6 05:37:56 bhnde01 kernel: EIP is at do_dlm_lock+0x134/0x14e [lock_dlm]
Oct  6 05:37:56 bhnde01 kernel: eax: 00000001   ebx: ffffffea   ecx: f35acc24  
edx: f8bd5221
Oct  6 05:37:56 bhnde01 kernel: esi: f8bd0798   edi: f7e3f200   ebp: f0476780  
esp: f35acc20
Oct  6 05:37:56 bhnde01 kernel: ds: 007b   es: 007b   ss: 0068
Oct  6 05:37:56 bhnde01 kernel: Process vsftpd (pid: 30746, threadinfo=f35ac000
task=f5be4030)
Oct  6 05:37:56 bhnde01 kernel: Stack: f8bd5221 20202020 32202020 20202020
20202020 31613120 35333234 32200018
Oct  6 05:37:56 bhnde01 kernel:        00000246 f0476780 00000001 00000003
f0476780 f8bd0828 00000005 f8bd8940
Oct  6 05:37:56 bhnde01 kernel:        f8cde000 f8c3d936 00000000 00000001
d9db9110 d9db90f4 f8cde000 f8c338fe
Oct  6 05:37:56 bhnde01 kernel: Call Trace:
Oct  6 05:37:56 bhnde01 kernel:  [<f8bd0828>] lm_dlm_lock+0x49/0x52 [lock_dlm]
Oct  6 05:37:56 bhnde01 kernel:  [<f8c3d936>] gfs_lm_lock+0x35/0x4d [gfs]
Oct  6 05:37:56 bhnde01 kernel:  [<f8c338fe>] gfs_glock_xmote_th+0x130/0x172 [gfs]
Oct  6 05:37:56 bhnde01 kernel:  [<f8c32fbd>] rq_promote+0xc8/0x147 [gfs]
Oct  6 05:37:56 bhnde01 kernel:  [<f8c331a9>] run_queue+0x91/0xc1 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c341b9>] gfs_glock_nq+0xcf/0x116 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c348f5>] nq_m_sync+0x44/0x64 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c34a5e>] gfs_glock_nq_m+0x149/0x15d [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c4bc52>] gfs_rename+0x166/0x543 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c2a4ec>] gfs_get_meta_buffer+0xb1/0x250 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<c02d2d6a>] __cond_resched+0x14/0x39
Oct  6 05:37:57 bhnde01 kernel:  [<c02d27e2>] wait_for_completion+0xc3/0xcb
Oct  6 05:37:57 bhnde01 kernel:  [<f8c36e9f>] gfs_copyin_dinode+0xf4/0x168 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8de42db>] cd_dcache_to_fn+0x11e/0x12c [cdi_dr]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c4baec>] gfs_rename+0x0/0x543 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8de639c>] cd_rename+0x127/0x22e [cdi_dr]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c4baec>] gfs_rename+0x0/0x543 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c4baec>] gfs_rename+0x0/0x543 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<c01694bb>] vfs_rename_other+0xac/0x125
Oct  6 05:37:57 bhnde01 kernel:  [<c016981f>] vfs_rename+0x2eb/0x338
Oct  6 05:37:57 bhnde01 kernel:  [<c01699b9>] sys_rename+0x14d/0x1e0
Oct  6 05:37:57 bhnde01 kernel:  [<c0143fa4>] buffered_rmqueue+0x17d/0x1a5
Oct  6 05:37:57 bhnde01 kernel:  [<c01707b2>] sys_getcwd+0x1ef/0x1fb
Oct  6 05:37:57 bhnde01 kernel:  [<c02d47bf>] syscall_call+0x7/0xb
Oct  6 05:37:57 bhnde01 kernel: Code: 26 50 0f bf 45 24 50 53 ff 75 08 ff 75 04
ff 75 0c ff 77 18 68 4c 53 bd f8 e8 32 21 55 c7 83 c4 38 68 21 52 bd f8 e8 25 21
55 c7 <0f> 0b ac 01 5e 51 bd f8 68 23 52 bd f8 e8 e0 18 55 c7 83 c4 20


Version-Release number of selected component (if applicable):
GFS-6.1.6-1
GFS-kernel-smp-2.6.9-58.3
GFS-kernel-2.6.9-58.3
kernel-smp-2.6.9-42.0.2.EL
cman-kernel-smp-2.6.9-45.5
dlm-kernel-smp-2.6.9-42.13

How reproducible:
Don't now. First time seen.

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:
We are running a two node cluster, which exports several GFS filesystems with NFS.

Sytem log of the two cluster nodes
First node:
Oct  6 05:37:52 bhnde01 kernel: s_test move use event 110
Oct  6 05:37:52 bhnde01 kernel: gfs_test recover event 110
Oct  6 05:37:52 bhnde01 kernel: gfs_test add node 1
Oct  6 05:37:52 bhnde01 kernel: gfs_test total nodes 2
Oct  6 05:37:52 bhnde01 kernel: gfs_test rebuild resource directory Oct  6
05:37:52 bhnde01 kernel: gfs_test rebuilt 140 resources
Oct  6 05:37:52 bhnde01 kernel: gfs_test purge requests
Oct  6 05:37:52 bhnde01 kernel: gfs_test purged 0 requests
Oct  6 05:37:52 bhnde01 kernel: gfs_test mark waiting requests
Oct  6 05:37:52 bhnde01 kernel: gfs_test marked 0 requests
Oct  6 05:37:52 bhnde01 kernel: gfs_test recover event 110 done
Oct  6 05:37:52 bhnde01 kernel: gfs_test move flags 0,0,1 ids 79,110,110
Oct  6 05:37:52 bhnde01 kernel: gfs_test process held requests
Oct  6 05:37:52 bhnde01 kernel: gfs_test processed 0 requests
Oct  6 05:37:52 bhnde01 kernel: gfs_test resend marked requests
Oct  6 05:37:52 bhnde01 kernel: gfs_test resent 0 requests
Oct  6 05:37:52 bhnde01 kernel: gfs_test recover event 110 finished
Oct  6 05:37:52 bhnde01 kernel: Magma move flags 1,0,0 ids 77,77,77
Oct  6 05:37:52 bhnde01 kernel: Magma move flags 0,1,0 ids 77,113,77
Oct  6 05:37:52 bhnde01 kernel: Magma move use event 113
Oct  6 05:37:52 bhnde01 kernel: Magma recover event 113
Oct  6 05:37:52 bhnde01 kernel: Magma add node 1
Oct  6 05:37:52 bhnde01 kernel: Magma total nodes 2
Oct  6 05:37:52 bhnde01 kernel: Magma rebuild resource directory
Oct  6 05:37:52 bhnde01 kernel: Magma rebuilt 2 resources
Oct  6 05:37:52 bhnde01 kernel: Magma purge requests
Oct  6 05:37:53 bhnde01 kernel: Magma purged 0 requests
Oct  6 05:37:53 bhnde01 kernel: Magma mark waiting requests
Oct  6 05:37:53 bhnde01 kernel: Magma marked 0 requests
Oct  6 05:37:53 bhnde01 kernel: Magma recover event 113 done
Oct  6 05:37:53 bhnde01 kernel: Magma move flags 0,0,1 ids 77,113,113
Oct  6 05:37:53 bhnde01 kernel: Magma process held requests
Oct  6 05:37:53 bhnde01 kernel: Magma processed 0 requests
Oct  6 05:37:53 bhnde01 kernel: Magma resend marked requests
Oct  6 05:37:53 bhnde01 kernel: Magma resent 0 requests
Oct  6 05:37:53 bhnde01 kernel: Magma recover event 113 finished
Oct  6 05:37:53 bhnde01 kernel: 746 remove 7,1a14380
Oct  6 05:37:53 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en plock 7,1a14381
Oct  6 05:37:53 bhnde01 kernel: 30746 req 7,1a14381 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:53 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en punlock 7,1a14381
Oct  6 05:37:53 bhnde01 kernel: 30746 remove 7,1a14381
Oct  6 05:37:53 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en plock 7,1a14382
Oct  6 05:37:53 bhnde01 kernel: 30746 req 7,1a14382 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:53 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en punlock 7,1a14382
Oct  6 05:37:53 bhnde01 kernel: 30746 remove 7,1a14382
Oct  6 05:37:53 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en plock 7,1a14383
Oct  6 05:37:53 bhnde01 kernel: 30746 req 7,1a14383 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:53 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en punlock 7,1a14383
Oct  6 05:37:53 bhnde01 kernel: 30746 remove 7,1a14383
Oct  6 05:37:53 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en plock 7,1a14384
Oct  6 05:37:53 bhnde01 kernel: 30746 req 7,1a14384 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:53 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en punlock 7,1a14384
Oct  6 05:37:53 bhnde01 kernel: 30746 remove 7,1a14384
Oct  6 05:37:53 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en plock 7,1a14385
Oct  6 05:37:53 bhnde01 kernel: 30746 req 7,1a14385 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:53 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en punlock 7,1a14385
Oct  6 05:37:53 bhnde01 kernel: 30746 remove 7,1a14385
Oct  6 05:37:53 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en plock 7,1a14386
Oct  6 05:37:53 bhnde01 kernel: 30746 req 7,1a14386 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:53 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en punlock 7,1a14386
Oct  6 05:37:53 bhnde01 kernel: 30746 remove 7,1a14386
Oct  6 05:37:53 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:53 bhnde01 kernel: 30746 en plock 7,1a14387
Oct  6 05:37:53 bhnde01 kernel: 30746 req 7,1a14387 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:54 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en punlock 7,1a14387
Oct  6 05:37:54 bhnde01 kernel: 30746 remove 7,1a14387
Oct  6 05:37:54 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en plock 7,1a14388
Oct  6 05:37:54 bhnde01 kernel: 30746 req 7,1a14388 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:54 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en punlock 7,1a14388
Oct  6 05:37:54 bhnde01 kernel: 30746 remove 7,1a14388
Oct  6 05:37:54 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en plock 7,1a14389
Oct  6 05:37:54 bhnde01 kernel: 30746 req 7,1a14389 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:54 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en punlock 7,1a14389
Oct  6 05:37:54 bhnde01 kernel: 30746 remove 7,1a14389
Oct  6 05:37:54 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en plock 7,1a1438a
Oct  6 05:37:54 bhnde01 kernel: 30746 req 7,1a1438a ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:54 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en punlock 7,1a1438a
Oct  6 05:37:54 bhnde01 kernel: 30746 remove 7,1a1438a
Oct  6 05:37:54 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en plock 7,1a1438b
Oct  6 05:37:54 bhnde01 kernel: 30746 req 7,1a1438b ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:54 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en punlock 7,1a1438b
Oct  6 05:37:54 bhnde01 kernel: 30746 remove 7,1a1438b
Oct  6 05:37:54 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en plock 7,1a1438c
Oct  6 05:37:54 bhnde01 kernel: 30746 req 7,1a1438c ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:54 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en punlock 7,1a1438c
Oct  6 05:37:54 bhnde01 kernel: 30746 remove 7,1a1438c
Oct  6 05:37:54 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en plock 7,1a1438d
Oct  6 05:37:54 bhnde01 kernel: 30746 req 7,1a1438d ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:54 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en punlock 7,1a1438d
Oct  6 05:37:54 bhnde01 kernel: 30746 remove 7,1a1438d
Oct  6 05:37:54 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en plock 7,1a1438e
Oct  6 05:37:54 bhnde01 kernel: 30746 req 7,1a1438e ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:54 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:54 bhnde01 kernel: 30746 en punlock 7,1a1438e
Oct  6 05:37:55 bhnde01 kernel: 30746 remove 7,1a1438e
Oct  6 05:37:55 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en plock 7,1a1438f
Oct  6 05:37:55 bhnde01 kernel: 30746 req 7,1a1438f ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:55 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en punlock 7,1a1438f
Oct  6 05:37:55 bhnde01 kernel: 30746 remove 7,1a1438f
Oct  6 05:37:55 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en plock 7,1a14390
Oct  6 05:37:55 bhnde01 kernel: 30746 req 7,1a14390 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:55 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en punlock 7,1a14390
Oct  6 05:37:55 bhnde01 kernel: 30746 remove 7,1a14390
Oct  6 05:37:55 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en plock 7,1a14391
Oct  6 05:37:55 bhnde01 kernel: 30746 req 7,1a14391 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:55 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en punlock 7,1a14391
Oct  6 05:37:55 bhnde01 kernel: 30746 remove 7,1a14391
Oct  6 05:37:55 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en plock 7,1a14392
Oct  6 05:37:55 bhnde01 kernel: 30746 req 7,1a14392 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:55 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en punlock 7,1a14392
Oct  6 05:37:55 bhnde01 kernel: 30746 remove 7,1a14392
Oct  6 05:37:55 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en plock 7,1a14394
Oct  6 05:37:55 bhnde01 kernel: 30746 req 7,1a14394 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:55 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en punlock 7,1a14394
Oct  6 05:37:55 bhnde01 kernel: 30746 remove 7,1a14394
Oct  6 05:37:55 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en plock 7,1a14395
Oct  6 05:37:55 bhnde01 kernel: 30746 req 7,1a14395 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:55 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en punlock 7,1a14395
Oct  6 05:37:55 bhnde01 kernel: 30746 remove 7,1a14395
Oct  6 05:37:55 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en plock 7,1a14396
Oct  6 05:37:55 bhnde01 kernel: 30746 req 7,1a14396 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:55 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en punlock 7,1a14396
Oct  6 05:37:55 bhnde01 kernel: 30746 remove 7,1a14396
Oct  6 05:37:55 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:55 bhnde01 kernel: 30746 en plock 7,1a14397
Oct  6 05:37:56 bhnde01 kernel: 30746 req 7,1a14397 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:56 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:56 bhnde01 kernel: 30746 en punlock 7,1a14397
Oct  6 05:37:56 bhnde01 kernel: 30746 remove 7,1a14397
Oct  6 05:37:56 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:56 bhnde01 kernel: 30746 en plock 7,1a14398
Oct  6 05:37:56 bhnde01 kernel: 30746 req 7,1a14398 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:56 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:56 bhnde01 kernel: 30746 en punlock 7,1a14398
Oct  6 05:37:56 bhnde01 kernel: 30746 remove 7,1a14398
Oct  6 05:37:56 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:56 bhnde01 kernel: 30746 en plock 7,1a14399
Oct  6 05:37:56 bhnde01 kernel: 30746 req 7,1a14399 ex 0-7fffffffffffffff lkf
2000 wait 1
Oct  6 05:37:56 bhnde01 kernel: 30746 ex plock 0
Oct  6 05:37:56 bhnde01 kernel: 30746 en punlock 7,1a14399
Oct  6 05:37:56 bhnde01 kernel: 30746 remove 7,1a14399
Oct  6 05:37:56 bhnde01 kernel: 30746 ex punlock 0
Oct  6 05:37:56 bhnde01 kernel:
Oct  6 05:37:56 bhnde01 kernel: lock_dlm:  Assertion failed on line 428 of file
/usr/src/build/765945-i686/BUILD/gfs-kernel-2.6.9-58/smp/src/dlm/lock.c
Oct  6 05:37:56 bhnde01 kernel: lock_dlm:  assertion:  "!error"
Oct  6 05:37:56 bhnde01 kernel: lock_dlm:  time = 3257107118
Oct  6 05:37:56 bhnde01 kernel: gfs_www_data: num=2,1a14235 err=-22 cur=3 req=5
lkf=44
Oct  6 05:37:56 bhnde01 kernel:
Oct  6 05:37:56 bhnde01 kernel: ------------[ cut here ]------------
Oct  6 05:37:56 bhnde01 kernel: kernel BUG at
/usr/src/build/765945-i686/BUILD/gfs-kernel-2.6.9-58/smp/src/dlm/lock.c:428!
Oct  6 05:37:56 bhnde01 kernel: invalid operand: 0000 [#1]
Oct  6 05:37:56 bhnde01 kernel: SMP
Oct  6 05:37:56 bhnde01 kernel: Modules linked in: cdi_dr(U) nfsd exportfs lockd
nfs_acl parport_pc lp parport autofs4 lock_dlm(U) gfs(U) lock_harness(U) dlm(U)
cman(U) md5 ipv6 sunrpc dm_mirror dm_mod ohci_hcd cpqphp tg3 bonding(U) floppy
ext3 jbd qla6312(U) qla2300(U) qla2xxx(U) qla2xxx_conf(U) cciss sd_mod scsi_mod
Oct  6 05:37:56 bhnde01 kernel: CPU:    1
Oct  6 05:37:56 bhnde01 kernel: EIP:    0060:[<f8bd0779>]    Tainted: P      VLI
Oct  6 05:37:56 bhnde01 kernel: EFLAGS: 00010246   (2.6.9-42.0.2.ELsmp)
Oct  6 05:37:56 bhnde01 kernel: EIP is at do_dlm_lock+0x134/0x14e [lock_dlm]
Oct  6 05:37:56 bhnde01 kernel: eax: 00000001   ebx: ffffffea   ecx: f35acc24  
edx: f8bd5221
Oct  6 05:37:56 bhnde01 kernel: esi: f8bd0798   edi: f7e3f200   ebp: f0476780  
esp: f35acc20
Oct  6 05:37:56 bhnde01 kernel: ds: 007b   es: 007b   ss: 0068
Oct  6 05:37:56 bhnde01 kernel: Process vsftpd (pid: 30746, threadinfo=f35ac000
task=f5be4030)
Oct  6 05:37:56 bhnde01 kernel: Stack: f8bd5221 20202020 32202020 20202020
20202020 31613120 35333234 32200018
Oct  6 05:37:56 bhnde01 kernel:        00000246 f0476780 00000001 00000003
f0476780 f8bd0828 00000005 f8bd8940
Oct  6 05:37:56 bhnde01 kernel:        f8cde000 f8c3d936 00000000 00000001
d9db9110 d9db90f4 f8cde000 f8c338fe
Oct  6 05:37:56 bhnde01 kernel: Call Trace:
Oct  6 05:37:56 bhnde01 kernel:  [<f8bd0828>] lm_dlm_lock+0x49/0x52 [lock_dlm]
Oct  6 05:37:56 bhnde01 kernel:  [<f8c3d936>] gfs_lm_lock+0x35/0x4d [gfs]
Oct  6 05:37:56 bhnde01 kernel:  [<f8c338fe>] gfs_glock_xmote_th+0x130/0x172 [gfs]
Oct  6 05:37:56 bhnde01 kernel:  [<f8c32fbd>] rq_promote+0xc8/0x147 [gfs]
Oct  6 05:37:56 bhnde01 kernel:  [<f8c331a9>] run_queue+0x91/0xc1 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c341b9>] gfs_glock_nq+0xcf/0x116 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c348f5>] nq_m_sync+0x44/0x64 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c34a5e>] gfs_glock_nq_m+0x149/0x15d [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c4bc52>] gfs_rename+0x166/0x543 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c2a4ec>] gfs_get_meta_buffer+0xb1/0x250 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<c02d2d6a>] __cond_resched+0x14/0x39
Oct  6 05:37:57 bhnde01 kernel:  [<c02d27e2>] wait_for_completion+0xc3/0xcb
Oct  6 05:37:57 bhnde01 kernel:  [<f8c36e9f>] gfs_copyin_dinode+0xf4/0x168 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8de42db>] cd_dcache_to_fn+0x11e/0x12c [cdi_dr]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c4baec>] gfs_rename+0x0/0x543 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8de639c>] cd_rename+0x127/0x22e [cdi_dr]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c4baec>] gfs_rename+0x0/0x543 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<f8c4baec>] gfs_rename+0x0/0x543 [gfs]
Oct  6 05:37:57 bhnde01 kernel:  [<c01694bb>] vfs_rename_other+0xac/0x125
Oct  6 05:37:57 bhnde01 kernel:  [<c016981f>] vfs_rename+0x2eb/0x338
Oct  6 05:37:57 bhnde01 kernel:  [<c01699b9>] sys_rename+0x14d/0x1e0
Oct  6 05:37:57 bhnde01 kernel:  [<c0143fa4>] buffered_rmqueue+0x17d/0x1a5
Oct  6 05:37:57 bhnde01 kernel:  [<c01707b2>] sys_getcwd+0x1ef/0x1fb
Oct  6 05:37:57 bhnde01 kernel:  [<c02d47bf>] syscall_call+0x7/0xb
Oct  6 05:37:57 bhnde01 kernel: Code: 26 50 0f bf 45 24 50 53 ff 75 08 ff 75 04
ff 75 0c ff 77 18 68 4c 53 bd f8 e8 32 21 55 c7 83 c4 38 68 21 52 bd f8 e8 25 21
55 c7 <0f> 0b ac 01 5e 51 bd f8 68 23 52 bd f8 e8 e0 18 55 c7 83 c4 20
Oct  6 05:41:32 bhnde01 syslogd 1.4.1: restart.


Second node:
Oct  6 05:37:01 bhnde02 clurgmgrd: [3540]: <info> Executing
/etc/init.d/mysql.server status
Oct  6 05:37:09 bhnde02 sshd(pam_unix)[11004]: session opened for user root by
(uid=0)
Oct  6 05:37:32 bhnde02 clurgmgrd: [3540]: <info> Executing
/etc/init.d/mysql.server status
Oct  6 05:38:17 bhnde02 kernel: CMAN: removing node bhnde01 from the cluster :
Missed too many heartbeats
Oct  6 05:38:17 bhnde02 fenced[3033]: bhnde01 not a cluster member after 0 sec
post_fail_delay
Oct  6 05:38:17 bhnde02 fenced[3033]: fencing node "bhnde01"
Oct  6 05:39:29 bhnde02 fenced[3033]: fence "bhnde01" success

Comment 1 David Teigland 2006-10-06 14:12:29 UTC

My first guess is that this is a duplicate of bug 199673.

Comment 3 David Teigland 2006-10-17 16:38:11 UTC

Please report if this is still a problem with the latest 4.4 errata release.

Comment 4 Chris Verhoef 2006-10-18 09:35:48 UTC

The KNMI will try it with the latest errata for 4.4.
They will inform you about the results

Comment 5 Kiersten (Kerri) Anderson 2006-11-10 16:26:15 UTC

Any updates on this one?

Comment 6 lunix4knmi 2006-11-13 09:09:35 UTC

Haven't had the time to do the update yet. Will probably happen in the next 2 weeks.

Comment 7 lunix4knmi 2006-11-23 10:54:15 UTC

Updates seem to have fixed the problem.

Comment 8 David Teigland 2006-11-29 22:25:32 UTC


*** This bug has been marked as a duplicate of 199673 ***

Note You need to log in before you can comment on or make changes to this bug.