Attachment 913001 Details for Bug 1114185 – Related log parts from two nodes.

Related log parts from two nodes.

gfs2_kernel_bug.txt (text/plain), 11.98 KB, created by Milos Jakubicek on 2014-06-28 09:47:53 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Milos Jakubicek

Created: 2014-06-28 09:47:53 UTC

Size: 11.98 KB

patch

obsolete

>Jun 26 14:31:10 node4 kernel: BUG: unable to handle kernel paging request at 0000000400000849
>Jun 26 14:31:10 node4 kernel: IP: [<ffffffff816f756e>] _raw_spin_lock+0xe/0x40
>Jun 26 14:31:10 node4 kernel: PGD 0 
>Jun 26 14:31:10 node4 kernel: Oops: 0002 [#1] SMP 
>Jun 26 14:31:10 node4 kernel: Modules linked in: gfs2 dlm sctp libcrc32c cfg80211 rfkill vfat fat joydev tpm_tis x86_pkg_temp_thermal coretemp kvm_intel kvm cdc_ether usbnet mii iTCO_wdt iTCO_vendor_support be2net ipmi_devintf gpio_ich shpchp wmi ipmi_si ipmi_msghandler tpm ioatdma crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel dca lpc_ich mfd_core i2c_i801 sb_edac microcode edac_core nfsd auth_rpcgss nfs_acl lockd sunrpc dm_service_time scsi_dh_rdac mgag200 i2c_algo_bit drm_kms_helper ttm drm mpt2sas i2c_core raid_class scsi_transport_sas dm_multipath
>Jun 26 14:31:10 node4 kernel: CPU: 1 PID: 16 Comm: kworker/1:0H Not tainted 3.14.8-200.fc20.x86_64 #1
>Jun 26 14:31:10 node4 kernel: Hardware name: IBM BladeCenter HS23 -[7875T79]-/00FG950, BIOS -[TKE136VUS-1.51]- 01/20/2014
>Jun 26 14:31:10 node4 kernel: Workqueue: glock_workqueue glock_work_func [gfs2]
>Jun 26 14:31:10 node4 kernel: task: ffff8810297f4280 ti: ffff88102910e000 task.ti: ffff88102910e000
>Jun 26 14:31:10 node4 kernel: RIP: 0010:[<ffffffff816f756e>]  [<ffffffff816f756e>] _raw_spin_lock+0xe/0x40
>Jun 26 14:31:10 node4 kernel: RSP: 0018:ffff88102910fb88  EFLAGS: 00010202
>Jun 26 14:31:10 node4 kernel: RAX: 0000000000010000 RBX: ffff880ecf655a28 RCX: 005ffff800000065
>Jun 26 14:31:10 node4 kernel: RDX: 005ffff800000865 RSI: 0000000000000000 RDI: 0000000400000849
>Jun 26 14:31:10 node4 kernel: RBP: ffff88102910fb88 R08: 5010000000000000 R09: 0ecf655a28080000
>Jun 26 14:31:10 node4 kernel: R10: f1129e59b14e8a02 R11: 0000000000000210 R12: 0000000400000001
>Jun 26 14:31:10 node4 kernel: R13: 0000000400000849 R14: ffffea003aa7b140 R15: 0000000400000971
>Jun 26 14:31:10 node4 kernel: FS:  0000000000000000(0000) GS:ffff88103fc20000(0000) knlGS:0000000000000000
>Jun 26 14:31:10 node4 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>Jun 26 14:31:10 node4 kernel: CR2: 0000000400000849 CR3: 0000000001c0c000 CR4: 00000000001407e0
>Jun 26 14:31:10 node4 kernel: Stack:
>Jun 26 14:31:10 node4 kernel:  ffff88102910fbd0 ffffffffa04a8320 ffff880ecf655a28 ffff880f00000002
>Jun 26 14:31:10 node4 kernel:  ffff880ecf655a28 0000000000000000 0000000000001000 ffff880ecf655a28
>Jun 26 14:31:10 node4 kernel:  0000000000001000 ffff88102910fbe0 ffffffff811749d2 ffff88102910fc28
>Jun 26 14:31:10 node4 kernel: Call Trace:
>Jun 26 14:31:10 node4 kernel:  [<ffffffffa04a8320>] gfs2_releasepage+0x70/0x1f0 [gfs2]
>Jun 26 14:31:10 node4 kernel:  [<ffffffff811749d2>] try_to_release_page+0x32/0x50
>Jun 26 14:31:10 node4 kernel:  [<ffffffff8121e4bb>] block_invalidatepage+0x10b/0x120
>Jun 26 14:31:10 node4 kernel:  [<ffffffff811848ab>] truncate_inode_page+0x8b/0x90
>Jun 26 14:31:10 node4 kernel:  [<ffffffff81184a71>] truncate_inode_pages_range+0x181/0x5c0
>Jun 26 14:31:10 node4 kernel:  [<ffffffffa04a3972>] rgrp_go_inval+0x42/0xb0 [gfs2]
>Jun 26 14:31:10 node4 kernel:  [<ffffffffa04a241a>] do_xmote+0x15a/0x270 [gfs2]
>Jun 26 14:31:10 node4 kernel:  [<ffffffff8101ceb5>] ? native_sched_clock+0x35/0x90
>Jun 26 14:31:10 node4 kernel:  [<ffffffffa04a266d>] run_queue+0x13d/0x280 [gfs2]
>Jun 26 14:31:10 node4 kernel:  [<ffffffffa04a280b>] glock_work_func+0x5b/0x140 [gfs2]
>Jun 26 14:31:10 node4 kernel:  [<ffffffff810a68e6>] process_one_work+0x176/0x430
>Jun 26 14:31:10 node4 kernel:  [<ffffffff810a757b>] worker_thread+0x11b/0x3a0
>Jun 26 14:31:10 node4 kernel:  [<ffffffff810a7460>] ? rescuer_thread+0x3b0/0x3b0
>Jun 26 14:31:10 node4 kernel:  [<ffffffff810ae2d1>] kthread+0xe1/0x100
>Jun 26 14:31:10 node4 kernel:  [<ffffffff810ae1f0>] ? insert_kthread_work+0x40/0x40
>Jun 26 14:31:10 node4 kernel:  [<ffffffff8170083c>] ret_from_fork+0x7c/0xb0
>Jun 26 14:31:10 node4 kernel:  [<ffffffff810ae1f0>] ? insert_kthread_work+0x40/0x40
>Jun 26 14:31:10 node4 kernel: Code: fa 66 0f 1f 44 00 00 f0 81 2f 00 00 10 00 74 05 e8 38 b0 c6 ff 48 89 d0 5d c3 0f 1f 00 0f 1f 44 00 00 55 48 89 e5 b8 00 00 01 00 <f0> 0f c1 07 89 c2 c1 ea 10 66 39 c2 75 02 5d c3 0f b7 07 89 d1 
>Jun 26 14:31:10 node4 kernel: RIP  [<ffffffff816f756e>] _raw_spin_lock+0xe/0x40
>Jun 26 14:31:10 node4 kernel:  RSP <ffff88102910fb88>
>Jun 26 14:31:10 node4 kernel: CR2: 0000000400000849
>Jun 26 14:31:10 node4 kernel: ---[ end trace 9c34dd901886dacf ]---
>Jun 26 14:31:10 node4 kernel: BUG: unable to handle kernel paging request at ffffffffffffffd8
>Jun 26 14:31:10 node4 kernel: IP: [<ffffffff810ae8c0>] kthread_data+0x10/0x20
>Jun 26 14:31:10 node4 kernel: PGD 1c0f067 PUD 1c11067 PMD 0 
>Jun 26 14:31:10 node4 kernel: Oops: 0000 [#2] SMP 
>Jun 26 14:31:10 node4 kernel: Modules linked in: gfs2 dlm sctp libcrc32c cfg80211 rfkill vfat fat joydev tpm_tis x86_pkg_temp_thermal coretemp kvm_intel kvm cdc_ether usbnet mii iTCO_wdt iTCO_vendor_support be2net ipmi_devintf gpio_ich shpchp wmi ipmi_si ipmi_msghandler tpm ioatdma crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel dca lpc_ich mfd_core i2c_i801 sb_edac microcode edac_core nfsd auth_rpcgss nfs_acl lockd sunrpc dm_service_time scsi_dh_rdac mgag200 i2c_algo_bit drm_kms_helper ttm drm mpt2sas i2c_core raid_class scsi_transport_sas dm_multipath
>Jun 26 14:31:10 node4 kernel: CPU: 1 PID: 16 Comm: kworker/1:0H Tainted: G      D      3.14.8-200.fc20.x86_64 #1
>Jun 26 14:31:10 node4 kernel: Hardware name: IBM BladeCenter HS23 -[7875T79]-/00FG950, BIOS -[TKE136VUS-1.51]- 01/20/2014
>Jun 26 14:31:10 node4 kernel: task: ffff8810297f4280 ti: ffff88102910e000 task.ti: ffff88102910e000
>Jun 26 14:31:10 node4 kernel: RIP: 0010:[<ffffffff810ae8c0>]  [<ffffffff810ae8c0>] kthread_data+0x10/0x20
>Jun 26 14:31:10 node4 kernel: RSP: 0018:ffff88102910f7b8  EFLAGS: 00010002
>Jun 26 14:31:10 node4 kernel: RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000006
>Jun 26 14:31:10 node4 kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8810297f4280
>Jun 26 14:31:10 node4 kernel: RBP: ffff88102910f7b8 R08: 000000000000b800 R09: ffff88103fc391d0
>Jun 26 14:31:10 node4 kernel: R10: ffffffff813310ca R11: ffffea004094b9c0 R12: ffff88103fc34640
>Jun 26 14:31:10 node4 kernel: R13: 0000000000000001 R14: ffff8810297f4270 R15: ffff8810297f4280
>Jun 26 14:31:10 node4 kernel: FS:  0000000000000000(0000) GS:ffff88103fc20000(0000) knlGS:0000000000000000
>Jun 26 14:31:10 node4 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>Jun 26 14:31:10 node4 kernel: CR2: 0000000000000028 CR3: 0000000001c0c000 CR4: 00000000001407e0
>Jun 26 14:31:10 node4 kernel: Stack:
>Jun 26 14:31:10 node4 kernel:  ffff88102910f7d0 ffffffff810a7c31 ffff8810297f4280 ffff88102910f830
>Jun 26 14:31:10 node4 kernel:  ffffffff816f3ee9 ffff8810297f4280 ffff88102910ffd8 0000000000014640
>Jun 26 14:31:10 node4 kernel:  0000000000014640 ffff8810297f4280 ffff8810297f4918 ffff88102910f508
>Jun 26 14:31:10 node4 kernel: Call Trace:
>Jun 26 14:31:10 node4 kernel:  [<ffffffff810a7c31>] wq_worker_sleeping+0x11/0x90
>Jun 26 14:31:10 node4 kernel:  [<ffffffff816f3ee9>] __schedule+0x4a9/0x740
>Jun 26 14:31:10 node4 kernel:  [<ffffffff816f41a9>] schedule+0x29/0x70
>Jun 26 14:31:10 node4 kernel:  [<ffffffff8108cb67>] do_exit+0x6b7/0xa30
>Jun 26 14:31:10 node4 kernel:  [<ffffffff810e30d8>] ? console_unlock+0x1e8/0x3f0
>Jun 26 14:31:10 node4 kernel:  [<ffffffff816f890c>] oops_end+0x9c/0xe0
>Jun 26 14:31:10 node4 kernel:  [<ffffffff816ebacd>] no_context+0x28d/0x2b0
>Jun 26 14:31:10 node4 kernel:  [<ffffffff816ebb63>] __bad_area_nosemaphore+0x73/0x1ca
>
>
>Jun 26 14:40:45 node1 kernel: BUG: unable to handle kernel paging request at 0000000400000849
>
>Jun 26 15:06:59 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 1177534796, entries2 (50) != g.offset (48)
>
>
>Jun 26 15:34:18 node1 kernel:  [<ffffffff816fb702>] do_page_fault+0x22/0x30
>Jun 26 15:34:18 node1 kernel:  [<ffffffff816f7d48>] page_fault+0x28/0x30
>Jun 26 15:34:18 node1 kernel: Code: fa 66 66 90 66 66 90 f0 81 2f 00 00 10 00 74 05 e8 38 b0 c6 ff 48 89 d0 5d c3 0f 1f 00 66 66 66 66 90 55 48 89 e5 b8 00 00 01 00 <f0> 0f c1 07 89 c2 c1 ea 10 66 39 c2 75 02 5d c3 0f b7 07 89 d1 
>Jun 26 15:34:18 node1 kernel: RIP  [<ffffffff816f756e>] _raw_spin_lock+0xe/0x40
>Jun 26 15:34:18 node1 kernel:  RSP <ffff881adcd7b6e8>
>Jun 26 15:34:18 node1 kernel: CR2: 0000000400000849
>Jun 26 15:34:18 node1 kernel: ---[ end trace 85c51ff6e5bac85f ]---
>
>
>Jun 26 16:02:50 node1 kernel: GFS2: fsid=ske:corpora2.1: Number of entries corrupt in dir leaf 180847588, entries2 (46) != g.offset (44)
>Jun 26 16:02:50 node1 kernel: GFS2: fsid=ske:corpora2.1: Number of entries corrupt in dir leaf 180847918, entries2 (51) != g.offset (49)
>
>
>Jun 26 16:02:29 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 180847588, entries2 (46) != g.offset (44)
>Jun 26 16:02:37 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 180847918, entries2 (51) != g.offset (49)
>Jun 26 16:02:58 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 180847588, entries2 (46) != g.offset (44)
>Jun 26 16:03:33 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 180847588, entries2 (46) != g.offset (44)
>Jun 26 16:04:15 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 180847918, entries2 (51) != g.offset (49)
>
>
>Jun 26 16:30:37 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (71) != g.offset (67)
>Jun 26 16:30:37 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (71) != g.offset (67)
>Jun 26 16:30:37 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (71) != g.offset (67)
>Jun 26 16:30:38 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (71) != g.offset (67)
>Jun 26 16:30:39 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:39 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:39 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:39 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:39 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:39 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:39 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:39 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:41 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:41 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:41 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:41 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:43 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:43 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:43 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)
>Jun 26 16:30:43 node4 kernel: GFS2: fsid=ske:corpora2.0: Number of entries corrupt in dir leaf 112708171, entries2 (39) != g.offset (35)

Actions: View

Attachments on bug 1114185: 913001