Attachment 294258 Details for Bug 431907 – It a piece of the /var/log/messages file with several examples of the lockup.

It a piece of the /var/log/messages file with several examples of the lockup.

messages (text/plain), 57.39 KB, created by Barry Donahue on 2008-02-07 19:30:27 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Barry Donahue

Created: 2008-02-07 19:30:27 UTC

Size: 57.39 KB

patch

obsolete

>Feb  7 13:18:34 ibm-js21-01 kernel: sd 1:0:0:0: rejecting I/O to device being removed
>Feb  7 13:18:34 ibm-js21-01 kernel: sd 1:0:0:0: rejecting I/O to device being removed
>Feb  7 13:18:34 ibm-js21-01 kernel: ext3_abort called.
>Feb  7 13:18:34 ibm-js21-01 kernel: EXT3-fs error (device sdb1): ext3_journal_start_sb: Detected aborted journal
>Feb  7 13:18:34 ibm-js21-01 kernel: Remounting filesystem read-only
>Feb  7 13:18:34 ibm-js21-01 kernel: EXT3-fs error (device sdb1) in ext3_prepare_write: IO failure
>Feb  7 13:18:34 ibm-js21-01 kernel: sd 1:0:0:0: rejecting I/O to device being removed
>Feb  7 13:18:34 ibm-js21-01 kernel: __journal_remove_journal_head: freeing b_committed_data
>Feb  7 13:18:34 ibm-js21-01 last message repeated 11 times
>Feb  7 13:18:34 ibm-js21-01 kernel: sd 1:0:0:0: rejecting I/O to device being removed
>Feb  7 13:18:34 ibm-js21-01 last message repeated 138 times
>Feb  7 13:18:34 ibm-js21-01 kernel: BUG: soft lockup - CPU#0 stuck for 10s! [events/0:8]
>Feb  7 13:18:34 ibm-js21-01 kernel: NIP: C0000000000AA8D0 LR: C0000000000ACA88 CTR: C0000000000AC988
>Feb  7 13:18:34 ibm-js21-01 kernel: REGS: c00000000fffbbc0 TRAP: 0901   Not tainted  (2.6.18-77.el5)
>Feb  7 13:18:34 ibm-js21-01 kernel: MSR: 8000000000009032 <EE,ME,IR,DR>  CR: 28000022  XER: 00000003
>Feb  7 13:18:34 ibm-js21-01 kernel: TASK = c00000000ff018e0[8] 'events/0' THREAD: c000000007508000 CPU: 0
>Feb  7 13:18:34 ibm-js21-01 kernel: GPR00: 8000000000009032 C00000000FFFBE40 C00000000057D670 000000000000001A 
>Feb  7 13:18:34 ibm-js21-01 kernel: GPR04: C00000000750B980 C00000000107ED80 C00000000FFFBF90 C000000000550118 
>Feb  7 13:18:34 ibm-js21-01 kernel: GPR08: 0000000000000000 0000000000000000 C000000000464E00 C000000000A50A70 
>Feb  7 13:18:34 ibm-js21-01 kernel: GPR12: 80000000F0000000 C000000000464E00 
>Feb  7 13:18:34 ibm-js21-01 kernel: NIP [C0000000000AA8D0] .handle_IRQ_event+0x48/0xf8
>Feb  7 13:18:34 ibm-js21-01 kernel: LR [C0000000000ACA88] .handle_fasteoi_irq+0x100/0x1bc
>Feb  7 13:18:34 ibm-js21-01 kernel: Call Trace:
>Feb  7 13:18:34 ibm-js21-01 kernel: [C00000000FFFBE40] [C0000000000AA940] .handle_IRQ_event+0xb8/0xf8 (unreliable)
>Feb  7 13:18:34 ibm-js21-01 kernel: [C00000000FFFBEF0] [C0000000000ACA88] .handle_fasteoi_irq+0x100/0x1bc
>Feb  7 13:18:34 ibm-js21-01 kernel: [C00000000FFFBF90] [C000000000026CDC] .call_handle_irq+0x1c/0x2c
>Feb  7 13:18:34 ibm-js21-01 kernel: [C00000000750B880] [C00000000000D0A4] .do_IRQ+0xf4/0x1a4
>Feb  7 13:18:34 ibm-js21-01 kernel: [C00000000750B910] [C0000000000044F4] hardware_interrupt_entry+0xc/0x10
>Feb  7 13:18:34 ibm-js21-01 kernel: --- Exception: 501 at .release_console_sem+0x210/0x2cc
>Feb  7 13:18:34 ibm-js21-01 kernel:     LR = .release_console_sem+0x20c/0x2cc
>Feb  7 13:18:34 ibm-js21-01 kernel: [C00000000750BCA0] [C0000000001E4490] .fb_flashcursor+0x384/0x3e0
>Feb  7 13:18:35 ibm-js21-01 kernel: [C00000000750BD50] [C00000000008153C] .run_workqueue+0xdc/0x168
>Feb  7 13:18:35 ibm-js21-01 kernel: [C00000000750BDF0] [C0000000000822A8] .worker_thread+0x128/0x198
>Feb  7 13:18:35 ibm-js21-01 kernel: [C00000000750BEE0] [C000000000086DD4] .kthread+0x128/0x178
>Feb  7 13:18:35 ibm-js21-01 kernel: [C00000000750BF90] [C000000000026FB0] .kernel_thread+0x4c/0x68
>Feb  7 13:18:35 ibm-js21-01 kernel: Instruction dump:
>Feb  7 13:18:35 ibm-js21-01 kernel: fba1ffe8 f8010010 f821ff51 7cbf2b78 7c791b78 7c9a2378 e8050008 7809dfe3 
>Feb  7 13:18:35 ibm-js21-01 kernel: 40820010 7c0000a6 60008000 7c010164 <3b600000> 3ba00000 e93f0000 e89f0028 
>Feb  7 13:18:35 ibm-js21-01 kernel: scsi 1:0:0:0: rejecting I/O to dead device
>Feb  7 13:18:35 ibm-js21-01 kernel: printk: 127435 messages suppressed.
>Feb  7 13:18:35 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 1545
>Feb  7 13:18:35 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:18:35 ibm-js21-01 kernel: scsi2 : iSCSI Initiator over TCP/IP
>Feb  7 13:18:35 ibm-js21-01 kernel: scsi3 : iSCSI Initiator over TCP/IP
>Feb  7 13:18:35 ibm-js21-01 kernel: scsi4 : iSCSI Initiator over TCP/IP
>Feb  7 13:18:35 ibm-js21-01 kernel: scsi5 : iSCSI Initiator over TCP/IP
>Feb  7 13:18:35 ibm-js21-01 kernel: scsi6 : iSCSI Initiator over TCP/IP
>Feb  7 13:18:35 ibm-js21-01 kernel: scsi7 : iSCSI Initiator over TCP/IP
>Feb  7 13:18:35 ibm-js21-01 kernel: scsi8 : iSCSI Initiator over TCP/IP
>Feb  7 13:18:35 ibm-js21-01 kernel: scsi9 : iSCSI Initiator over TCP/IP
>Feb  7 13:18:35 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:18:35 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdb: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:18:35 ibm-js21-01 kernel: sdb: Write Protect is off
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdb: drive cache: write through
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdb: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:18:35 ibm-js21-01 kernel: sdb: Write Protect is off
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdb: drive cache: write through
>Feb  7 13:18:35 ibm-js21-01 kernel:  sdb: sdb1
>Feb  7 13:18:35 ibm-js21-01 kernel: sd 9:0:0:0: Attached scsi disk sdb
>Feb  7 13:18:35 ibm-js21-01 kernel: sd 9:0:0:0: Attached scsi generic sg2 type 0
>Feb  7 13:18:35 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:18:35 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdc: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:18:35 ibm-js21-01 kernel: sdc: Write Protect is off
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdc: drive cache: write through
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdc: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:18:35 ibm-js21-01 kernel: sdc: Write Protect is off
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdc: drive cache: write through
>Feb  7 13:18:35 ibm-js21-01 kernel:  sdc: sdc1
>Feb  7 13:18:35 ibm-js21-01 kernel: sd 9:0:0:1: Attached scsi disk sdc
>Feb  7 13:18:35 ibm-js21-01 kernel: sd 9:0:0:1: Attached scsi generic sg3 type 0
>Feb  7 13:18:35 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:18:35 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdd: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:18:35 ibm-js21-01 kernel: sdd: Write Protect is off
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdd: drive cache: write through
>Feb  7 13:18:35 ibm-js21-01 kernel: SCSI device sdd: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:18:36 ibm-js21-01 kernel: sdd: Write Protect is off
>Feb  7 13:18:36 ibm-js21-01 kernel: SCSI device sdd: drive cache: write through
>Feb  7 13:18:36 ibm-js21-01 kernel:  sdd:
>Feb  7 13:18:36 ibm-js21-01 kernel: sd 9:0:0:2: Attached scsi disk sdd
>Feb  7 13:18:36 ibm-js21-01 kernel: sd 9:0:0:2: Attached scsi generic sg4 type 0
>Feb  7 13:18:36 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:18:36 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:18:36 ibm-js21-01 kernel: SCSI device sde: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:18:36 ibm-js21-01 kernel: sde: Write Protect is off
>Feb  7 13:18:36 ibm-js21-01 kernel: SCSI device sde: drive cache: write through
>Feb  7 13:18:36 ibm-js21-01 kernel: SCSI device sde: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:18:36 ibm-js21-01 kernel: sde: Write Protect is off
>Feb  7 13:18:36 ibm-js21-01 kernel: SCSI device sde: drive cache: write through
>Feb  7 13:18:36 ibm-js21-01 kernel:  sde:
>Feb  7 13:18:36 ibm-js21-01 kernel: sd 9:0:0:3: Attached scsi disk sde
>Feb  7 13:18:36 ibm-js21-01 kernel: sd 9:0:0:3: Attached scsi generic sg5 type 0
>Feb  7 13:18:38 ibm-js21-01 kernel: kjournald starting.  Commit interval 5 seconds
>Feb  7 13:18:38 ibm-js21-01 kernel: EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
>Feb  7 13:18:38 ibm-js21-01 kernel: EXT3 FS on sdb1, internal journal
>Feb  7 13:18:38 ibm-js21-01 kernel: EXT3-fs: recovery complete.
>Feb  7 13:18:38 ibm-js21-01 kernel: EXT3-fs: mounted filesystem with ordered data mode.
>Feb  7 13:18:43 ibm-js21-01 kernel: iscsi: cmd 0x2a is not queued (6)
>Feb  7 13:18:43 ibm-js21-01 last message repeated 6 times
>Feb  7 13:18:43 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:18:43 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2975088
>Feb  7 13:18:43 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371882
>Feb  7 13:18:43 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:18:43 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371883
>Feb  7 13:18:43 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:18:46 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371884
>Feb  7 13:18:46 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:18:46 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:18:46 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:18:46 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371885
>Feb  7 13:18:46 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:18:46 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:18:47 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:18:51 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371886
>Feb  7 13:18:51 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:18:53 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:18:58 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:18:59 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371887
>Feb  7 13:18:59 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:19:01 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:19:12 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:19:14 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371888
>Feb  7 13:20:01 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:01 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371889
>Feb  7 13:20:02 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:02 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371890
>Feb  7 13:20:02 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:02 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 371891
>Feb  7 13:20:02 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:02 ibm-js21-01 kernel: iscsi: cmd 0x2a is not queued (6)
>Feb  7 13:20:02 ibm-js21-01 last message repeated 5 times
>Feb  7 13:20:02 ibm-js21-01 iscsid: iSCSI logger with pid=3952 started!
>Feb  7 13:20:02 ibm-js21-01 kernel: iscsi: cmd 0x2a is not queued (6)
>Feb  7 13:20:02 ibm-js21-01 last message repeated 19 times
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:02 ibm-js21-01 last message repeated 3 times
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2974064
>Feb  7 13:20:02 ibm-js21-01 kernel: EXT3-fs error (device sdb1): read_block_bitmap: Cannot read block bitmap - block_group = 15, block_bitmap = 491520
>Feb  7 13:20:02 ibm-js21-01 kernel: Aborting journal on device sdb1.
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2973040
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2972016
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2970992
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2969968
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2968944
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2968104
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2967080
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2966056
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2965032
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2964008
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2962984
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2961960
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2960936
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 900248
>Feb  7 13:20:02 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:02 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 899224
>Feb  7 13:20:03 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:03 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 898200
>Feb  7 13:20:03 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:03 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 897176
>Feb  7 13:20:03 ibm-js21-01 iscsid: transport class version 2.0-724. iscsid version 2.0-868
>Feb  7 13:20:03 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:03 ibm-js21-01 iscsid: iSCSI daemon with pid=3953 started!
>Feb  7 13:20:03 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 896152
>Feb  7 13:20:03 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:03 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:03 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:03 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 895128
>Feb  7 13:20:03 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:03 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:03 ibm-js21-01 iscsid: connection10:0 is operational now
>Feb  7 13:20:03 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 894104
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 893080
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:04 ibm-js21-01 iscsid: connection11:0 is operational now
>Feb  7 13:20:04 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 892040
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 891016
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:04 ibm-js21-01 iscsid: connection12:0 is operational now
>Feb  7 13:20:04 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 889992
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 888968
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:04 ibm-js21-01 iscsid: connection13:0 is operational now
>Feb  7 13:20:04 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 887944
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 886920
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:04 ibm-js21-01 iscsid: connection14:0 is operational now
>Feb  7 13:20:04 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2887728
>Feb  7 13:20:04 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:04 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:05 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:05 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2625584
>Feb  7 13:20:05 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:05 ibm-js21-01 iscsid: connection15:0 is operational now
>Feb  7 13:20:05 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2371640
>Feb  7 13:20:05 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:05 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:05 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2976112
>Feb  7 13:20:05 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:05 ibm-js21-01 kernel: EXT3-fs error (device sdb1) in ext3_ordered_writepage: IO failure
>Feb  7 13:20:05 ibm-js21-01 iscsid: connection16:0 is operational now
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:05 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:05 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:05 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:05 ibm-js21-01 iscsid: connection17:0 is operational now
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:05 ibm-js21-01 last message repeated 67 times
>Feb  7 13:20:05 ibm-js21-01 kernel: ext3_abort called.
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:05 ibm-js21-01 kernel: printk: 13030 messages suppressed.
>Feb  7 13:20:05 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 1545
>Feb  7 13:20:05 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:05 ibm-js21-01 kernel: ext3_abort called.
>Feb  7 13:20:05 ibm-js21-01 kernel: EXT3-fs error (device sdb1): ext3_journal_start_sb: Detected aborted journal
>Feb  7 13:20:05 ibm-js21-01 kernel: Remounting filesystem read-only
>Feb  7 13:20:05 ibm-js21-01 kernel: EXT3-fs error (device sdb1) in ext3_prepare_write: IO failure
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:05 ibm-js21-01 last message repeated 1133 times
>Feb  7 13:20:05 ibm-js21-01 kernel: __journal_remove_journal_head: freeing b_committed_data
>Feb  7 13:20:05 ibm-js21-01 last message repeated 11 times
>Feb  7 13:20:05 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:06 ibm-js21-01 last message repeated 132 times
>Feb  7 13:20:06 ibm-js21-01 kernel: BUG: soft lockup - CPU#0 stuck for 10s! [iscsid:3419]
>Feb  7 13:20:06 ibm-js21-01 kernel: NIP: C0000000000AA8D0 LR: C0000000000ACA88 CTR: C0000000000AC988
>Feb  7 13:20:06 ibm-js21-01 kernel: REGS: c00000000fffbbc0 TRAP: 0901   Not tainted  (2.6.18-77.el5)
>Feb  7 13:20:06 ibm-js21-01 kernel: MSR: 8000000000009032 <EE,ME,IR,DR>  CR: 28008422  XER: 00000003
>Feb  7 13:20:06 ibm-js21-01 kernel: TASK = c00000000ff0da60[3419] 'iscsid' THREAD: c000000002938000 CPU: 0
>Feb  7 13:20:06 ibm-js21-01 kernel: GPR00: 8000000000009032 C00000000FFFBE40 C00000000057D670 000000000000001A 
>Feb  7 13:20:06 ibm-js21-01 kernel: GPR04: C00000000293A300 C00000000107ED80 C00000000FFFBF90 C000000000550118 
>Feb  7 13:20:06 ibm-js21-01 kernel: GPR08: 0000000000000000 0000000000000000 C000000000464E00 C000000000A50A70 
>Feb  7 13:20:06 ibm-js21-01 kernel: GPR12: 80000000F0000000 C000000000464E00 
>Feb  7 13:20:06 ibm-js21-01 kernel: NIP [C0000000000AA8D0] .handle_IRQ_event+0x48/0xf8
>Feb  7 13:20:06 ibm-js21-01 kernel: LR [C0000000000ACA88] .handle_fasteoi_irq+0x100/0x1bc
>Feb  7 13:20:06 ibm-js21-01 kernel: Call Trace:
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000FFFBE40] [C00000000FFFBF00] 0xc00000000fffbf00 (unreliable)
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000FFFBEF0] [C0000000000ACA88] .handle_fasteoi_irq+0x100/0x1bc
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000FFFBF90] [C000000000026CDC] .call_handle_irq+0x1c/0x2c
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293A200] [C00000000000D0A4] .do_IRQ+0xf4/0x1a4
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293A290] [C0000000000044F4] hardware_interrupt_entry+0xc/0x10
>Feb  7 13:20:06 ibm-js21-01 kernel: --- Exception: 501 at .vprintk+0x370/0x41c
>Feb  7 13:20:06 ibm-js21-01 kernel:     LR = .vprintk+0x360/0x41c
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293A660] [C00000000006CAB4] .printk+0x4c/0xf0
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293A700] [D00000000018FDA4] .ext3_abort+0x48/0x128 [ext3]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293A7A0] [D00000000018FECC] .ext3_journal_start_sb+0x48/0x78 [ext3]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293A820] [D000000000188AC0] .ext3_ordered_writepage+0x90/0x1fc [ext3]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293A8D0] [C000000000122638] .mpage_writepages+0x24c/0x480
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293AA50] [C0000000000C039C] .do_writepages+0x74/0xa8
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293AAD0] [C0000000001206E4] .__writeback_single_inode+0x1d0/0x3cc
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293ABC0] [C000000000120D74] .sync_sb_inodes+0x20c/0x320
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293AC70] [C000000000120F58] .sync_inodes_sb+0xd0/0x118
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293AD50] [C0000000000F2A1C] .__fsync_super+0x1c/0x114
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293ADD0] [C0000000000F2C10] .fsync_super+0x18/0x30
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293AE60] [C0000000000F2C70] .fsync_bdev+0x48/0x74
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293AEF0] [C0000000001A9CD8] .invalidate_partition+0x48/0x84
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293AF80] [C00000000014C5D4] .del_gendisk+0x40/0x1c8
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B020] [D000000000141EDC] .sd_remove+0x44/0xa4 [sd_mod]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B0C0] [C00000000021F9A0] .__device_release_driver+0xb0/0xf4
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B150] [C00000000021FEAC] .device_release_driver+0x58/0x9c
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B1E0] [C00000000021EEA4] .bus_remove_device+0xbc/0x104
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B270] [C00000000021D014] .device_del+0x17c/0x24c
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B310] [D0000000001DCAF0] .__scsi_remove_device+0x50/0xc8 [scsi_mod]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B3A0] [D0000000001D9910] .scsi_forget_host+0x50/0x9c [scsi_mod]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B430] [D0000000001D20E4] .scsi_remove_host+0xa8/0x158 [scsi_mod]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B4C0] [D000000000953510] .iscsi_session_teardown+0x78/0x178 [libiscsi]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B560] [D00000000097018C] .iscsi_tcp_session_destroy+0x28/0x40 [iscsi_tcp]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B5F0] [D0000000009327A8] .iscsi_if_rx+0x3f4/0xaec [scsi_transport_iscsi]
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B6E0] [C0000000002E828C] .netlink_data_ready+0x40/0xa4
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B760] [C0000000002E6608] .netlink_sendskb+0x4c/0x9c
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B7F0] [C0000000002E8208] .netlink_sendmsg+0x358/0x39c
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293B8F0] [C0000000002B6B3C] .sock_sendmsg+0x114/0x15c
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293BAF0] [C0000000002B7DC0] .sys_sendmsg+0x218/0x2ac
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293BD10] [C0000000002DA7A4] .compat_sys_sendmsg+0x14/0x28
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293BD90] [C0000000002DBAC0] .compat_sys_socketcall+0x1e4/0x214
>Feb  7 13:20:06 ibm-js21-01 kernel: [C00000000293BE30] [C0000000000086A4] syscall_exit+0x0/0x40
>Feb  7 13:20:06 ibm-js21-01 kernel: Instruction dump:
>Feb  7 13:20:06 ibm-js21-01 kernel: fba1ffe8 f8010010 f821ff51 7cbf2b78 7c791b78 7c9a2378 e8050008 7809dfe3 
>Feb  7 13:20:06 ibm-js21-01 kernel: 40820010 7c0000a6 60008000 7c010164 <3b600000> 3ba00000 e93f0000 e89f0028 
>Feb  7 13:20:06 ibm-js21-01 kernel: EXT3-fs error (device sdb1): ext3_journal_start_sb: Detected aborted journal
>Feb  7 13:20:06 ibm-js21-01 kernel: printk: 108626 messages suppressed.
>Feb  7 13:20:07 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 490542
>Feb  7 13:20:07 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:07 ibm-js21-01 kernel: BUG: soft lockup - CPU#1 stuck for 65s! [kblockd/1:276]
>Feb  7 13:20:07 ibm-js21-01 kernel: sd 9:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:07 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 0
>Feb  7 13:20:07 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:07 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 1
>Feb  7 13:20:07 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:07 ibm-js21-01 kernel: NIP: D0000000001D90E4 LR: D0000000001D8D50 CTR: C0000000001B0E8C
>Feb  7 13:20:07 ibm-js21-01 kernel: REGS: c000000002c1b890 TRAP: 0901   Not tainted  (2.6.18-77.el5)
>Feb  7 13:20:07 ibm-js21-01 kernel: MSR: 8000000000009032 <EE,ME,IR,DR>  CR: 28000022  XER: 0000000A
>Feb  7 13:20:07 ibm-js21-01 kernel: TASK = c00000000ff0f290[276] 'kblockd/1' THREAD: c000000002c18000 CPU: 1
>Feb  7 13:20:07 ibm-js21-01 kernel: GPR00: 8000000000009032 C000000002C1BB10 D000000000210560 0000000000000000 
>Feb  7 13:20:07 ibm-js21-01 kernel: GPR04: 0000000000000000 0000000000000001 0000000000000000 0000000000000007 
>Feb  7 13:20:07 ibm-js21-01 kernel: GPR08: C000000000465000 C0000000F78852C0 0000000000000000 C0000000001B1448 
>Feb  7 13:20:07 ibm-js21-01 kernel: GPR12: C000000002DC2160 C000000000465000 
>Feb  7 13:20:07 ibm-js21-01 kernel: NIP [D0000000001D90E4] .scsi_request_fn+0x420/0x494 [scsi_mod]
>Feb  7 13:20:07 ibm-js21-01 kernel: LR [D0000000001D8D50] .scsi_request_fn+0x8c/0x494 [scsi_mod]
>Feb  7 13:20:07 ibm-js21-01 kernel: Call Trace:
>Feb  7 13:20:07 ibm-js21-01 kernel: [C000000002C1BB10] [D0000000001D8D50] .scsi_request_fn+0x8c/0x494 [scsi_mod] (unreliable)
>Feb  7 13:20:07 ibm-js21-01 kernel: [C000000002C1BBB0] [C0000000001A59CC] .__generic_unplug_device+0x54/0x6c
>Feb  7 13:20:07 ibm-js21-01 kernel: [C000000002C1BC30] [C0000000001A7878] .generic_unplug_device+0x30/0x78
>Feb  7 13:20:07 ibm-js21-01 kernel: [C000000002C1BCC0] [C0000000001A84D4] .blk_unplug_work+0x7c/0x94
>Feb  7 13:20:07 ibm-js21-01 kernel: [C000000002C1BD50] [C00000000008153C] .run_workqueue+0xdc/0x168
>Feb  7 13:20:07 ibm-js21-01 kernel: [C000000002C1BDF0] [C0000000000822A8] .worker_thread+0x128/0x198
>Feb  7 13:20:07 ibm-js21-01 kernel: [C000000002C1BEE0] [C000000000086DD4] .kthread+0x128/0x178
>Feb  7 13:20:07 ibm-js21-01 kernel: [C000000002C1BF90] [C000000000026FB0] .kernel_thread+0x4c/0x68
>Feb  7 13:20:07 ibm-js21-01 kernel: Instruction dump:
>Feb  7 13:20:07 ibm-js21-01 kernel: e93b01e8 2fa00000 41be0010 7c0004ac 38000000 980d01c9 7c2004ac 38000000 
>Feb  7 13:20:07 ibm-js21-01 kernel: 90090000 7c0000a6 60008000 7c010164 <387f00e8> 48005c11 e8410028 e87b01e8 
>Feb  7 13:20:07 ibm-js21-01 kernel: scsi 9:0:0:0: rejecting I/O to dead device
>Feb  7 13:20:07 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 1545
>Feb  7 13:20:07 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:07 ibm-js21-01 kernel: scsi10 : iSCSI Initiator over TCP/IP
>Feb  7 13:20:07 ibm-js21-01 kernel: scsi11 : iSCSI Initiator over TCP/IP
>Feb  7 13:20:07 ibm-js21-01 kernel: scsi12 : iSCSI Initiator over TCP/IP
>Feb  7 13:20:07 ibm-js21-01 kernel: scsi13 : iSCSI Initiator over TCP/IP
>Feb  7 13:20:07 ibm-js21-01 kernel: scsi14 : iSCSI Initiator over TCP/IP
>Feb  7 13:20:07 ibm-js21-01 kernel: scsi15 : iSCSI Initiator over TCP/IP
>Feb  7 13:20:07 ibm-js21-01 kernel: scsi16 : iSCSI Initiator over TCP/IP
>Feb  7 13:20:07 ibm-js21-01 kernel: scsi17 : iSCSI Initiator over TCP/IP
>Feb  7 13:20:07 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:20:07 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:20:07 ibm-js21-01 kernel: SCSI device sdb: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:20:07 ibm-js21-01 kernel: sdb: Write Protect is off
>Feb  7 13:20:07 ibm-js21-01 kernel: SCSI device sdb: drive cache: write through
>Feb  7 13:20:07 ibm-js21-01 kernel: SCSI device sdb: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:20:07 ibm-js21-01 kernel: sdb: Write Protect is off
>Feb  7 13:20:07 ibm-js21-01 kernel: SCSI device sdb: drive cache: write through
>Feb  7 13:20:08 ibm-js21-01 kernel:  sdb: sdb1
>Feb  7 13:20:08 ibm-js21-01 kernel: sd 17:0:0:0: Attached scsi disk sdb
>Feb  7 13:20:08 ibm-js21-01 kernel: sd 17:0:0:0: Attached scsi generic sg2 type 0
>Feb  7 13:20:08 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:20:08 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sdc: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:20:08 ibm-js21-01 kernel: sdc: Write Protect is off
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sdc: drive cache: write through
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sdc: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:20:08 ibm-js21-01 kernel: sdc: Write Protect is off
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sdc: drive cache: write through
>Feb  7 13:20:08 ibm-js21-01 kernel:  sdc: sdc1
>Feb  7 13:20:08 ibm-js21-01 kernel: sd 17:0:0:1: Attached scsi disk sdc
>Feb  7 13:20:08 ibm-js21-01 kernel: sd 17:0:0:1: Attached scsi generic sg3 type 0
>Feb  7 13:20:08 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:20:08 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sdd: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:20:08 ibm-js21-01 kernel: sdd: Write Protect is off
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sdd: drive cache: write through
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sdd: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:20:08 ibm-js21-01 kernel: sdd: Write Protect is off
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sdd: drive cache: write through
>Feb  7 13:20:08 ibm-js21-01 kernel:  sdd:
>Feb  7 13:20:08 ibm-js21-01 kernel: sd 17:0:0:2: Attached scsi disk sdd
>Feb  7 13:20:08 ibm-js21-01 kernel: sd 17:0:0:2: Attached scsi generic sg4 type 0
>Feb  7 13:20:08 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:20:08 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sde: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:20:08 ibm-js21-01 kernel: sde: Write Protect is off
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sde: drive cache: write through
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sde: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:20:08 ibm-js21-01 kernel: sde: Write Protect is off
>Feb  7 13:20:08 ibm-js21-01 kernel: SCSI device sde: drive cache: write through
>Feb  7 13:20:08 ibm-js21-01 kernel:  sde:
>Feb  7 13:20:08 ibm-js21-01 kernel: sd 17:0:0:3: Attached scsi disk sde
>Feb  7 13:20:08 ibm-js21-01 kernel: sd 17:0:0:3: Attached scsi generic sg5 type 0
>Feb  7 13:20:09 ibm-js21-01 kernel: kjournald starting.  Commit interval 5 seconds
>Feb  7 13:20:09 ibm-js21-01 kernel: EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
>Feb  7 13:20:09 ibm-js21-01 kernel: EXT3 FS on sdb1, internal journal
>Feb  7 13:20:09 ibm-js21-01 kernel: EXT3-fs: recovery complete.
>Feb  7 13:20:09 ibm-js21-01 kernel: EXT3-fs: mounted filesystem with ordered data mode.
>Feb  7 13:20:14 ibm-js21-01 kernel: iscsi: cmd 0x2a is not queued (6)
>Feb  7 13:20:14 ibm-js21-01 last message repeated 6 times
>Feb  7 13:20:14 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:14 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2995576
>Feb  7 13:20:15 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374443
>Feb  7 13:20:15 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:15 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374444
>Feb  7 13:20:15 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:15 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374445
>Feb  7 13:20:15 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:15 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374446
>Feb  7 13:20:15 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:15 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374447
>Feb  7 13:20:15 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:15 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374448
>Feb  7 13:20:15 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:17 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374449
>Feb  7 13:20:17 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:20:17 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:22 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:20:23 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374450
>Feb  7 13:20:23 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:20:23 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:23 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:20:24 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374451
>Feb  7 13:20:24 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:20:24 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:24 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:20:24 ibm-js21-01 kernel: iscsi: cmd 0x2a is not queued (6)
>Feb  7 13:20:25 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:20:25 ibm-js21-01 kernel: iscsi: cmd 0x2a is not queued (6)
>Feb  7 13:20:25 ibm-js21-01 iscsid: received iferror -22
>Feb  7 13:20:25 ibm-js21-01 kernel: iscsi: cmd 0x2a is not queued (6)
>Feb  7 13:20:28 ibm-js21-01 last message repeated 23 times
>Feb  7 13:20:28 ibm-js21-01 kernel: sd 17:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:28 ibm-js21-01 kernel: sd 17:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:28 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:28 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2994552
>Feb  7 13:20:28 ibm-js21-01 kernel: EXT3-fs error (device sdb1): read_block_bitmap: Cannot read block bitmap - block_group = 15, block_bitmap = 491520
>Feb  7 13:20:28 ibm-js21-01 kernel: Aborting journal on device sdb1.
>Feb  7 13:20:28 ibm-js21-01 kernel: printk: 247 messages suppressed.
>Feb  7 13:20:28 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 374315
>Feb  7 13:20:28 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:28 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:28 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2993528
>Feb  7 13:20:28 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:28 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2992496
>Feb  7 13:20:29 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:29 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2991472
>Feb  7 13:20:29 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:29 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2990448
>Feb  7 13:20:30 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:30 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2989424
>Feb  7 13:20:30 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:30 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2988400
>Feb  7 13:20:30 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:31 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2987376
>Feb  7 13:20:31 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:31 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2986352
>Feb  7 13:20:31 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:31 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2985328
>Feb  7 13:20:32 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:32 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2984304
>Feb  7 13:20:32 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:32 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2983280
>Feb  7 13:20:32 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:33 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2982256
>Feb  7 13:20:33 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:33 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2981232
>Feb  7 13:20:33 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:34 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2980208
>Feb  7 13:20:34 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:34 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2979184
>Feb  7 13:20:35 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:35 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2978160
>Feb  7 13:20:35 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:35 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2978152
>Feb  7 13:20:35 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:36 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2977128
>Feb  7 13:20:36 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:36 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2976120
>Feb  7 13:20:36 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:36 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2975096
>Feb  7 13:20:36 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:36 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2974064
>Feb  7 13:20:37 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:37 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2973040
>Feb  7 13:20:37 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:37 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2972016
>Feb  7 13:20:38 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:38 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2970984
>Feb  7 13:20:38 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:38 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2969960
>Feb  7 13:20:38 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:39 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2968936
>Feb  7 13:20:39 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:39 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2968120
>Feb  7 13:20:39 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:39 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2967096
>Feb  7 13:20:39 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:40 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2966072
>Feb  7 13:20:40 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:40 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2965040
>Feb  7 13:20:40 ibm-js21-01 kernel: sd 17:0:0:0: SCSI error: return code = 0x00010000
>Feb  7 13:20:40 ibm-js21-01 kernel: end_request: I/O error, dev sdb, sector 2996600
>Feb  7 13:20:40 ibm-js21-01 kernel: EXT3-fs error (device sdb1) in ext3_ordered_writepage: IO failure
>Feb  7 13:20:41 ibm-js21-01 kernel: sd 17:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:44 ibm-js21-01 last message repeated 77 times
>Feb  7 13:20:44 ibm-js21-01 kernel: printk: 13540 messages suppressed.
>Feb  7 13:20:44 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 1545
>Feb  7 13:20:44 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:20:44 ibm-js21-01 kernel: sd 17:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:44 ibm-js21-01 kernel: ext3_abort called.
>Feb  7 13:20:44 ibm-js21-01 kernel: sd 17:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:44 ibm-js21-01 kernel: ext3_abort called.
>Feb  7 13:20:44 ibm-js21-01 kernel: EXT3-fs error (device sdb1): ext3_journal_start_sb: Detected aborted journal
>Feb  7 13:20:44 ibm-js21-01 kernel: Remounting filesystem read-only
>Feb  7 13:20:44 ibm-js21-01 kernel: EXT3-fs error (device sdb1) in ext3_prepare_write: IO failure
>Feb  7 13:20:44 ibm-js21-01 kernel: sd 17:0:0:0: rejecting I/O to device being removed
>Feb  7 13:20:47 ibm-js21-01 last message repeated 1101 times
>Feb  7 13:20:47 ibm-js21-01 kernel: __journal_remove_journal_head: freeing b_committed_data
>Feb  7 13:20:47 ibm-js21-01 last message repeated 11 times
>Feb  7 13:20:47 ibm-js21-01 kernel: sd 17:0:0:0: rejecting I/O to device being removed
>Feb  7 13:21:32 ibm-js21-01 last message repeated 132 times
>Feb  7 13:21:32 ibm-js21-01 kernel: BUG: soft lockup - CPU#0 stuck for 10s! [iscsid:3953]
>Feb  7 13:21:32 ibm-js21-01 kernel: NIP: C0000000000AA8D0 LR: C0000000000ACA88 CTR: C0000000000AC988
>Feb  7 13:21:32 ibm-js21-01 kernel: REGS: c00000000fffbbc0 TRAP: 0901   Not tainted  (2.6.18-77.el5)
>Feb  7 13:21:32 ibm-js21-01 kernel: MSR: 8000000000009032 <EE,ME,IR,DR>  CR: 28008482  XER: 00000002
>Feb  7 13:21:32 ibm-js21-01 kernel: TASK = c0000000f7d959e0[3953] 'iscsid' THREAD: c00000000295c000 CPU: 0
>Feb  7 13:21:32 ibm-js21-01 kernel: GPR00: 8000000000009032 C00000000FFFBE40 C00000000057D670 0000000000000012 
>Feb  7 13:21:32 ibm-js21-01 kernel: GPR04: C00000000295E300 C0000000F7BCB700 C00000000FFFBF90 C000000000550118 
>Feb  7 13:21:32 ibm-js21-01 kernel: GPR08: 0000000000000000 0000000000000000 C000000000464E00 C000000000A50A50 
>Feb  7 13:21:32 ibm-js21-01 kernel: GPR12: 80000000F0000000 C000000000464E00 
>Feb  7 13:21:32 ibm-js21-01 kernel: NIP [C0000000000AA8D0] .handle_IRQ_event+0x48/0xf8
>Feb  7 13:21:32 ibm-js21-01 kernel: LR [C0000000000ACA88] .handle_fasteoi_irq+0x100/0x1bc
>Feb  7 13:21:32 ibm-js21-01 kernel: Call Trace:
>Feb  7 13:21:32 ibm-js21-01 kernel: [C00000000FFFBE40] [C0000000000AA940] .handle_IRQ_event+0xb8/0xf8 (unreliable)
>Feb  7 13:21:32 ibm-js21-01 kernel: [C00000000FFFBEF0] [C0000000000ACA88] .handle_fasteoi_irq+0x100/0x1bc
>Feb  7 13:21:32 ibm-js21-01 kernel: [C00000000FFFBF90] [C000000000026CDC] .call_handle_irq+0x1c/0x2c
>Feb  7 13:21:32 ibm-js21-01 kernel: [C00000000295E200] [C00000000000D0A4] .do_IRQ+0xf4/0x1a4
>Feb  7 13:21:32 ibm-js21-01 kernel: [C00000000295E290] [C0000000000044F4] hardware_interrupt_entry+0xc/0x10
>Feb  7 13:21:32 ibm-js21-01 kernel: --- Exception: 501 at .vprintk+0x370/0x41c
>Feb  7 13:21:32 ibm-js21-01 kernel:     LR = .vprintk+0x360/0x41c
>Feb  7 13:21:32 ibm-js21-01 kernel: [C00000000295E660] [C00000000006CAB4] .printk+0x4c/0xf0
>Feb  7 13:21:32 ibm-js21-01 kernel: [C00000000295E700] [D00000000018FDA4] .ext3_abort+0x48/0x128 [ext3]
>Feb  7 13:21:32 ibm-js21-01 kernel: [C00000000295E7A0] [D00000000018FECC] .ext3_journal_start_sb+0x48/0x78 [ext3]
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295E820] [D000000000188AC0] .ext3_ordered_writepage+0x90/0x1fc [ext3]
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295E8D0] [C000000000122638] .mpage_writepages+0x24c/0x480
>Feb  7 13:21:33 ibm-js21-01 iscsid: iSCSI logger with pid=4458 started!
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295EA50] [C0000000000C039C] .do_writepages+0x74/0xa8
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295EAD0] [C0000000001206E4] .__writeback_single_inode+0x1d0/0x3cc
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295EBC0] [C000000000120D74] .sync_sb_inodes+0x20c/0x320
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295EC70] [C000000000120F58] .sync_inodes_sb+0xd0/0x118
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295ED50] [C0000000000F2A1C] .__fsync_super+0x1c/0x114
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295EDD0] [C0000000000F2C10] .fsync_super+0x18/0x30
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295EE60] [C0000000000F2C70] .fsync_bdev+0x48/0x74
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295EEF0] [C0000000001A9CD8] .invalidate_partition+0x48/0x84
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295EF80] [C00000000014C5D4] .del_gendisk+0x40/0x1c8
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F020] [D000000000141EDC] .sd_remove+0x44/0xa4 [sd_mod]
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F0C0] [C00000000021F9A0] .__device_release_driver+0xb0/0xf4
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F150] [C00000000021FEAC] .device_release_driver+0x58/0x9c
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F1E0] [C00000000021EEA4] .bus_remove_device+0xbc/0x104
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F270] [C00000000021D014] .device_del+0x17c/0x24c
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F310] [D0000000001DCAF0] .__scsi_remove_device+0x50/0xc8 [scsi_mod]
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F3A0] [D0000000001D9910] .scsi_forget_host+0x50/0x9c [scsi_mod]
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F430] [D0000000001D20E4] .scsi_remove_host+0xa8/0x158 [scsi_mod]
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F4C0] [D000000000953510] .iscsi_session_teardown+0x78/0x178 [libiscsi]
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F560] [D00000000097018C] .iscsi_tcp_session_destroy+0x28/0x40 [iscsi_tcp]
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F5F0] [D0000000009327A8] .iscsi_if_rx+0x3f4/0xaec [scsi_transport_iscsi]
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F6E0] [C0000000002E828C] .netlink_data_ready+0x40/0xa4
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F760] [C0000000002E6608] .netlink_sendskb+0x4c/0x9c
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F7F0] [C0000000002E8208] .netlink_sendmsg+0x358/0x39c
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295F8F0] [C0000000002B6B3C] .sock_sendmsg+0x114/0x15c
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295FAF0] [C0000000002B7DC0] .sys_sendmsg+0x218/0x2ac
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295FD10] [C0000000002DA7A4] .compat_sys_sendmsg+0x14/0x28
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295FD90] [C0000000002DBAC0] .compat_sys_socketcall+0x1e4/0x214
>Feb  7 13:21:33 ibm-js21-01 kernel: [C00000000295FE30] [C0000000000086A4] syscall_exit+0x0/0x40
>Feb  7 13:21:33 ibm-js21-01 kernel: Instruction dump:
>Feb  7 13:21:33 ibm-js21-01 kernel: fba1ffe8 f8010010 f821ff51 7cbf2b78 7c791b78 7c9a2378 e8050008 7809dfe3 
>Feb  7 13:21:33 ibm-js21-01 kernel: 40820010 7c0000a6 60008000 7c010164 <3b600000> 3ba00000 e93f0000 e89f0028 
>Feb  7 13:21:33 ibm-js21-01 kernel: EXT3-fs error (device sdb1): ext3_journal_start_sb: Detected aborted journal
>Feb  7 13:21:33 ibm-js21-01 kernel: printk: 105551 messages suppressed.
>Feb  7 13:21:33 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 1
>Feb  7 13:21:34 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:21:34 ibm-js21-01 iscsid: transport class version 2.0-724. iscsid version 2.0-868
>Feb  7 13:21:34 ibm-js21-01 kernel: BUG: soft lockup - CPU#1 stuck for 63s! [kblockd/1:276]
>Feb  7 13:21:34 ibm-js21-01 iscsid: iSCSI daemon with pid=4459 started!
>Feb  7 13:21:34 ibm-js21-01 kernel: NIP: D0000000001D90E4 LR: D0000000001D8D50 CTR: C0000000001B0E8C
>Feb  7 13:21:34 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:34 ibm-js21-01 kernel: REGS: c000000002c1b890 TRAP: 0901   Not tainted  (2.6.18-77.el5)
>Feb  7 13:21:34 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:34 ibm-js21-01 kernel: MSR: 8000000000009032 <EE,ME,IR,DR>  CR: 28000022  XER: 0000000A
>Feb  7 13:21:34 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:34 ibm-js21-01 kernel: TASK = c00000000ff0f290[276] 'kblockd/1' THREAD: c000000002c18000 CPU: 1
>Feb  7 13:21:34 ibm-js21-01 iscsid: connection18:0 is operational now
>Feb  7 13:21:34 ibm-js21-01 kernel: GPR00: 8000000000009032 C000000002C1BB10 D000000000210560 0000000000000000 
>Feb  7 13:21:34 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:34 ibm-js21-01 kernel: GPR04: 0000000000000000 0000000000000001 0000000000000000 0000000000000007 
>Feb  7 13:21:34 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:34 ibm-js21-01 kernel: GPR08: C000000000465000 C0000000F78858E8 0000000000000000 C0000000001B1448 
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: GPR12: C000000002DCAB60 C000000000465000 
>Feb  7 13:21:35 ibm-js21-01 iscsid: connection19:0 is operational now
>Feb  7 13:21:35 ibm-js21-01 kernel: NIP [D0000000001D90E4] .scsi_request_fn+0x420/0x494 [scsi_mod]
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: LR [D0000000001D8D50] .scsi_request_fn+0x8c/0x494 [scsi_mod]
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: Call Trace:
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: [C000000002C1BB10] [D0000000001D8D50] .scsi_request_fn+0x8c/0x494 [scsi_mod] (unreliable)
>Feb  7 13:21:35 ibm-js21-01 iscsid: connection20:0 is operational now
>Feb  7 13:21:35 ibm-js21-01 kernel: [C000000002C1BBB0] [C0000000001A59CC] .__generic_unplug_device+0x54/0x6c
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: [C000000002C1BC30] [C0000000001A7878] .generic_unplug_device+0x30/0x78
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: [C000000002C1BCC0] [C0000000001A84D4] .blk_unplug_work+0x7c/0x94
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: [C000000002C1BD50] [C00000000008153C] .run_workqueue+0xdc/0x168
>Feb  7 13:21:35 ibm-js21-01 iscsid: connection21:0 is operational now
>Feb  7 13:21:35 ibm-js21-01 kernel: [C000000002C1BDF0] [C0000000000822A8] .worker_thread+0x128/0x198
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: [C000000002C1BEE0] [C000000000086DD4] .kthread+0x128/0x178
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: [C000000002C1BF90] [C000000000026FB0] .kernel_thread+0x4c/0x68
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: Instruction dump:
>Feb  7 13:21:35 ibm-js21-01 iscsid: connection22:0 is operational now
>Feb  7 13:21:35 ibm-js21-01 kernel: e93b01e8 2fa00000 41be0010 7c0004ac 38000000 980d01c9 7c2004ac 38000000 
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: 90090000 7c0000a6 60008000 7c010164 <387f00e8> 48005c11 e8410028 e87b01e8 
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: scsi 17:0:0:0: rejecting I/O to dead device
>Feb  7 13:21:35 ibm-js21-01 iscsid: connection23:0 is operational now
>Feb  7 13:21:35 ibm-js21-01 kernel: Buffer I/O error on device sdb1, logical block 1545
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: lost page write due to I/O error on sdb1
>Feb  7 13:21:35 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:35 ibm-js21-01 kernel: scsi18 : iSCSI Initiator over TCP/IP
>Feb  7 13:21:36 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:36 ibm-js21-01 kernel: scsi19 : iSCSI Initiator over TCP/IP
>Feb  7 13:21:36 ibm-js21-01 iscsid: connection24:0 is operational now
>Feb  7 13:21:36 ibm-js21-01 kernel: scsi20 : iSCSI Initiator over TCP/IP
>Feb  7 13:21:36 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:36 ibm-js21-01 kernel: scsi21 : iSCSI Initiator over TCP/IP
>Feb  7 13:21:36 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:36 ibm-js21-01 kernel: scsi22 : iSCSI Initiator over TCP/IP
>Feb  7 13:21:36 ibm-js21-01 iscsid: received iferror -38
>Feb  7 13:21:36 ibm-js21-01 kernel: scsi23 : iSCSI Initiator over TCP/IP
>Feb  7 13:21:36 ibm-js21-01 iscsid: connection25:0 is operational now
>Feb  7 13:21:36 ibm-js21-01 kernel: scsi24 : iSCSI Initiator over TCP/IP
>Feb  7 13:21:36 ibm-js21-01 kernel: scsi25 : iSCSI Initiator over TCP/IP
>Feb  7 13:21:36 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:21:36 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdb: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:21:36 ibm-js21-01 kernel: sdb: Write Protect is off
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdb: drive cache: write through
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdb: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:21:36 ibm-js21-01 kernel: sdb: Write Protect is off
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdb: drive cache: write through
>Feb  7 13:21:36 ibm-js21-01 kernel:  sdb: sdb1
>Feb  7 13:21:36 ibm-js21-01 kernel: sd 25:0:0:0: Attached scsi disk sdb
>Feb  7 13:21:36 ibm-js21-01 kernel: sd 25:0:0:0: Attached scsi generic sg2 type 0
>Feb  7 13:21:36 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:21:36 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdc: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:21:36 ibm-js21-01 kernel: sdc: Write Protect is off
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdc: drive cache: write through
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdc: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:21:36 ibm-js21-01 kernel: sdc: Write Protect is off
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdc: drive cache: write through
>Feb  7 13:21:36 ibm-js21-01 kernel:  sdc: sdc1
>Feb  7 13:21:36 ibm-js21-01 kernel: sd 25:0:0:1: Attached scsi disk sdc
>Feb  7 13:21:36 ibm-js21-01 kernel: sd 25:0:0:1: Attached scsi generic sg3 type 0
>Feb  7 13:21:36 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:21:36 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdd: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:21:36 ibm-js21-01 kernel: sdd: Write Protect is off
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdd: drive cache: write through
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdd: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:21:36 ibm-js21-01 kernel: sdd: Write Protect is off
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sdd: drive cache: write through
>Feb  7 13:21:36 ibm-js21-01 kernel:  sdd:
>Feb  7 13:21:36 ibm-js21-01 kernel: sd 25:0:0:2: Attached scsi disk sdd
>Feb  7 13:21:36 ibm-js21-01 kernel: sd 25:0:0:2: Attached scsi generic sg4 type 0
>Feb  7 13:21:36 ibm-js21-01 kernel:   Vendor: EMC       Model: Celerra iSCSI     Rev: 0001
>Feb  7 13:21:36 ibm-js21-01 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
>Feb  7 13:21:36 ibm-js21-01 kernel: SCSI device sde: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:21:37 ibm-js21-01 kernel: sde: Write Protect is off
>Feb  7 13:21:37 ibm-js21-01 kernel: SCSI device sde: drive cache: write through
>Feb  7 13:21:37 ibm-js21-01 kernel: SCSI device sde: 122880000 512-byte hdwr sectors (62915 MB)
>Feb  7 13:21:37 ibm-js21-01 kernel: sde: Write Protect is off
>Feb  7 13:21:37 ibm-js21-01 kernel: SCSI device sde: drive cache: write through
>Feb  7 13:21:37 ibm-js21-01 kernel:  sde:
>Feb  7 13:21:37 ibm-js21-01 kernel: sd 25:0:0:3: Attached scsi disk sde
>Feb  7 13:21:37 ibm-js21-01 kernel: sd 25:0:0:3: Attached scsi generic sg5 type 0

Actions: View

Attachments on bug 431907: 294258