Attachment 706698 Details for Bug 918889 – 3.7.3-101.fc17.i686.PAE dmesg output

3.7.3-101.fc17.i686.PAE dmesg output

dmesg-2 (text/plain), 241.23 KB, created by calvin on 2013-03-07 15:41:47 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: calvin

Created: 2013-03-07 15:41:47 UTC

Size: 241.23 KB

patch

obsolete

>[10355.019735] Descriptor sense data with sense descriptors (in hex):
>[10355.019738]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10355.019759]         26 8c e2 51 
>[10355.019769] sd 2:0:0:0: [sda]  
>[10355.019775] Add. Sense: No additional sense information
>[10355.019783] sd 2:0:0:0: [sda] CDB: 
>[10355.019789] Read(10): 28 00 26 8c e2 00 00 01 00 00
>[10355.019813] end_request: I/O error, dev sda, sector 646767104
>[10355.036359] ata3: EH complete
>[10361.834737] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10361.851230] ata3.00: BMDMA2 stat 0x686d0009
>[10361.867000] ata3: SError: { Handshk UnrecFIS }
>[10361.882453] ata3.00: failed command: READ DMA EXT
>[10361.897313] ata3.00: cmd 25/00:00:00:53:83/00:01:18:00:00/e0 tag 0 dma 131072 in
>[10361.897313]          res 51/04:8f:71:53:83/00:00:18:00:00/e0 Emask 0x1 (device error)
>[10361.958423] ata3.00: status: { DRDY ERR }
>[10361.973691] ata3.00: error: { ABRT }
>[10361.991561] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10361.995324] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10361.995330] ata3.00: configured for UDMA/33
>[10361.995355] sd 2:0:0:0: [sda]  
>[10361.995358] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10361.995362] sd 2:0:0:0: [sda]  
>[10361.995364] Sense Key : Aborted Command [current] [descriptor]
>[10361.995369] Descriptor sense data with sense descriptors (in hex):
>[10361.995372]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10361.995386]         18 83 53 71 
>[10361.995392] sd 2:0:0:0: [sda]  
>[10361.995396] Add. Sense: No additional sense information
>[10361.995399] sd 2:0:0:0: [sda] CDB: 
>[10361.995401] Read(10): 28 00 18 83 53 00 00 01 00 00
>[10361.995414] end_request: I/O error, dev sda, sector 411259648
>[10362.010913] ata3: EH complete
>[10374.499489] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10374.517797] ata4.00: BMDMA2 stat 0x650001
>[10374.537615] ata4: SError: { Handshk UnrecFIS }
>[10374.555412] ata4.00: failed command: WRITE DMA EXT
>[10374.572093] ata4.00: cmd 35/00:00:00:70:01/00:02:0b:00:00/e0 tag 0 dma 262144 out
>[10374.572093]          res 51/04:f0:10:70:01/00:01:0b:00:00/e0 Emask 0x1 (device error)
>[10374.642040] ata4.00: status: { DRDY ERR }
>[10374.658676] ata4.00: error: { ABRT }
>[10374.690265] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10374.693835] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10374.693853] ata4.00: configured for UDMA/33
>[10374.693896] sd 3:0:0:0: [sdb]  
>[10374.693902] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10374.693907] sd 3:0:0:0: [sdb]  
>[10374.693911] Sense Key : Aborted Command [current] [descriptor]
>[10374.693919] Descriptor sense data with sense descriptors (in hex):
>[10374.693922]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10374.693942]         0b 01 70 10 
>[10374.693951] sd 3:0:0:0: [sdb]  
>[10374.693956] Add. Sense: No additional sense information
>[10374.693961] sd 3:0:0:0: [sdb] CDB: 
>[10374.693964] Write(10): 2a 00 0b 01 70 00 00 02 00 00
>[10374.693983] end_request: I/O error, dev sdb, sector 184643584
>[10374.711646] Buffer I/O error on device sdb1, logical block 23080192
>[10374.730065] Buffer I/O error on device sdb1, logical block 23080193
>[10374.748145] Buffer I/O error on device sdb1, logical block 23080194
>[10374.764557] Buffer I/O error on device sdb1, logical block 23080195
>[10374.780309] Buffer I/O error on device sdb1, logical block 23080196
>[10374.794590] Buffer I/O error on device sdb1, logical block 23080197
>[10374.808939] Buffer I/O error on device sdb1, logical block 23080198
>[10374.824635] Buffer I/O error on device sdb1, logical block 23080199
>[10374.840371] Buffer I/O error on device sdb1, logical block 23080200
>[10374.854904] Buffer I/O error on device sdb1, logical block 23080201
>[10374.868439] Buffer I/O error on device sdb1, logical block 23080202
>[10374.882380] Buffer I/O error on device sdb1, logical block 23080203
>[10374.895663] Buffer I/O error on device sdb1, logical block 23080204
>[10374.907138] Buffer I/O error on device sdb1, logical block 23080205
>[10374.918937] Buffer I/O error on device sdb1, logical block 23080206
>[10374.930368] Buffer I/O error on device sdb1, logical block 23080207
>[10374.939328] Buffer I/O error on device sdb1, logical block 23080208
>[10374.949694] Buffer I/O error on device sdb1, logical block 23080209
>[10374.959197] Buffer I/O error on device sdb1, logical block 23080210
>[10374.967823] Buffer I/O error on device sdb1, logical block 23080211
>[10374.975367] Buffer I/O error on device sdb1, logical block 23080212
>[10374.982976] Buffer I/O error on device sdb1, logical block 23080213
>[10374.990280] Buffer I/O error on device sdb1, logical block 23080214
>[10374.997163] Buffer I/O error on device sdb1, logical block 23080215
>[10375.002583] Buffer I/O error on device sdb1, logical block 23080216
>[10375.007417] Buffer I/O error on device sdb1, logical block 23080217
>[10375.011663] Buffer I/O error on device sdb1, logical block 23080218
>[10375.016635] Buffer I/O error on device sdb1, logical block 23080219
>[10375.020399] Buffer I/O error on device sdb1, logical block 23080220
>[10375.023970] Buffer I/O error on device sdb1, logical block 23080221
>[10375.027718] Buffer I/O error on device sdb1, logical block 23080222
>[10375.030923] Buffer I/O error on device sdb1, logical block 23080223
>[10375.034056] Buffer I/O error on device sdb1, logical block 23080224
>[10375.037167] Buffer I/O error on device sdb1, logical block 23080225
>[10375.040870] Buffer I/O error on device sdb1, logical block 23080226
>[10375.043948] Buffer I/O error on device sdb1, logical block 23080227
>[10375.047099] Buffer I/O error on device sdb1, logical block 23080228
>[10375.050126] Buffer I/O error on device sdb1, logical block 23080229
>[10375.053651] Buffer I/O error on device sdb1, logical block 23080230
>[10375.057294] Buffer I/O error on device sdb1, logical block 23080231
>[10375.060728] Buffer I/O error on device sdb1, logical block 23080232
>[10375.063808] Buffer I/O error on device sdb1, logical block 23080233
>[10375.066674] Buffer I/O error on device sdb1, logical block 23080234
>[10375.070415] Buffer I/O error on device sdb1, logical block 23080235
>[10375.073754] Buffer I/O error on device sdb1, logical block 23080236
>[10375.076852] Buffer I/O error on device sdb1, logical block 23080237
>[10375.080255] Buffer I/O error on device sdb1, logical block 23080238
>[10375.083619] Buffer I/O error on device sdb1, logical block 23080239
>[10375.087163] Buffer I/O error on device sdb1, logical block 23080240
>[10375.090868] Buffer I/O error on device sdb1, logical block 23080241
>[10375.094099] Buffer I/O error on device sdb1, logical block 23080242
>[10375.097022] Buffer I/O error on device sdb1, logical block 23080243
>[10375.100128] Buffer I/O error on device sdb1, logical block 23080244
>[10375.103546] Buffer I/O error on device sdb1, logical block 23080245
>[10375.106813] Buffer I/O error on device sdb1, logical block 23080246
>[10375.107594] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10375.107597] ata3.00: BMDMA2 stat 0x606d0009
>[10375.107600] ata3: SError: { Handshk UnrecFIS }
>[10375.107620] ata3.00: failed command: READ DMA EXT
>[10375.107629] ata3.00: cmd 25/00:00:00:0b:d0/00:01:12:00:00/e0 tag 0 dma 131072 in
>[10375.107629]          res 51/04:ef:11:0b:d0/00:00:12:00:00/e0 Emask 0x1 (device error)
>[10375.107631] ata3.00: status: { DRDY ERR }
>[10375.107633] ata3.00: error: { ABRT }
>[10375.161421] Buffer I/O error on device sdb1, logical block 23080247
>[10375.162340] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10375.165621] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10375.165627] ata3.00: configured for UDMA/33
>[10375.165651] sd 2:0:0:0: [sda]  
>[10375.165654] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10375.165656] sd 2:0:0:0: [sda]  
>[10375.165660] Sense Key : Aborted Command [current] [descriptor]
>[10375.165663] Descriptor sense data with sense descriptors (in hex):
>[10375.165676]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10375.165680]         12 d0 0b 11 
>[10375.165683] sd 2:0:0:0: [sda]  
>[10375.165686] Add. Sense: No additional sense information
>[10375.165689] sd 2:0:0:0: [sda] CDB: 
>[10375.165698] Read(10): 28 00 12 d0 0b 00 00 01 00 00
>[10375.165701] end_request: I/O error, dev sda, sector 315624192
>[10375.165751] ata3: EH complete
>[10375.177002] Buffer I/O error on device sdb1, logical block 23080248
>[10375.185290] Buffer I/O error on device sdb1, logical block 23080249
>[10375.194307] Buffer I/O error on device sdb1, logical block 23080250
>[10375.203163] Buffer I/O error on device sdb1, logical block 23080251
>[10375.212805] Buffer I/O error on device sdb1, logical block 23080252
>[10375.222595] Buffer I/O error on device sdb1, logical block 23080253
>[10375.233448] Buffer I/O error on device sdb1, logical block 23080254
>[10375.243253] Buffer I/O error on device sdb1, logical block 23080255
>[10375.253362] EXT4-fs warning (device sdb1): ext4_end_bio:319: I/O error writing to inode 38015918 (offset 15597568 size 262144 starting block 23080448)
>[10375.253416] ata4: EH complete
>[10375.807184] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10375.807184] Do you have a strange power saving mode enabled?
>[10375.807184] Dazed and confused, but trying to continue
>[10375.807184] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10375.807184] Do you have a strange power saving mode enabled?
>[10375.807184] Dazed and confused, but trying to continue
>[10375.807184] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[10375.807184] Do you have a strange power saving mode enabled?
>[10375.807184] Dazed and confused, but trying to continue
>[10375.807184] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10375.807184] Do you have a strange power saving mode enabled?
>[10375.807184] Dazed and confused, but trying to continue
>[10375.807184] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[10375.807184] Do you have a strange power saving mode enabled?
>[10375.807184] Dazed and confused, but trying to continue
>[10375.807184] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10376.001550] Do you have a strange power saving mode enabled?
>[10376.001550] Dazed and confused, but trying to continue
>[10376.001550] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[10376.001550] Do you have a strange power saving mode enabled?
>[10376.001550] Dazed and confused, but trying to continue
>[10376.068502] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10376.080248] ------------[ cut here ]------------
>[10376.080248] WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0xac/0xd0()
>[10376.080248] Hardware name: Sun Fire(tm) V60 370-6037               
>[10376.080248] Watchdog detected hard LOCKUP on cpu 2
>[10376.080248] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[10376.080248] Pid: 192, comm: scsi_eh_2 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1
>[10376.080248] Call Trace:
>[10376.080248]  [<c044d6a2>] warn_slowpath_common+0x72/0xa0
>[10376.080248]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[10376.080248]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[10376.080248]  [<c04ccb60>] ? touch_nmi_watchdog+0x70/0x70
>[10376.080248]  [<c044d773>] warn_slowpath_fmt+0x33/0x40
>[10376.080248]  [<c04ccc0c>] watchdog_overflow_callback+0xac/0xd0
>[10376.080248]  [<c04ff986>] __perf_event_overflow+0xa6/0x270
>[10376.080248]  [<c042021f>] ? x86_perf_event_set_period+0x11f/0x230
>[10376.080248]  [<c05004f5>] perf_event_overflow+0x15/0x20
>[10376.080248]  [<c0422033>] p4_pmu_handle_irq+0x143/0x1f0
>[10376.080248]  [<c047191a>] ? hrtimer_forward+0x10a/0x250
>[10376.080248]  [<c098424b>] perf_event_nmi_handler+0x1b/0x20
>[10376.080248]  [<c09839c9>] nmi_handle.isra.0+0x39/0x60
>[10376.080248]  [<c07a6740>] ? ata_sff_wait_after_reset+0x110/0x110
>[10376.080248]  [<c0983bcf>] do_nmi+0x1df/0x3f0
>[10376.080248]  [<c04806c7>] ? sched_clock_cpu+0xe7/0x190
>[10376.080248]  [<c07a6740>] ? ata_sff_wait_after_reset+0x110/0x110
>[10376.080248]  [<c0983107>] nmi_stack_correct+0x2f/0x34
>[10376.080248]  [<c07a6070>] ? ata_pci_sff_prepare_host+0xb0/0xb0
>[10376.080248]  [<c07a6740>] ? ata_sff_wait_after_reset+0x110/0x110
>[10376.080248]  [<c045007b>] ? devkmsg_read+0x1eb/0x430
>[10376.080248]  [<c04300e0>] ? perf_ibs_init+0x1e0/0x280
>[10376.080248]  [<c0693be3>] ? ioread16+0x33/0x40
>[10376.080248]  [<c07a60a3>] ? ata_sff_drain_fifo+0x33/0x80
>[10376.080248]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10376.080248]  [<c07a6a80>] ? ata_pio_sectors+0xa0/0xa0
>[10376.080248]  [<c07a6dc7>] ata_sff_error_handler+0x67/0xe0
>[10376.080248]  [<c07a007b>] ? ata_eh_acquire+0x2b/0x60
>[10376.080248]  [<c07a6e93>] ata_bmdma_error_handler+0x53/0xf0
>[10376.080248]  [<c07a4333>] ata_scsi_port_error_handler+0x433/0x790
>[10376.080248]  [<c07a8b76>] ? ata_sff_flush_pio_task+0x16/0x60
>[10376.080248]  [<c07a4705>] ata_scsi_error+0x75/0xa0
>[10376.080248]  [<c077c63f>] scsi_error_handler+0xdf/0x610
>[10376.080248]  [<c047e590>] ? default_wake_function+0x10/0x20
>[10376.080248]  [<c04761b5>] ? __wake_up_common+0x45/0x70
>[10376.080248]  [<c047931e>] ? complete+0x4e/0x60
>[10376.080248]  [<c077c560>] ? scsi_eh_get_sense+0x1b0/0x1b0
>[10376.080248]  [<c046d534>] kthread+0x94/0xa0
>[10376.080248]  [<c0989937>] ret_from_kernel_thread+0x1b/0x28
>[10376.080248]  [<c046d4a0>] ? kthread_create_on_node+0xc0/0xc0
>[10376.080248] ---[ end trace 8435b7bed5fce0eb ]---
>[10400.127822] BUG: soft lockup - CPU#1 stuck for 22s! [swapper/1:0]
>[10400.268521] BUG: soft lockup - CPU#3 stuck for 23s! [swapper/3:0]
>[10400.268521] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver
>[10400.127822] Modules linked in:
>[10400.268521]  fscache
>[10400.127822]  nfsv3
>[10400.268521]  lockd
>[10400.127822]  nfs_acl
>[10400.268521]  sunrpc
>[10400.127822]  nfsv4
>[10400.268521]  ip6t_REJECT
>[10400.127822]  auth_rpcgss
>[10400.268521]  ip6table_filter
>[10400.127822]  nfs
>[10400.268521]  ip6_tables
>[10400.127822]  dns_resolver
>[10400.268521]  iptable_raw
>[10400.127822]  fscache
>[10400.268521]  e1000
>[10400.127822]  lockd
>[10400.268521]  i2c_i801
>[10400.127822]  sunrpc
>[10400.268521]  iTCO_wdt
>[10400.127822]  ip6t_REJECT
>[10400.268521]  iTCO_vendor_support
>[10400.127822]  ip6table_filter
>[10400.268521]  lpc_ich
>[10400.127822]  ip6_tables
>[10400.268521]  serio_raw
>[10400.127822]  iptable_raw
>[10400.268521]  i2c_core
>[10400.127822]  e1000
>[10400.268521]  mfd_core
>[10400.127822]  i2c_i801
>[10400.268521]  e7xxx_edac
>[10400.127822]  iTCO_wdt
>[10400.268521]  edac_core
>[10400.127822]  iTCO_vendor_support
>[10400.268521]  microcode
>[10400.127822]  lpc_ich
>[10400.268521]  binfmt_misc
>[10400.127822]  serio_raw
>[10400.268521]  uinput
>[10400.127822]  i2c_core
>[10400.268521]  sata_sil
>[10400.127822] 
>[10400.127822] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10400.127822] EIP: 0060:[<c043e474>] EFLAGS: 00000246 CPU: 1
>[10400.268521]  mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[10400.268521] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10400.127822] EIP is at native_safe_halt+0x4/0x10
>[10400.268521] EIP: 0060:[<c04857bb>] EFLAGS: 00000246 CPU: 3
>[10400.127822] EAX: 00000000 EBX: c0c321ec ECX: 00000000 EDX: 00000000
>[10400.127822] ESI: 00000000 EDI: 00000001 EBP: f70b7f64 ESP: f70b7f64
>[10400.268521] EIP is at update_sd_lb_stats+0x33b/0x480
>[10400.127822]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10400.268521] EAX: f70adb00 EBX: f7096e40 ECX: f7121e80 EDX: f7096e00
>[10400.127822] CR0: 8005003b CR2: b7470000 CR3: 31718000 CR4: 000007f0
>[10400.268521] ESI: f7121f24 EDI: 00000200 EBP: f7121e50 ESP: f7121de0
>[10400.127822] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10400.268521]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10400.127822] DR6: ffff0ff0 DR7: 00000400
>[10400.268521] CR0: 8005003b CR2: 452a57c0 CR3: 36451000 CR4: 000007f0
>[10400.127822] Process swapper/1 (pid: 0, ti=f70b6000 task=f70b8c90 task.ti=f70b6000)
>[10400.268521] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10400.127822] Stack:
>[10400.268521] DR6: ffff0ff0 DR7: 00000400
>[10400.268521] Process swapper/3 (pid: 0, ti=f7120000 task=f70ba5b0 task.ti=f70c2000)
>[10400.268521] Stack:
>[10400.268521]  f70b7f7c c0419486 00000000
>[10400.127822]  f7121f8c
>[10400.268521]  c0c321ec
>[10400.127822]  00000000
>[10400.268521]  00000000
>[10400.127822]  00000233
>[10400.268521]  00000000
>[10400.127822]  f7121e20
>[10400.268521]  f70b7f90
>[10400.127822]  00000200
>[10400.268521]  c041a086
>[10400.127822] 
>[10400.127822]  00000233 00000002
>[10400.268521]  70949314
>[10400.127822]  c0cdc0c0
>[10400.268521] 
>[10400.268521]  80eb97b9 f9cb5eee
>[10400.127822]  f7096e40
>[10400.268521]  f70b7fb4
>[10400.127822]  f7121e80
>[10400.268521]  c09736e7
>[10400.127822]  f7096e10
>[10400.268521]  00000000
>[10400.127822]  00000000
>[10400.268521]  00000000
>[10400.127822]  00000001
>[10400.268521]  00000000
>[10400.127822] 
>[10400.127822]  00000001 00000200
>[10400.268521]  00000000
>[10400.127822]  00000200
>[10400.268521] 
>[10400.268521]  9a89d653 f9cb5eee
>[10400.127822]  00000233
>[10400.268521]  0602080b
>[10400.127822]  00000200
>[10400.268521]  00000000
>[10400.127822]  00000001
>[10400.268521]  00000000
>[10400.127822]  00000400
>[10400.268521]  00000000
>[10400.127822]  00000001
>[10400.268521]  00000000
>[10400.127822] 
>[10400.127822] Call Trace:
>[10400.268521]  00000000 00000001 00000000
>[10400.268521] Call Trace:
>[10400.127822]  [<c0419486>] default_idle+0x46/0x190
>[10400.268521]  [<c048592b>] find_busiest_group+0x2b/0x420
>[10400.127822]  [<c041a086>] cpu_idle+0xb6/0xe0
>[10400.268521]  [<c048473b>] ? enqueue_entity+0xcb/0x4f0
>[10400.127822]  [<c09736e7>] start_secondary+0x250/0x255
>[10400.268521]  [<c047b9da>] ? update_rq_clock+0x16a/0x290
>[10400.268521] Code: 8d
>[10400.268521]  [<c0485db8>] load_balance+0x98/0x660
>[10400.268521]  bc 27 00 00
>[10400.268521]  [<c0486409>] rebalance_domains+0x89/0x150
>[10400.268521]  00 00 55
>[10400.268521]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[10400.268521]  89 e5 fa
>[10400.268521]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[10400.268521]  5d c3 8d
>[10400.268521]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[10400.268521]  76 00
>[10400.127822]  <IRQ> 
>[10400.268521]  8d
>[10400.268521]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[10400.268521]  bc 27
>[10400.268521]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10400.268521]  00 00
>[10400.268521]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10400.268521]  00 00
>[10400.268521]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10400.268521]  55 89
>[10400.268521]  [<c0419486>] ? default_idle+0x46/0x190
>[10400.268521]  e5 fb
>[10400.268521]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[10400.268521]  5d c3
>[10400.268521]  [<c09736e7>] ? start_secondary+0x250/0x255
>[10400.268521]  8d 76 00
>[10400.127822] Code:
>[10400.268521]  8d
>[10400.127822]  20
>[10400.268521]  bc
>[10400.127822]  00
>[10400.268521]  27
>[10400.127822]  00
>[10400.268521]  00
>[10400.127822]  00
>[10400.268521]  00
>[10400.127822]  89
>[10400.268521]  00
>[10400.127822]  c3
>[10400.268521]  00
>[10400.127822]  8b
>[10400.268521]  55
>[10400.127822]  45
>[10400.268521]  89
>[10400.127822]  b8
>[10400.268521]  e5
>[10400.127822]  e8
>[10400.268521]  fb
>[10400.127822]  a2
>[10400.268521]  <f4>
>[10400.127822]  d6
>[10400.268521]  5d
>[10400.127822]  20
>[10400.268521]  c3
>[10400.127822]  00
>[10400.268521]  89
>[10400.127822]  39
>[10400.268521]  f6
>[10400.127822]  c3
>[10400.268521]  8d
>[10400.127822]  0f
>[10400.268521]  bc
>[10400.127822]  87
>[10400.268521]  27
>[10400.127822]  3f
>[10400.268521]  00
>[10400.127822]  01
>[10400.268521]  00
>[10400.127822]  00
>[10400.268521]  00
>[10400.127822]  00
>[10400.268521]  00
>[10400.127822]  66
>[10400.268521]  55
>[10400.127822]  90
>[10400.268521]  89
>[10400.127822]  8b
>[10400.268521]  e5
>[10400.127822]  06
>[10400.268521]  f4
>[10400.127822]  8d
>[10400.268521]  5d
>[10400.127822]  b6
>[10400.268521]  c3
>[10400.127822]  00
>[10400.268521]  8d
>[10400.127822]  00
>[10400.268521]  76
>[10400.127822]  00
>[10400.268521]  00
>[10400.127822] 
>[10400.268521]  00 8b 5d b0 8b 1b 39 58 08 89 5d b0 <0f> 85 1f fd ff ff e9 94 fe ff ff 8b 5d d0 8b 4d d8 8b 45 e0 8b
>[10428.114108] BUG: soft lockup - CPU#1 stuck for 22s! [swapper/1:0]
>[10428.413759] BUG: soft lockup - CPU#3 stuck for 22s! [swapper/3:0]
>[10428.413759] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core
>[10428.114108] Modules linked in:
>[10428.413759]  mfd_core
>[10428.114108]  nfsv3
>[10428.413759]  e7xxx_edac
>[10428.114108]  nfs_acl
>[10428.413759]  edac_core
>[10428.114108]  nfsv4
>[10428.413759]  microcode
>[10428.114108]  auth_rpcgss
>[10428.413759]  binfmt_misc
>[10428.114108]  nfs
>[10428.413759]  uinput
>[10428.114108]  dns_resolver
>[10428.413759]  sata_sil
>[10428.114108] 
>[10428.114108] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10428.114108] EIP: 0060:[<c098255f>] EFLAGS: 00000286 CPU: 1
>[10428.114108] EIP is at _raw_spin_unlock_irqrestore+0xf/0x20
>[10428.114108] EAX: 00000286 EBX: f1704a80 ECX: 00000400 EDX: 00000286
>[10428.114108] ESI: f75c90c0 EDI: f75c9534 EBP: f70d9f3c ESP: f70d9f3c
>[10428.114108]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10428.114108] CR0: 8005003b CR2: b7470000 CR3: 31718000 CR4: 000007f0
>[10428.114108] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10428.114108] DR6: ffff0ff0 DR7: 00000400
>[10428.114108] Process swapper/1 (pid: 0, ti=f70d8000 task=f70b8c90 task.ti=f70b6000)
>[10428.114108] Stack:
>[10428.114108]  fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core
>[10428.413759]  f70d9f68
>[10428.114108]  e7xxx_edac
>[10428.413759]  c0483c73
>[10428.114108]  edac_core
>[10428.413759]  00000004
>[10428.114108]  microcode
>[10428.413759]  c0cdc0c0
>[10428.114108]  binfmt_misc
>[10428.413759]  00000286
>[10428.114108]  uinput
>[10428.413759]  00000001
>[10428.114108]  sata_sil
>[10428.413759] 
>[10428.413759] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10428.114108]  f1704a80 f1704acc
>[10428.413759] EIP: 0060:[<c098255f>] EFLAGS: 00000286 CPU: 3
>[10428.413759] EIP is at _raw_spin_unlock_irqrestore+0xf/0x20
>[10428.413759] EAX: 00000286 EBX: efc4d000 ECX: 00000400 EDX: 00000286
>[10428.413759] ESI: f75e50c0 EDI: f75e5534 EBP: f7121f3c ESP: f7121f3c
>[10428.413759]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10428.413759] CR0: 8005003b CR2: 452a57c0 CR3: 36451000 CR4: 000007f0
>[10428.413759] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10428.413759] DR6: ffff0ff0 DR7: 00000400
>[10428.114108]  c0cdc0c0 00000001 009b7485 f70d9f9c c04863bf ffffff10 c045cd45 f75c90c0
>[10428.413759] Process swapper/3 (pid: 0, ti=f7120000 task=f70ba5b0 task.ti=f70c2000)
>[10428.413759] Stack:
>[10428.413759]  00000001 00000001
>[10428.114108]  f7121f68
>[10428.413759]  00000001
>[10428.114108]  c0483c73
>[10428.413759]  00000000
>[10428.114108]  0000000c
>[10428.413759]  00000001
>[10428.114108]  c0cdc0c0
>[10428.413759]  c0cdc0c0
>[10428.114108]  00000286
>[10428.413759]  00000001
>[10428.114108]  00000003
>[10428.413759]  00000001
>[10428.114108] 
>[10428.114108] Call Trace:
>[10428.413759]  efc4d000 efc4d04c
>[10428.114108]  [<c0483c73>] update_shares+0xa3/0xd0
>[10428.114108]  c0cdc0c0 00000003
>[10428.114108]  [<c04863bf>] rebalance_domains+0x3f/0x150
>[10428.114108]  009b7594 f7121f9c
>[10428.114108]  [<c045cd45>] ? run_timer_softirq+0x125/0x210
>[10428.114108]  c04863bf 00000001
>[10428.114108]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[10428.114108]  f70d4000 f75e50c0
>[10428.413759] 
>[10428.114108]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[10428.114108]  00000003
>[10428.114108]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[10428.114108]  00000001 00000001
>[10428.413759]  <IRQ> 
>[10428.114108]  00000000
>[10428.114108]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[10428.114108]  00000001 c0cdc0c0
>[10428.114108]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10428.114108]  00000003 00000001
>[10428.413759] 
>[10428.114108]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10428.413759] Call Trace:
>[10428.114108]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10428.413759]  [<c0483c73>] update_shares+0xa3/0xd0
>[10428.114108]  [<c0419486>] ? default_idle+0x46/0x190
>[10428.413759]  [<c04863bf>] rebalance_domains+0x3f/0x150
>[10428.114108]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[10428.413759]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[10428.114108]  [<c09736e7>] ? start_secondary+0x250/0x255
>[10428.413759]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[10428.413759] Code: ce
>[10428.413759]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[10428.413759]  38 d1
>[10428.114108]  <IRQ> 
>[10428.413759]  74
>[10428.413759]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[10428.413759]  10 8d
>[10428.413759]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10428.413759]  b4 26
>[10428.413759]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10428.413759]  00 00 00
>[10428.413759]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10428.413759]  00 f3
>[10428.413759]  [<c0419486>] ? default_idle+0x46/0x190
>[10428.413759]  90 0f
>[10428.413759]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[10428.413759]  b6 10
>[10428.413759]  [<c09736e7>] ? start_secondary+0x250/0x255
>[10428.413759]  38 ca
>[10428.114108] Code:
>[10428.413759]  75
>[10428.114108]  ce
>[10428.413759]  f7
>[10428.114108]  38
>[10428.413759]  5d
>[10428.114108]  d1
>[10428.413759]  c3
>[10428.114108]  74
>[10428.413759]  90
>[10428.114108]  10
>[10428.413759]  8d
>[10428.114108]  8d
>[10428.413759]  74
>[10428.114108]  b4
>[10428.413759]  26
>[10428.114108]  26
>[10428.413759]  00
>[10428.114108]  00
>[10428.413759]  55
>[10428.114108]  00
>[10428.413759]  89
>[10428.114108]  00
>[10428.413759]  e5
>[10428.114108]  00
>[10428.413759]  3e
>[10428.114108]  f3
>[10428.413759]  8d
>[10428.114108]  90
>[10428.413759]  74
>[10428.114108]  0f
>[10428.413759]  26
>[10428.114108]  b6
>[10428.413759]  00
>[10428.114108]  10
>[10428.413759]  80
>[10428.114108]  38
>[10428.413759]  00
>[10428.114108]  ca
>[10428.413759]  01
>[10428.114108]  75
>[10428.413759]  89
>[10428.114108]  f7
>[10428.413759]  d0
>[10428.114108]  5d
>[10428.413759]  50
>[10428.114108]  c3
>[10428.413759]  9d
>[10428.114108]  90
>[10428.413759]  <8d>
>[10428.114108]  8d
>[10428.413759]  74
>[10428.114108]  74
>[10428.413759]  26
>[10428.114108]  26
>[10428.413759]  00
>[10428.114108]  00
>[10428.413759]  5d
>[10428.114108]  55
>[10428.413759]  c3
>[10428.114108]  89
>[10428.413759]  8d
>[10428.114108]  e5
>[10428.413759]  74
>[10428.114108]  3e
>[10428.413759]  26
>[10428.114108]  8d
>[10428.413759]  00
>[10428.114108]  74
>[10428.413759]  8d
>[10428.114108]  26
>[10428.413759]  bc
>[10428.114108]  00
>[10428.413759]  27
>[10428.114108]  80
>[10428.413759]  00
>[10428.114108]  00
>[10428.413759]  00
>[10428.114108]  01
>[10428.413759]  00
>[10428.114108]  89
>[10428.413759]  00
>[10428.114108]  d0
>[10428.413759]  55
>[10428.114108]  50
>[10428.413759]  89
>[10428.114108]  9d
>[10428.413759]  e5
>[10428.114108]  <8d>
>[10428.413759]  3e
>[10428.114108] 
>[10428.413759]  74 26 00 5d c3 8d 74 26 00 8d bc 27 00 00 00 00 55 89 e5 3e
>[10435.887513] INFO: rcu_sched detected stalls on CPUs/tasks: { 0} (detected by 3, t=60161 jiffies)
>[10435.887513] sending NMI to all CPUs:
>[10376.080248] NMI backtrace for cpu 2
>[10376.080248] Pid: 192, comm: scsi_eh_2 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10376.080248] EIP: 0060:[<c0693c22>] EFLAGS: 00000096 CPU: 2
>[10376.080248] EIP is at ioread8+0x32/0x40
>[10376.080248] EAX: f7e240ff EBX: f6424000 ECX: c07a6070 EDX: f7e2a140
>[10376.080248] ESI: 0000568e EDI: c07a6740 EBP: f641de4c ESP: f641de48
>[10376.080248]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10376.080248] CR0: 8005003b CR2: 0a08989c CR3: 3656a000 CR4: 000007f0
>[10376.080248] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10376.080248] DR6: ffff0ff0 DR7: 00000400
>[10376.080248] Process scsi_eh_2 (pid: 192, ti=f641c000 task=f176d7f0 task.ti=f641c000)
>[10376.080248] Stack:
>[10376.080248]  c07a5870 f641de6c c07a60ae c0982de0 f6424000 00000000 00000202 f6424000
>[10376.080248]  c07a6a80 f641de90 c07a6dc7 0141007b c07a007b f64240c4 00000202 f6424000
>[10376.080248]  f64240c4 00000202 f641dea4 c07a6e93 00000000 00000001 f64256d4 f641deec
>[10376.080248] Call Trace:
>[10376.080248]  [<c07a5870>] ? ata_sff_check_status+0x10/0x20
>[10376.080248]  [<c07a60ae>] ata_sff_drain_fifo+0x3e/0x80
>[10376.080248]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10376.080248]  [<c07a6a80>] ? ata_pio_sectors+0xa0/0xa0
>[10376.080248]  [<c07a6dc7>] ata_sff_error_handler+0x67/0xe0
>[10376.080248]  [<c07a007b>] ? ata_eh_acquire+0x2b/0x60
>[10376.080248]  [<c07a6e93>] ata_bmdma_error_handler+0x53/0xf0
>[10376.080248]  [<c07a4333>] ata_scsi_port_error_handler+0x433/0x790
>[10376.080248]  [<c07a8b76>] ? ata_sff_flush_pio_task+0x16/0x60
>[10376.080248]  [<c07a4705>] ata_scsi_error+0x75/0xa0
>[10376.080248]  [<c077c63f>] scsi_error_handler+0xdf/0x610
>[10376.080248]  [<c047e590>] ? default_wake_function+0x10/0x20
>[10376.080248]  [<c04761b5>] ? __wake_up_common+0x45/0x70
>[10376.080248]  [<c047931e>] ? complete+0x4e/0x60
>[10376.080248]  [<c077c560>] ? scsi_eh_get_sense+0x1b0/0x1b0
>[10376.080248]  [<c046d534>] kthread+0x94/0xa0
>[10376.080248]  [<c0989937>] ret_from_kernel_thread+0x1b/0x28
>[10376.080248]  [<c046d4a0>] ? kthread_create_on_node+0xc0/0xc0
>[10376.080248] Code: 3d 00 00 01 00 76 0a 0f b7 d0 ec 0f b6 c0 c3 66 90 55 ba 42 55 b2 c0 89 e5 e8 cb fd ff ff b8 ff 00 00 00 5d c3 8d 74 26 00 8a 00 <0f> b6 c0 c3 8d 76 00 8d bc 27 00 00 00 00 55 3d ff ff 03 00 89
>[10436.097407] NMI backtrace for cpu 1
>[10436.097407] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10436.097407] EIP: 0060:[<c048177f>] EFLAGS: 00000246 CPU: 1
>[10436.097407] EIP is at tg_load_down+0x3f/0x80
>[10436.097407] EAX: 00000000 EBX: f640d600 ECX: 00000000 EDX: 00000000
>[10436.097407] ESI: 0000000c EDI: 00001283 EBP: f70d9eb4 ESP: f70d9ea8
>[10436.097407]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10436.097407] CR0: 8005003b CR2: b7470000 CR3: 31718000 CR4: 000007f0
>[10436.097407] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10436.097407] DR6: ffff0ff0 DR7: 00000400
>[10436.097407] Process swapper/1 (pid: 0, ti=f70d8000 task=f70b8c90 task.ti=f70b6000)
>[10436.097407] Stack:
>[10436.097407]  f640d600 c0481740 00000000 f70d9ed4 c047be3f c0475fe0 c0d36b40 f640d6a0
>[10436.097407]  f75e50c0 00000043 c0cdc0c0 f70d9f68 c0485f21 00000003 0000097d 1dcd6500
>[10436.097407]  00000000 00000001 f75c4930 00000001 f015f000 c0cdc0c0 f70ad100 007b2e0f
>[10436.097407] Call Trace:
>[10436.097407]  [<c0481740>] ? task_waking_fair+0x40/0x40
>[10436.097407]  [<c047be3f>] walk_tg_tree_from+0x1f/0x90
>[10436.097407]  [<c0475fe0>] ? smpboot_park_threads+0x60/0x60
>[10436.097407]  [<c0485f21>] load_balance+0x201/0x660
>[10436.097407]  [<c0486409>] rebalance_domains+0x89/0x150
>[10436.097407]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[10436.097407]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[10436.097407]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[10436.097407]  <IRQ> 
>[10436.097407]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[10436.097407]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10436.097407]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10436.097407]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10436.097407]  [<c0419486>] ? default_idle+0x46/0x190
>[10436.097407]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[10436.097407]  [<c09736e7>] ? start_secondary+0x250/0x255
>[10436.097407] Code: 89 c3 8b 80 9c 00 00 00 85 c0 74 40 8b 40 24 8d 34 95 00 00 00 00 8b 7b 20 8b 04 90 8b 14 97 8b 48 58 0f af 0a 31 d2 8b 38 89 c8 <83> c7 01 f7 f7 8b 53 24 8b 14 32 89 42 58 8b 5d f4 31 c0 8b 75
>[10435.887513] NMI backtrace for cpu 3
>[10435.887513] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10435.887513] EIP: 0060:[<c068d880>] EFLAGS: 00000006 CPU: 3
>[10435.887513] EIP is at __const_udelay+0x0/0x20
>[10435.887513] EAX: 00418958 EBX: 00002710 ECX: fffff000 EDX: fffff000
>[10435.887513] ESI: c0bd8a00 EDI: 00000001 EBP: f7121d5c ESP: f7121d50
>[10435.887513]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10437.420154] CR0: 8005003b CR2: 452a57c0 CR3: 36451000 CR4: 000007f0
>[10437.420154] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10437.420154] DR6: ffff0ff0 DR7: 00000400
>[10437.420154] Process swapper/3 (pid: 0, ti=f7120000 task=f70ba5b0 task.ti=f70c2000)
>[10437.420154] Stack:
>[10437.420154]  c0435a2a c0ae75ef f75e0ba0 f7121d90 c04d5550 c0afef44 00000003 0000eb01
>[10437.420154]  00000000 00000083 00000001 00000003 c0bd8a00 f70ba5b0 00000000 00000003
>[10437.420154]  f7121da4 c045d6ab f75e0980 d4ad5718 0000097d f7121dc4 c04a1208 f7121dc4
>[10437.420154] Call Trace:
>[10437.420154]  [<c0435a2a>] ? arch_trigger_all_cpu_backtrace+0x5a/0x80
>[10437.420154]  [<c04d5550>] rcu_check_callbacks+0x4b0/0x540
>[10437.420154]  [<c045d6ab>] update_process_times+0x3b/0x70
>[10437.420154]  [<c04a1208>] tick_sched_timer+0x68/0xf0
>[10437.420154]  [<c0471370>] ? __remove_hrtimer+0x40/0xa0
>[10437.420154]  [<c047152d>] __run_hrtimer+0x6d/0x190
>[10437.420154]  [<c04a11a0>] ? tick_nohz_handler+0x100/0x100
>[10437.420154]  [<c04721bf>] hrtimer_interrupt+0xef/0x280
>[10437.420154]  [<c0434689>] smp_apic_timer_interrupt+0x59/0x90
>[10437.420154]  [<c048592b>] ? find_busiest_group+0x2b/0x420
>[10437.420154]  [<c0982de0>] apic_timer_interrupt+0x34/0x3c
>[10437.420154]  [<c048177f>] ? tg_load_down+0x3f/0x80
>[10437.420154]  [<c0481740>] ? task_waking_fair+0x40/0x40
>[10437.420154]  [<c047be3f>] walk_tg_tree_from+0x1f/0x90
>[10437.420154]  [<c0475fe0>] ? smpboot_park_threads+0x60/0x60
>[10437.420154]  [<c0485f21>] load_balance+0x201/0x660
>[10437.420154]  [<c0486409>] rebalance_domains+0x89/0x150
>[10437.420154]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[10437.420154]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[10437.420154]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[10437.420154]  <IRQ> 
>[10437.420154]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[10437.420154]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10437.420154]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10437.420154]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10437.420154]  [<c0419486>] ? default_idle+0x46/0x190
>[10437.420154]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[10437.420154]  [<c09736e7>] ? start_secondary+0x250/0x255
>[10437.420154] Code: 00 00 00 00 8d bc 27 00 00 00 00 48 75 fd 48 5d c3 8d 76 00 8d bc 27 00 00 00 00 55 89 e5 3e 8d 74 26 00 ff 15 c0 fa bf c0 5d c3 <55> 89 e5 64 8b 15 dc bf cd c0 c1 e0 02 69 d2 fa 00 00 00 f7 e2
>[10440.719519] ------------[ cut here ]------------
>[10440.719519] WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0xac/0xd0()
>[10440.719519] Hardware name: Sun Fire(tm) V60 370-6037               
>[10440.719519] Watchdog detected hard LOCKUP on cpu 1
>[10440.719519] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[10440.719519] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1
>[10440.719519] Call Trace:
>[10440.719519]  [<c044d6a2>] warn_slowpath_common+0x72/0xa0
>[10440.719519]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[10440.719519]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[10440.719519]  [<c04ccb60>] ? touch_nmi_watchdog+0x70/0x70
>[10440.719519]  [<c044d773>] warn_slowpath_fmt+0x33/0x40
>[10440.719519]  [<c04ccc0c>] watchdog_overflow_callback+0xac/0xd0
>[10440.719519]  [<c04ff986>] __perf_event_overflow+0xa6/0x270
>[10440.719519]  [<c042021f>] ? x86_perf_event_set_period+0x11f/0x230
>[10440.719519]  [<c05004f5>] perf_event_overflow+0x15/0x20
>[10440.719519]  [<c0422033>] p4_pmu_handle_irq+0x143/0x1f0
>[10440.719519]  [<c04184d8>] ? sched_clock+0x8/0x10
>[10440.719519]  [<c098424b>] perf_event_nmi_handler+0x1b/0x20
>[10440.719519]  [<c09839c9>] nmi_handle.isra.0+0x39/0x60
>[10440.719519]  [<c0983bcf>] do_nmi+0x1df/0x3f0
>[10440.719519]  [<c0455bb7>] ? irq_exit+0x67/0xb0
>[10440.719519]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10440.719519]  [<c0983107>] nmi_stack_correct+0x2f/0x34
>[10440.719519]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10440.719519]  [<c0419486>] default_idle+0x46/0x190
>[10440.719519]  [<c041a086>] cpu_idle+0xb6/0xe0
>[10440.719519]  [<c09736e7>] start_secondary+0x250/0x255
>[10440.719519] ---[ end trace 8435b7bed5fce0ec ]---
>[10437.420154] ------------[ cut here ]------------
>[10437.420154] WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0xac/0xd0()
>[10437.420154] Hardware name: Sun Fire(tm) V60 370-6037               
>[10437.420154] Watchdog detected hard LOCKUP on cpu 3
>[10437.420154] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[10437.420154] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1
>[10437.420154] Call Trace:
>[10437.420154]  [<c044d6a2>] warn_slowpath_common+0x72/0xa0
>[10437.420154]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[10437.420154]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[10437.420154]  [<c04ccb60>] ? touch_nmi_watchdog+0x70/0x70
>[10437.420154]  [<c044d773>] warn_slowpath_fmt+0x33/0x40
>[10437.420154]  [<c04ccc0c>] watchdog_overflow_callback+0xac/0xd0
>[10437.420154]  [<c04ff986>] __perf_event_overflow+0xa6/0x270
>[10437.420154]  [<c042021f>] ? x86_perf_event_set_period+0x11f/0x230
>[10437.420154]  [<c05004f5>] perf_event_overflow+0x15/0x20
>[10437.420154]  [<c0422033>] p4_pmu_handle_irq+0x143/0x1f0
>[10437.420154]  [<c042207c>] ? p4_pmu_handle_irq+0x18c/0x1f0
>[10437.420154]  [<c098424b>] perf_event_nmi_handler+0x1b/0x20
>[10437.420154]  [<c09839c9>] nmi_handle.isra.0+0x39/0x60
>[10437.420154]  [<c0983ad8>] do_nmi+0xe8/0x3f0
>[10437.420154]  [<c0983107>] nmi_stack_correct+0x2f/0x34
>[10437.420154]  [<c098007b>] ? console_conditional_schedule+0x2b/0x30
>[10437.420154]  [<c068d932>] ? delay_tsc+0x32/0x60
>[10437.420154]  [<c068d89e>] __const_udelay+0x1e/0x20
>[10437.420154]  [<c0435a2a>] arch_trigger_all_cpu_backtrace+0x5a/0x80
>[10437.420154]  [<c04d5550>] rcu_check_callbacks+0x4b0/0x540
>[10437.420154]  [<c045d6ab>] update_process_times+0x3b/0x70
>[10437.420154]  [<c04a1208>] tick_sched_timer+0x68/0xf0
>[10437.420154]  [<c0471370>] ? __remove_hrtimer+0x40/0xa0
>[10437.420154]  [<c047152d>] __run_hrtimer+0x6d/0x190
>[10437.420154]  [<c04a11a0>] ? tick_nohz_handler+0x100/0x100
>[10437.420154]  [<c04721bf>] hrtimer_interrupt+0xef/0x280
>[10437.420154]  [<c0434689>] smp_apic_timer_interrupt+0x59/0x90
>[10437.420154]  [<c048592b>] ? find_busiest_group+0x2b/0x420
>[10437.420154]  [<c0982de0>] apic_timer_interrupt+0x34/0x3c
>[10437.420154]  [<c048177f>] ? tg_load_down+0x3f/0x80
>[10437.420154]  [<c0481740>] ? task_waking_fair+0x40/0x40
>[10437.420154]  [<c047be3f>] walk_tg_tree_from+0x1f/0x90
>[10437.420154]  [<c0475fe0>] ? smpboot_park_threads+0x60/0x60
>[10437.420154]  [<c0485f21>] load_balance+0x201/0x660
>[10437.420154]  [<c0486409>] rebalance_domains+0x89/0x150
>[10437.420154]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[10437.420154]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[10437.420154]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[10437.420154]  <IRQ>  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[10437.420154]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10437.420154]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10437.420154]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10437.420154]  [<c0419486>] ? default_idle+0x46/0x190
>[10437.420154]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[10437.420154]  [<c09736e7>] ? start_secondary+0x250/0x255
>[10437.420154] ---[ end trace 8435b7bed5fce0ed ]---
>[10460.386066] BUG: soft lockup - CPU#3 stuck for 21s! [swapper/3:0]
>[10460.386066] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[10460.386066] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10460.386066] EIP: 0060:[<c048555b>] EFLAGS: 00000297 CPU: 3
>[10460.386066] EIP is at update_sd_lb_stats+0xdb/0x480
>[10460.386066] EAX: 00000003 EBX: 00000003 ECX: 00000000 EDX: ffffffff
>[10460.386066] ESI: f7121f24 EDI: f7096c50 EBP: f7121e50 ESP: f7121de0
>[10460.386066]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10460.386066] CR0: 8005003b CR2: 452a57c0 CR3: 36451000 CR4: 000007f0
>[10460.386066] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10460.386066] DR6: ffff0ff0 DR7: 00000400
>[10460.386066] Process swapper/3 (pid: 0, ti=f7120000 task=f70ba5b0 task.ti=f70c2000)
>[10460.386066] Stack:
>[10460.386066]  f7121f8c 00000000 00000233 f7121e20 00000003 00000000 00000000 c0cdc0c0
>[10460.386066]  f7096c40 f7121e80 f7096c50 ffffffff ffffffff 00000000 ffffffff 00000000
>[10460.386066]  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
>[10460.386066] Call Trace:
>[10460.386066]  [<c048592b>] find_busiest_group+0x2b/0x420
>[10460.386066]  [<c0485db8>] load_balance+0x98/0x660
>[10460.386066]  [<c0486409>] rebalance_domains+0x89/0x150
>[10460.386066]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[10460.386066]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[10460.386066]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[10460.386066]  <IRQ> 
>[10460.386066]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[10460.386066]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10460.386066]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10460.386066]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10460.386066]  [<c0419486>] ? default_idle+0x46/0x190
>[10460.386066]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[10460.386066]  [<c09736e7>] ? start_secondary+0x250/0x255
>[10460.386066] Code: ff c7 45 c4 00 00 00 00 89 7d b8 90 8d 74 26 00 8b 4e 24 89 d8 8b 55 b8 e8 e3 e0 1f 00 3b 05 a4 2a c3 c0 89 c3 0f 8d dd 00 00 00 <8b> 04 9d c0 1f c3 c0 8b 55 ac 8b 7c 02 04 8b 45 bc 85 c0 74 78
>[10468.116929] BUG: soft lockup - CPU#1 stuck for 23s! [swapper/1:0]
>[10468.116929] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[10468.116929] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10468.116929] EIP: 0060:[<c045cd45>] EFLAGS: 00000286 CPU: 1
>[10468.116929] EIP is at run_timer_softirq+0x125/0x210
>[10468.116929] EAX: 009b26dd EBX: f70d9fa4 ECX: 009b26de EDX: f70d06e8
>[10468.116929] ESI: 00000001 EDI: f70d0000 EBP: f70d9fb8 ESP: f70d9f80
>[10468.116929]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10468.116929] CR0: 8005003b CR2: b7470000 CR3: 31718000 CR4: 000007f0
>[10468.116929] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10468.116929] DR6: ffff0ff0 DR7: 00000400
>[10468.116929] Process swapper/1 (pid: 0, ti=f70d8000 task=f70b8c90 task.ti=f70b6000)
>[10468.116929] Stack:
>[10468.116929]  00000001 00000001 00000000 f70d0a14 f70d0814 00000001 00000001 f70d9fb8
>[10468.116929]  f70d9fa4 f70d9fa4 f70d9fa4 00000082 00000001 c0bb8a04 f70d9ff8 c045596f
>[10468.116929]  00000000 00000000 00000000 00000000 00000000 00000000 00200040 00000001
>[10468.116929] Call Trace:
>[10468.116929]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[10468.116929]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[10468.116929]  <IRQ> 
>[10468.116929]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[10468.116929]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10468.116929]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10468.116929]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10468.116929]  [<c0419486>] ? default_idle+0x46/0x190
>[10468.116929]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[10468.116929]  [<c09736e7>] ? start_secondary+0x250/0x255
>[10468.116929] Code: 89 f8 e8 ff 57 52 00 8b 5d ec 3b 5d e8 75 8f a1 40 8a bb c0 8b 4f 08 39 c8 0f 89 3f ff ff ff c7 47 04 00 00 00 00 80 07 01 fb 90 <8d> 74 26 00 83 c4 2c 5b 5e 5f 5d c3 8d b4 26 00 00 00 00 8b 35
>[10488.305479] BUG: soft lockup - CPU#3 stuck for 22s! [swapper/3:0]
>[10488.305479] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[10488.305479] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10488.305479] EIP: 0060:[<c0481784>] EFLAGS: 00000202 CPU: 3
>[10488.305479] EIP is at tg_load_down+0x44/0x80
>[10488.305479] EAX: 00000001 EBX: f6406a00 ECX: 00000402 EDX: 00000001
>[10488.305479] ESI: 00000004 EDI: 00000401 EBP: f7121eb4 ESP: f7121ea8
>[10488.305479]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10488.305479] CR0: 8005003b CR2: 452a57c0 CR3: 36451000 CR4: 000007f0
>[10488.305479] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10488.305479] DR6: ffff0ff0 DR7: 00000400
>[10488.305479] Process swapper/3 (pid: 0, ti=f7120000 task=f70ba5b0 task.ti=f70c2000)
>[10488.305479] Stack:
>[10488.305479]  f6406a00 c0481740 00000000 f7121ed4 c047be3f c0475fe0 c0d36b40 f6406aa0
>[10488.305479]  f75c90c0 0000002c c0cdc0c0 f7121f68 c0485f21 00000001 eb4d2f1c 00000989
>[10488.305479]  f75e0838 00000003 f75e0930 00000001 f0d6b200 c0cdc0c0 f70ad300 004e0815
>[10488.305479] Call Trace:
>[10488.305479]  [<c0481740>] ? task_waking_fair+0x40/0x40
>[10488.305479]  [<c047be3f>] walk_tg_tree_from+0x1f/0x90
>[10488.305479]  [<c0475fe0>] ? smpboot_park_threads+0x60/0x60
>[10488.305479]  [<c0485f21>] load_balance+0x201/0x660
>[10488.305479]  [<c0486409>] rebalance_domains+0x89/0x150
>[10488.305479]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[10488.305479]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[10488.305479]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[10488.305479]  <IRQ> 
>[10488.305479]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[10488.305479]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[10488.305479]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[10488.305479]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[10488.305479]  [<c0419486>] ? default_idle+0x46/0x190
>[10488.305479]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[10488.305479]  [<c09736e7>] ? start_secondary+0x250/0x255
>[10488.305479] Code: 00 00 00 85 c0 74 40 8b 40 24 8d 34 95 00 00 00 00 8b 7b 20 8b 04 90 8b 14 97 8b 48 58 0f af 0a 31 d2 8b 38 89 c8 83 c7 01 f7 f7 <8b> 53 24 8b 14 32 89 42 58 8b 5d f4 31 c0 8b 75 f8 8b 7d fc 89
>[10376.080248] ata3: drained 65536 bytes to clear DRQ
>[10492.949979] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x6 frozen
>[10492.949983] ata3: SError: { Handshk UnrecFIS }
>[10492.949987] ata3.00: failed command: READ DMA EXT
>[10492.949996] ata3.00: cmd 25/00:00:00:57:08/00:01:14:00:00/e0 tag 0 dma 131072 in
>[10492.949996]          res ff/ff:ff:ff:ff:ff/ff:ff:ff:ff:ff/ff Emask 0x2 (HSM violation)
>[10492.949998] ata3.00: status: { Busy }
>[10492.950000] ata3.00: error: { ICRC UNC IDNF ABRT }
>[10492.950021] ata3: hard resetting link
>[10492.981371] Do you have a strange power saving mode enabled?
>[10492.981371] Dazed and confused, but trying to continue
>[10492.981371] NMI backtrace for cpu 0
>[10492.981371] Pid: 0, comm: swapper/0 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10492.981371] EIP: 0060:[<c085b20f>] EFLAGS: 00000057 CPU: 0
>[10492.981371] EIP is at acpi_pm_read+0xf/0x20
>[10492.981371] EAX: 00c41ace EBX: c0c159c0 ECX: c0bb3f48 EDX: 00000408
>[10492.981371] ESI: f75b67b0 EDI: 00f074aa EBP: c0bb3e64 ESP: c0bb3e64
>[10492.981371]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10492.981371] CR0: 8005003b CR2: b7763600 CR3: 36451000 CR4: 000007f0
>[10492.981371] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10492.981371] DR6: ffff0ff0 DR7: 00000400
>[10492.981371] Process swapper/0 (pid: 0, ti=c0bb2000 task=c0bbf220 task.ti=c0bb2000)
>[10492.981371] Stack:
>[10492.981371]  c0bb3e80 c04990be 00002887 00000000 f75b6980 f75b67b0 00000001 c0bb3ea0
>[10492.981371]  c04a11c5 c0bb3ea0 c0471370 c0bb3f48 f75b6980 f75b67b0 00000001 c0bb3ed0
>[10492.981371]  c047152d 00000000 f75b6838 00000017 c04a11a0 f75b67b0 f75b6780 c0bb3f14
>[10492.981371] Call Trace:
>[10492.981371]  [<c04990be>] ktime_get+0x5e/0x100
>[10492.981371]  [<c04a11c5>] tick_sched_timer+0x25/0xf0
>[10492.981371]  [<c0471370>] ? __remove_hrtimer+0x40/0xa0
>[10492.981371]  [<c047152d>] __run_hrtimer+0x6d/0x190
>[10492.981371]  [<c04a11a0>] ? tick_nohz_handler+0x100/0x100
>[10492.981371]  [<c04721bf>] hrtimer_interrupt+0xef/0x280
>[10492.981371]  [<c04806c7>] ? sched_clock_cpu+0xe7/0x190
>[10492.981371]  [<c047c325>] ? check_preempt_curr+0x65/0x90
>[10492.981371]  [<c0434689>] smp_apic_timer_interrupt+0x59/0x90
>[10492.981371]  [<c047c98e>] ? scheduler_ipi+0x4e/0xe0
>[10492.981371]  [<c0982de0>] apic_timer_interrupt+0x34/0x3c
>[10492.981371]  [<c043e475>] ? native_safe_halt+0x5/0x10
>[10492.981371]  [<c0419486>] default_idle+0x46/0x190
>[10492.981371]  [<c041a086>] cpu_idle+0xb6/0xe0
>[10492.981371]  [<c096113d>] rest_init+0x5d/0x60
>[10492.981371]  [<c0c36a14>] start_kernel+0x36d/0x373
>[10492.981371]  [<c0c364dc>] ? repair_env_string+0x51/0x51
>[10492.981371]  [<c0c362f8>] i386_start_kernel+0x9b/0xa2
>[10492.981371] Code: 04 00 10 00 00 89 14 24 e8 df 21 e3 ff 83 c4 14 5b 5d c3 66 0f 1f 84 00 00 00 00 00 55 89 e5 3e 8d 74 26 00 8b 15 40 3c c3 c0 ed <31> d2 25 ff ff ff 00 5d c3 90 8d b4 26 00 00 00 00 55 89 e5 57
>[10494.481075] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
>[10494.500422] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10494.503672] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10494.503680] ata3.00: configured for UDMA/33
>[10494.503706] sd 2:0:0:0: [sda]  
>[10494.503709] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10494.503712] sd 2:0:0:0: [sda]  
>[10494.503715] Sense Key : Aborted Command [current] [descriptor]
>[10494.503720] Descriptor sense data with sense descriptors (in hex):
>[10494.503722]         72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 ff ff 
>[10494.503736]         ff ff ff ff 
>[10494.503743] sd 2:0:0:0: [sda]  
>[10494.503748] Add. Sense: Scsi parity error
>[10494.503751] sd 2:0:0:0: [sda] CDB: 
>[10494.503753] Read(10): 28 00 14 08 57 00 00 01 00 00
>[10494.503766] end_request: I/O error, dev sda, sector 336090880
>[10494.518966] ata3: EH complete
>[10494.574959] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10496.441715] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10496.459454] ata4.00: BMDMA2 stat 0x6d0009
>[10496.478300] ata4: SError: { Handshk UnrecFIS }
>[10496.498126] ata4.00: failed command: READ DMA EXT
>[10496.517980] ata4.00: cmd 25/00:00:00:ce:f2/00:01:26:00:00/e0 tag 0 dma 131072 in
>[10496.517980]          res 51/04:df:21:ce:f2/00:00:26:00:00/e0 Emask 0x1 (device error)
>[10496.588647] ata4.00: status: { DRDY ERR }
>[10496.614203] ata4.00: error: { ABRT }
>[10496.661680] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10496.666613] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10496.666624] ata4.00: configured for UDMA/33
>[10496.666654] sd 3:0:0:0: [sdb]  
>[10496.666658] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10496.666661] sd 3:0:0:0: [sdb]  
>[10496.666664] Sense Key : Aborted Command [current] [descriptor]
>[10496.666669] Descriptor sense data with sense descriptors (in hex):
>[10496.666671]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10496.666685]         26 f2 ce 21 
>[10496.666692] sd 3:0:0:0: [sdb]  
>[10496.666695] Add. Sense: No additional sense information
>[10496.666698] sd 3:0:0:0: [sdb] CDB: 
>[10496.666701] Read(10): 28 00 26 f2 ce 00 00 01 00 00
>[10496.666714] end_request: I/O error, dev sdb, sector 653446656
>[10496.682372] ata4: EH complete
>[10497.533960] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10497.564424] ata4.00: BMDMA2 stat 0x650001
>[10497.589381] ata4: SError: { Handshk UnrecFIS }
>[10497.613486] ata4.00: failed command: WRITE DMA EXT
>[10497.637805] ata4.00: cmd 35/00:00:00:50:ff/00:04:0a:00:00/e0 tag 0 dma 524288 out
>[10497.637805]          res 51/04:60:a0:50:ff/00:03:0a:00:00/e0 Emask 0x1 (device error)
>[10497.746962] ata4.00: status: { DRDY ERR }
>[10497.776810] ata4.00: error: { ABRT }
>[10497.827160] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10497.830800] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10497.830821] ata4.00: configured for UDMA/33
>[10497.830884] sd 3:0:0:0: [sdb]  
>[10497.830890] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10497.830895] sd 3:0:0:0: [sdb]  
>[10497.830900] Sense Key : Aborted Command [current] [descriptor]
>[10497.830906] Descriptor sense data with sense descriptors (in hex):
>[10497.830910]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10497.830929]         0a ff 50 a0 
>[10497.830939] sd 3:0:0:0: [sdb]  
>[10497.830945] Add. Sense: No additional sense information
>[10497.830950] sd 3:0:0:0: [sdb] CDB: 
>[10497.830953] Write(10): 2a 00 0a ff 50 00 00 04 00 00
>[10497.830974] end_request: I/O error, dev sdb, sector 184504320
>[10497.849348] Buffer I/O error on device sdb1, logical block 23062784
>[10497.871765] Buffer I/O error on device sdb1, logical block 23062785
>[10497.892654] Buffer I/O error on device sdb1, logical block 23062786
>[10497.916027] Buffer I/O error on device sdb1, logical block 23062787
>[10497.938174] Buffer I/O error on device sdb1, logical block 23062788
>[10497.958422] Buffer I/O error on device sdb1, logical block 23062789
>[10497.978683] Buffer I/O error on device sdb1, logical block 23062790
>[10497.997890] Buffer I/O error on device sdb1, logical block 23062791
>[10498.015739] Buffer I/O error on device sdb1, logical block 23062792
>[10498.030190] Buffer I/O error on device sdb1, logical block 23062793
>[10498.044905] Buffer I/O error on device sdb1, logical block 23062794
>[10498.059789] Buffer I/O error on device sdb1, logical block 23062795
>[10498.076251] Buffer I/O error on device sdb1, logical block 23062796
>[10498.090718] Buffer I/O error on device sdb1, logical block 23062797
>[10498.101866] Buffer I/O error on device sdb1, logical block 23062798
>[10498.112555] Buffer I/O error on device sdb1, logical block 23062799
>[10498.122784] Buffer I/O error on device sdb1, logical block 23062800
>[10498.132801] Buffer I/O error on device sdb1, logical block 23062801
>[10498.143461] Buffer I/O error on device sdb1, logical block 23062802
>[10498.153593] Buffer I/O error on device sdb1, logical block 23062803
>[10498.162332] Buffer I/O error on device sdb1, logical block 23062804
>[10498.172327] Buffer I/O error on device sdb1, logical block 23062805
>[10498.180556] Buffer I/O error on device sdb1, logical block 23062806
>[10498.188117] Buffer I/O error on device sdb1, logical block 23062807
>[10498.196423] Buffer I/O error on device sdb1, logical block 23062808
>[10498.202066] Buffer I/O error on device sdb1, logical block 23062809
>[10498.208933] Buffer I/O error on device sdb1, logical block 23062810
>[10498.213935] Buffer I/O error on device sdb1, logical block 23062811
>[10498.219161] Buffer I/O error on device sdb1, logical block 23062812
>[10498.223309] Buffer I/O error on device sdb1, logical block 23062813
>[10498.226868] Buffer I/O error on device sdb1, logical block 23062814
>[10498.230137] Buffer I/O error on device sdb1, logical block 23062815
>[10498.233699] Buffer I/O error on device sdb1, logical block 23062816
>[10498.237408] Buffer I/O error on device sdb1, logical block 23062817
>[10498.240741] Buffer I/O error on device sdb1, logical block 23062818
>[10498.244179] Buffer I/O error on device sdb1, logical block 23062819
>[10498.247628] Buffer I/O error on device sdb1, logical block 23062820
>[10498.251127] Buffer I/O error on device sdb1, logical block 23062821
>[10498.254531] Buffer I/O error on device sdb1, logical block 23062822
>[10498.257783] Buffer I/O error on device sdb1, logical block 23062823
>[10498.260987] Buffer I/O error on device sdb1, logical block 23062824
>[10498.264172] Buffer I/O error on device sdb1, logical block 23062825
>[10498.267605] Buffer I/O error on device sdb1, logical block 23062826
>[10498.270720] Buffer I/O error on device sdb1, logical block 23062827
>[10498.274363] Buffer I/O error on device sdb1, logical block 23062828
>[10498.277880] Buffer I/O error on device sdb1, logical block 23062829
>[10498.281301] Buffer I/O error on device sdb1, logical block 23062830
>[10498.284528] Buffer I/O error on device sdb1, logical block 23062831
>[10498.287618] Buffer I/O error on device sdb1, logical block 23062832
>[10498.290953] Buffer I/O error on device sdb1, logical block 23062833
>[10498.294223] Buffer I/O error on device sdb1, logical block 23062834
>[10498.297534] Buffer I/O error on device sdb1, logical block 23062835
>[10498.300653] Buffer I/O error on device sdb1, logical block 23062836
>[10498.304087] Buffer I/O error on device sdb1, logical block 23062837
>[10498.307525] Buffer I/O error on device sdb1, logical block 23062838
>[10498.311051] Buffer I/O error on device sdb1, logical block 23062839
>[10498.314502] Buffer I/O error on device sdb1, logical block 23062840
>[10498.317716] Buffer I/O error on device sdb1, logical block 23062841
>[10498.321361] Buffer I/O error on device sdb1, logical block 23062842
>[10498.324682] Buffer I/O error on device sdb1, logical block 23062843
>[10498.328144] Buffer I/O error on device sdb1, logical block 23062844
>[10498.331363] Buffer I/O error on device sdb1, logical block 23062845
>[10498.334731] Buffer I/O error on device sdb1, logical block 23062846
>[10498.337996] Buffer I/O error on device sdb1, logical block 23062847
>[10498.341143] Buffer I/O error on device sdb1, logical block 23062848
>[10498.344530] Buffer I/O error on device sdb1, logical block 23062849
>[10498.348089] Buffer I/O error on device sdb1, logical block 23062850
>[10498.351642] Buffer I/O error on device sdb1, logical block 23062851
>[10498.355578] Buffer I/O error on device sdb1, logical block 23062852
>[10498.359063] Buffer I/O error on device sdb1, logical block 23062853
>[10498.362638] Buffer I/O error on device sdb1, logical block 23062854
>[10498.366336] Buffer I/O error on device sdb1, logical block 23062855
>[10498.369810] Buffer I/O error on device sdb1, logical block 23062856
>[10498.373248] Buffer I/O error on device sdb1, logical block 23062857
>[10498.376850] Buffer I/O error on device sdb1, logical block 23062858
>[10498.380516] Buffer I/O error on device sdb1, logical block 23062859
>[10498.383973] Buffer I/O error on device sdb1, logical block 23062860
>[10498.387512] Buffer I/O error on device sdb1, logical block 23062861
>[10498.390794] Buffer I/O error on device sdb1, logical block 23062862
>[10498.394233] Buffer I/O error on device sdb1, logical block 23062863
>[10498.397601] Buffer I/O error on device sdb1, logical block 23062864
>[10498.401101] Buffer I/O error on device sdb1, logical block 23062865
>[10498.404460] Buffer I/O error on device sdb1, logical block 23062866
>[10498.407982] Buffer I/O error on device sdb1, logical block 23062867
>[10498.411384] Buffer I/O error on device sdb1, logical block 23062868
>[10498.414905] Buffer I/O error on device sdb1, logical block 23062869
>[10498.418268] Buffer I/O error on device sdb1, logical block 23062870
>[10498.421363] Buffer I/O error on device sdb1, logical block 23062871
>[10498.424469] Buffer I/O error on device sdb1, logical block 23062872
>[10498.427746] Buffer I/O error on device sdb1, logical block 23062873
>[10498.430922] Buffer I/O error on device sdb1, logical block 23062874
>[10498.434118] Buffer I/O error on device sdb1, logical block 23062875
>[10498.437219] Buffer I/O error on device sdb1, logical block 23062876
>[10498.440749] Buffer I/O error on device sdb1, logical block 23062877
>[10498.444475] Buffer I/O error on device sdb1, logical block 23062878
>[10498.447651] Buffer I/O error on device sdb1, logical block 23062879
>[10498.451028] Buffer I/O error on device sdb1, logical block 23062880
>[10498.454226] Buffer I/O error on device sdb1, logical block 23062881
>[10498.457396] Buffer I/O error on device sdb1, logical block 23062882
>[10498.460720] Buffer I/O error on device sdb1, logical block 23062883
>[10498.464091] Buffer I/O error on device sdb1, logical block 23062884
>[10498.467605] Buffer I/O error on device sdb1, logical block 23062885
>[10498.470749] Buffer I/O error on device sdb1, logical block 23062886
>[10498.474098] Buffer I/O error on device sdb1, logical block 23062887
>[10498.477401] Buffer I/O error on device sdb1, logical block 23062888
>[10498.480700] Buffer I/O error on device sdb1, logical block 23062889
>[10498.484280] Buffer I/O error on device sdb1, logical block 23062890
>[10498.487523] Buffer I/O error on device sdb1, logical block 23062891
>[10498.490999] Buffer I/O error on device sdb1, logical block 23062892
>[10498.494034] Buffer I/O error on device sdb1, logical block 23062893
>[10498.497287] Buffer I/O error on device sdb1, logical block 23062894
>[10498.500737] Buffer I/O error on device sdb1, logical block 23062895
>[10498.504057] Buffer I/O error on device sdb1, logical block 23062896
>[10498.509710] Buffer I/O error on device sdb1, logical block 23062897
>[10498.514311] Buffer I/O error on device sdb1, logical block 23062898
>[10498.517499] Buffer I/O error on device sdb1, logical block 23062899
>[10498.520966] Buffer I/O error on device sdb1, logical block 23062900
>[10498.524110] Buffer I/O error on device sdb1, logical block 23062901
>[10498.527205] Buffer I/O error on device sdb1, logical block 23062902
>[10498.530677] Buffer I/O error on device sdb1, logical block 23062903
>[10498.534001] Buffer I/O error on device sdb1, logical block 23062904
>[10498.537358] Buffer I/O error on device sdb1, logical block 23062905
>[10498.540819] Buffer I/O error on device sdb1, logical block 23062906
>[10498.544401] Buffer I/O error on device sdb1, logical block 23062907
>[10498.547774] Buffer I/O error on device sdb1, logical block 23062908
>[10498.551099] Buffer I/O error on device sdb1, logical block 23062909
>[10498.555030] Buffer I/O error on device sdb1, logical block 23062910
>[10498.560116] Buffer I/O error on device sdb1, logical block 23062911
>[10498.563263] EXT4-fs warning (device sdb1): ext4_end_bio:319: I/O error writing to inode 38015964 (offset 9437184 size 524288 starting block 23063040)
>[10498.563362] ata4: EH complete
>[10499.591850] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10499.596822] ata4.00: BMDMA2 stat 0x6d0009
>[10499.604149] ata4: SError: { Handshk UnrecFIS }
>[10499.610803] ata4.00: failed command: READ DMA EXT
>[10499.617467] ata4.00: cmd 25/00:00:00:85:f4/00:01:26:00:00/e0 tag 0 dma 131072 in
>[10499.617467]          res 51/04:ef:11:85:f4/00:00:26:00:00/e0 Emask 0x1 (device error)
>[10499.643528] ata4.00: status: { DRDY ERR }
>[10499.652901] ata4.00: error: { ABRT }
>[10499.685040] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10499.692488] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10499.692495] ata4.00: configured for UDMA/33
>[10499.692522] sd 3:0:0:0: [sdb]  
>[10499.692525] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10499.692528] sd 3:0:0:0: [sdb]  
>[10499.692531] Sense Key : Aborted Command [current] [descriptor]
>[10499.692536] Descriptor sense data with sense descriptors (in hex):
>[10499.692539]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10499.692553]         26 f4 85 11 
>[10499.692559] sd 3:0:0:0: [sdb]  
>[10499.692563] Add. Sense: No additional sense information
>[10499.692566] sd 3:0:0:0: [sdb] CDB: 
>[10499.692568] Read(10): 28 00 26 f4 85 00 00 01 00 00
>[10499.692581] end_request: I/O error, dev sdb, sector 653559040
>[10499.700925] ata4: EH complete
>[10500.306121] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10502.982163] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[10502.990535] ata3.00: BMDMA2 stat 0x686d0009
>[10503.001427] ata3.00: failed command: READ DMA EXT
>[10503.014479] ata3.00: cmd 25/00:00:00:10:af/00:01:1f:00:00/e0 tag 0 dma 131072 in
>[10503.014479]          res 51/04:af:51:10:af/00:00:1f:00:00/e0 Emask 0x1 (device error)
>[10503.059703] ata3.00: status: { DRDY ERR }
>[10503.069821] ata3.00: error: { ABRT }
>[10503.084591] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10503.087599] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10503.087610] ata3.00: configured for UDMA/33
>[10503.087640] sd 2:0:0:0: [sda]  
>[10503.087645] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10503.087650] sd 2:0:0:0: [sda]  
>[10503.087653] Sense Key : Aborted Command [current] [descriptor]
>[10503.087660] Descriptor sense data with sense descriptors (in hex):
>[10503.087663]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10503.087682]         1f af 10 51 
>[10503.087691] sd 2:0:0:0: [sda]  
>[10503.087696] Add. Sense: No additional sense information
>[10503.087700] sd 2:0:0:0: [sda] CDB: 
>[10503.087703] Read(10): 28 00 1f af 10 00 00 01 00 00
>[10503.087720] end_request: I/O error, dev sda, sector 531566592
>[10503.098441] ata3: EH complete
>[10505.625442] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10506.811618] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10506.824368] ata4.00: BMDMA2 stat 0x6d0009
>[10506.837187] ata4: SError: { Handshk UnrecFIS }
>[10506.850066] ata4.00: failed command: READ DMA EXT
>[10506.863298] ata4.00: cmd 25/00:00:00:fa:91/00:01:52:00:00/e0 tag 0 dma 131072 in
>[10506.863298]          res 51/04:1f:e1:fa:91/00:00:52:00:00/e0 Emask 0x1 (device error)
>[10506.921351] ata4.00: status: { DRDY ERR }
>[10506.936518] ata4.00: error: { ABRT }
>[10506.979148] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10506.982604] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10506.982610] ata4.00: configured for UDMA/33
>[10506.982635] sd 3:0:0:0: [sdb]  
>[10506.982638] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10506.982642] sd 3:0:0:0: [sdb]  
>[10506.982644] Sense Key : Aborted Command [current] [descriptor]
>[10506.982649] Descriptor sense data with sense descriptors (in hex):
>[10506.982652]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10506.982666]         52 91 fa e1 
>[10506.982672] sd 3:0:0:0: [sdb]  
>[10506.982676] Add. Sense: No additional sense information
>[10506.982679] sd 3:0:0:0: [sdb] CDB: 
>[10506.982681] Read(10): 28 00 52 91 fa 00 00 01 00 00
>[10506.982694] end_request: I/O error, dev sdb, sector 1385298432
>[10506.998001] ata4: EH complete
>[10510.557625] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10512.368156] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[10512.384253] ata3.00: BMDMA2 stat 0x686d0009
>[10512.399760] ata3.00: failed command: READ DMA EXT
>[10512.414707] ata3.00: cmd 25/00:00:00:98:2b/00:01:1c:00:00/e0 tag 0 dma 131072 in
>[10512.414707]          res 51/04:ef:11:98:2b/00:00:1c:00:00/e0 Emask 0x1 (device error)
>[10512.474694] ata3.00: status: { DRDY ERR }
>[10512.490125] ata3.00: error: { ABRT }
>[10512.508727] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10512.511592] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10512.511598] ata3.00: configured for UDMA/33
>[10512.511623] sd 2:0:0:0: [sda]  
>[10512.511626] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10512.511629] sd 2:0:0:0: [sda]  
>[10512.511632] Sense Key : Aborted Command [current] [descriptor]
>[10512.511637] Descriptor sense data with sense descriptors (in hex):
>[10512.511639]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10512.511653]         1c 2b 98 11 
>[10512.511659] sd 2:0:0:0: [sda]  
>[10512.511663] Add. Sense: No additional sense information
>[10512.511666] sd 2:0:0:0: [sda] CDB: 
>[10512.511668] Read(10): 28 00 1c 2b 98 00 00 01 00 00
>[10512.511680] end_request: I/O error, dev sda, sector 472619008
>[10512.526940] ata3: EH complete
>[10521.141435] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10521.851271] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10521.866940] ata4.00: BMDMA2 stat 0x6d0009
>[10521.882489] ata4: SError: { Handshk UnrecFIS }
>[10521.897623] ata4.00: failed command: READ DMA EXT
>[10521.912469] ata4.00: cmd 25/00:00:00:df:c3/00:01:2c:00:00/e0 tag 0 dma 131072 in
>[10521.912469]          res 51/04:4f:b1:df:c3/00:00:2c:00:00/e0 Emask 0x1 (device error)
>[10521.972907] ata4.00: status: { DRDY ERR }
>[10521.988239] ata4.00: error: { ABRT }
>[10522.023935] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10522.027595] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10522.027601] ata4.00: configured for UDMA/33
>[10522.027625] sd 3:0:0:0: [sdb]  
>[10522.027628] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10522.027632] sd 3:0:0:0: [sdb]  
>[10522.027634] Sense Key : Aborted Command [current] [descriptor]
>[10522.027639] Descriptor sense data with sense descriptors (in hex):
>[10522.027641]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10522.027656]         2c c3 df b1 
>[10522.027662] sd 3:0:0:0: [sdb]  
>[10522.027665] Add. Sense: No additional sense information
>[10522.027669] sd 3:0:0:0: [sdb] CDB: 
>[10522.027671] Read(10): 28 00 2c c3 df 00 00 01 00 00
>[10522.027683] end_request: I/O error, dev sdb, sector 751034112
>[10522.043076] ata4: EH complete
>[10532.251200] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10532.266948] ata4.00: BMDMA2 stat 0x6d0009
>[10532.282520] ata4: SError: { Handshk UnrecFIS }
>[10532.297597] ata4.00: failed command: READ DMA
>[10532.312328] ata4.00: cmd c8/00:00:00:8d:72/00:00:00:00:00/e1 tag 0 dma 131072 in
>[10532.312328]          res 51/04:4f:b1:8d:72/00:00:2c:00:00/e1 Emask 0x1 (device error)
>[10532.372779] ata4.00: status: { DRDY ERR }
>[10532.388136] ata4.00: error: { ABRT }
>[10532.413358] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10532.416606] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10532.416612] ata4.00: configured for UDMA/33
>[10532.416636] sd 3:0:0:0: [sdb]  
>[10532.416639] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10532.416643] sd 3:0:0:0: [sdb]  
>[10532.416645] Sense Key : Aborted Command [current] [descriptor]
>[10532.416650] Descriptor sense data with sense descriptors (in hex):
>[10532.416653]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10532.416667]         01 72 8d b1 
>[10532.416673] sd 3:0:0:0: [sdb]  
>[10532.416677] Add. Sense: No additional sense information
>[10532.416680] sd 3:0:0:0: [sdb] CDB: 
>[10532.416682] Read(10): 28 00 01 72 8d 00 00 01 00 00
>[10532.416695] end_request: I/O error, dev sdb, sector 24284416
>[10532.432427] ata4: EH complete
>[10541.754869] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10550.344536] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10550.360186] ata4.00: BMDMA2 stat 0x6d0009
>[10550.375309] ata4: SError: { Handshk UnrecFIS }
>[10550.390158] ata4.00: failed command: READ DMA EXT
>[10550.404744] ata4.00: cmd 25/00:00:00:89:7f/00:01:2a:00:00/e0 tag 0 dma 131072 in
>[10550.404744]          res 51/04:2f:d1:89:7f/00:00:2a:00:00/e0 Emask 0x1 (device error)
>[10550.464985] ata4.00: status: { DRDY ERR }
>[10550.480121] ata4.00: error: { ABRT }
>[10550.520202] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10550.523595] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10550.523601] ata4.00: configured for UDMA/33
>[10550.523624] sd 3:0:0:0: [sdb]  
>[10550.523627] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10550.523630] sd 3:0:0:0: [sdb]  
>[10550.523633] Sense Key : Aborted Command [current] [descriptor]
>[10550.523638] Descriptor sense data with sense descriptors (in hex):
>[10550.523640]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10550.523655]         2a 7f 89 d1 
>[10550.523661] sd 3:0:0:0: [sdb]  
>[10550.523665] Add. Sense: No additional sense information
>[10550.523668] sd 3:0:0:0: [sdb] CDB: 
>[10550.523670] Read(10): 28 00 2a 7f 89 00 00 01 00 00
>[10550.523683] end_request: I/O error, dev sdb, sector 713001216
>[10550.539235] ata4: EH complete
>[10553.536137] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10557.464230] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[10557.480201] ata3.00: BMDMA2 stat 0x686d0009
>[10557.496540] ata3.00: failed command: READ DMA EXT
>[10557.512178] ata3.00: cmd 25/00:00:30:d8:2f/00:01:1a:00:00/e0 tag 0 dma 131072 in
>[10557.512178]          res 51/04:1f:11:d9:2f/00:00:1a:00:00/e0 Emask 0x1 (device error)
>[10557.575438] ata3.00: status: { DRDY ERR }
>[10557.591302] ata3.00: error: { ABRT }
>[10557.610220] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10557.613630] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10557.613640] ata3.00: configured for UDMA/33
>[10557.613667] sd 2:0:0:0: [sda]  
>[10557.613670] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10557.613674] sd 2:0:0:0: [sda]  
>[10557.613676] Sense Key : Aborted Command [current] [descriptor]
>[10557.613682] Descriptor sense data with sense descriptors (in hex):
>[10557.613684]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10557.613698]         1a 2f d9 11 
>[10557.613704] sd 2:0:0:0: [sda]  
>[10557.613708] Add. Sense: No additional sense information
>[10557.613711] sd 2:0:0:0: [sda] CDB: 
>[10557.613714] Read(10): 28 00 1a 2f d8 30 00 01 00 00
>[10557.613727] end_request: I/O error, dev sda, sector 439343152
>[10557.629562] ata3: EH complete
>[10559.083090] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10559.099368] ata4.00: BMDMA2 stat 0x6d0009
>[10559.115499] ata4: SError: { Handshk UnrecFIS }
>[10559.131483] ata4.00: failed command: READ DMA EXT
>[10559.146991] ata4.00: cmd 25/00:00:00:9c:3b/00:01:42:00:00/e0 tag 0 dma 131072 in
>[10559.146991]          res 51/04:2f:d1:9c:3b/00:00:42:00:00/e0 Emask 0x1 (device error)
>[10559.208143] ata4.00: status: { DRDY ERR }
>[10559.224208] ata4.00: error: { ABRT }
>[10559.266544] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10559.269642] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10559.269648] ata4.00: configured for UDMA/33
>[10559.269673] sd 3:0:0:0: [sdb]  
>[10559.269677] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10559.269680] sd 3:0:0:0: [sdb]  
>[10559.269683] Sense Key : Aborted Command [current] [descriptor]
>[10559.269688] Descriptor sense data with sense descriptors (in hex):
>[10559.269690]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10559.269704]         42 3b 9c d1 
>[10559.269710] sd 3:0:0:0: [sdb]  
>[10559.269714] Add. Sense: No additional sense information
>[10559.269717] sd 3:0:0:0: [sdb] CDB: 
>[10559.269719] Read(10): 28 00 42 3b 9c 00 00 01 00 00
>[10559.269732] end_request: I/O error, dev sdb, sector 1111202816
>[10559.285359] ata4: EH complete
>[10559.900889] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10559.918781] ata4.00: BMDMA2 stat 0x6d0009
>[10559.936311] ata4: SError: { Handshk UnrecFIS }
>[10559.953883] ata4.00: failed command: READ DMA EXT
>[10559.969529] ata4.00: cmd 25/00:00:00:f3:4e/00:01:42:00:00/e0 tag 0 dma 131072 in
>[10559.969529]          res 51/04:8f:71:f3:4e/00:00:42:00:00/e0 Emask 0x1 (device error)
>[10560.035776] ata4.00: status: { DRDY ERR }
>[10560.053364] ata4.00: error: { ABRT }
>[10560.096398] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10560.099693] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10560.099712] ata4.00: configured for UDMA/33
>[10560.099753] sd 3:0:0:0: [sdb]  
>[10560.099758] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10560.099764] sd 3:0:0:0: [sdb]  
>[10560.099768] Sense Key : Aborted Command [current] [descriptor]
>[10560.099775] Descriptor sense data with sense descriptors (in hex):
>[10560.099778]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10560.099797]         42 4e f3 71 
>[10560.099809] sd 3:0:0:0: [sdb]  
>[10560.099814] Add. Sense: No additional sense information
>[10560.099820] sd 3:0:0:0: [sdb] CDB: 
>[10560.099824] Read(10): 28 00 42 4e f3 00 00 01 00 00
>[10560.099842] end_request: I/O error, dev sdb, sector 1112470272
>[10560.117934] ata4: EH complete
>[10562.064264] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10562.339908] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10562.356551] ata3.00: BMDMA2 stat 0x69650001
>[10562.372792] ata3: SError: { Handshk UnrecFIS }
>[10562.390246] ata3.00: failed command: WRITE DMA EXT
>[10562.405519] ata3.00: cmd 35/00:00:00:59:67/00:04:10:00:00/e0 tag 0 dma 524288 out
>[10562.405519]          res 51/04:10:f0:5c:67/00:00:10:00:00/e0 Emask 0x1 (device error)
>[10562.466706] ata3.00: status: { DRDY ERR }
>[10562.482341] ata3.00: error: { ABRT }
>[10562.501074] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10562.508362] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10562.508373] ata3.00: configured for UDMA/33
>[10562.508403] sd 2:0:0:0: [sda]  
>[10562.508407] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10562.508410] sd 2:0:0:0: [sda]  
>[10562.508412] Sense Key : Aborted Command [current] [descriptor]
>[10562.508418] Descriptor sense data with sense descriptors (in hex):
>[10562.508420]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10562.508434]         10 67 5c f0 
>[10562.508440] sd 2:0:0:0: [sda]  
>[10562.508444] Add. Sense: No additional sense information
>[10562.508447] sd 2:0:0:0: [sda] CDB: 
>[10562.508449] Write(10): 2a 00 10 67 59 00 00 04 00 00
>[10562.508480] end_request: I/O error, dev sda, sector 275208448
>[10562.524345] Buffer I/O error on device sda6, logical block 24905504
>[10562.540035] Buffer I/O error on device sda6, logical block 24905505
>[10562.556962] Buffer I/O error on device sda6, logical block 24905506
>[10562.571808] Buffer I/O error on device sda6, logical block 24905507
>[10562.586231] Buffer I/O error on device sda6, logical block 24905508
>[10562.600328] Buffer I/O error on device sda6, logical block 24905509
>[10562.614105] Buffer I/O error on device sda6, logical block 24905510
>[10562.627452] Buffer I/O error on device sda6, logical block 24905511
>[10562.641323] Buffer I/O error on device sda6, logical block 24905512
>[10562.654422] Buffer I/O error on device sda6, logical block 24905513
>[10562.667216] Buffer I/O error on device sda6, logical block 24905514
>[10562.679855] Buffer I/O error on device sda6, logical block 24905515
>[10562.691937] Buffer I/O error on device sda6, logical block 24905516
>[10562.703555] Buffer I/O error on device sda6, logical block 24905517
>[10562.714496] Buffer I/O error on device sda6, logical block 24905518
>[10562.724979] Buffer I/O error on device sda6, logical block 24905519
>[10562.735273] Buffer I/O error on device sda6, logical block 24905520
>[10562.744780] Buffer I/O error on device sda6, logical block 24905521
>[10562.754390] Buffer I/O error on device sda6, logical block 24905522
>[10562.763180] Buffer I/O error on device sda6, logical block 24905523
>[10562.771671] Buffer I/O error on device sda6, logical block 24905524
>[10562.779926] Buffer I/O error on device sda6, logical block 24905525
>[10562.787667] Buffer I/O error on device sda6, logical block 24905526
>[10562.794802] Buffer I/O error on device sda6, logical block 24905527
>[10562.801388] Buffer I/O error on device sda6, logical block 24905528
>[10562.807326] Buffer I/O error on device sda6, logical block 24905529
>[10562.812929] Buffer I/O error on device sda6, logical block 24905530
>[10562.817817] Buffer I/O error on device sda6, logical block 24905531
>[10562.822635] Buffer I/O error on device sda6, logical block 24905532
>[10562.826807] Buffer I/O error on device sda6, logical block 24905533
>[10562.830118] Buffer I/O error on device sda6, logical block 24905534
>[10562.833306] Buffer I/O error on device sda6, logical block 24905535
>[10562.837274] Buffer I/O error on device sda6, logical block 24905536
>[10562.841063] Buffer I/O error on device sda6, logical block 24905537
>[10562.844944] Buffer I/O error on device sda6, logical block 24905538
>[10562.848709] Buffer I/O error on device sda6, logical block 24905539
>[10562.852587] Buffer I/O error on device sda6, logical block 24905540
>[10562.856349] Buffer I/O error on device sda6, logical block 24905541
>[10562.860297] Buffer I/O error on device sda6, logical block 24905542
>[10562.863948] Buffer I/O error on device sda6, logical block 24905543
>[10562.867711] Buffer I/O error on device sda6, logical block 24905544
>[10562.871553] Buffer I/O error on device sda6, logical block 24905545
>[10562.875121] Buffer I/O error on device sda6, logical block 24905546
>[10562.878738] Buffer I/O error on device sda6, logical block 24905547
>[10562.883036] Buffer I/O error on device sda6, logical block 24905548
>[10562.886948] Buffer I/O error on device sda6, logical block 24905549
>[10562.890832] Buffer I/O error on device sda6, logical block 24905550
>[10562.894481] Buffer I/O error on device sda6, logical block 24905551
>[10562.898403] Buffer I/O error on device sda6, logical block 24905552
>[10562.902129] Buffer I/O error on device sda6, logical block 24905553
>[10562.906440] Buffer I/O error on device sda6, logical block 24905554
>[10562.910208] Buffer I/O error on device sda6, logical block 24905555
>[10562.913999] Buffer I/O error on device sda6, logical block 24905556
>[10562.917751] Buffer I/O error on device sda6, logical block 24905557
>[10562.921634] Buffer I/O error on device sda6, logical block 24905558
>[10562.925284] Buffer I/O error on device sda6, logical block 24905559
>[10562.929051] Buffer I/O error on device sda6, logical block 24905560
>[10562.932592] Buffer I/O error on device sda6, logical block 24905561
>[10562.936449] Buffer I/O error on device sda6, logical block 24905562
>[10562.940559] Buffer I/O error on device sda6, logical block 24905563
>[10562.944150] Buffer I/O error on device sda6, logical block 24905564
>[10562.947719] Buffer I/O error on device sda6, logical block 24905565
>[10562.951164] Buffer I/O error on device sda6, logical block 24905566
>[10562.954251] Buffer I/O error on device sda6, logical block 24905567
>[10562.957920] Buffer I/O error on device sda6, logical block 24905568
>[10562.960935] Buffer I/O error on device sda6, logical block 24905569
>[10562.963868] Buffer I/O error on device sda6, logical block 24905570
>[10562.966937] Buffer I/O error on device sda6, logical block 24905571
>[10562.969835] Buffer I/O error on device sda6, logical block 24905572
>[10562.972965] Buffer I/O error on device sda6, logical block 24905573
>[10562.976051] Buffer I/O error on device sda6, logical block 24905574
>[10562.979552] Buffer I/O error on device sda6, logical block 24905575
>[10562.983040] Buffer I/O error on device sda6, logical block 24905576
>[10562.986446] Buffer I/O error on device sda6, logical block 24905577
>[10562.989552] Buffer I/O error on device sda6, logical block 24905578
>[10562.992494] Buffer I/O error on device sda6, logical block 24905579
>[10562.996341] Buffer I/O error on device sda6, logical block 24905580
>[10563.000595] Buffer I/O error on device sda6, logical block 24905581
>[10563.004218] Buffer I/O error on device sda6, logical block 24905582
>[10563.007995] Buffer I/O error on device sda6, logical block 24905583
>[10563.011743] Buffer I/O error on device sda6, logical block 24905584
>[10563.015515] Buffer I/O error on device sda6, logical block 24905585
>[10563.019319] Buffer I/O error on device sda6, logical block 24905586
>[10563.022669] Buffer I/O error on device sda6, logical block 24905587
>[10563.026345] Buffer I/O error on device sda6, logical block 24905588
>[10563.029834] Buffer I/O error on device sda6, logical block 24905589
>[10563.033702] Buffer I/O error on device sda6, logical block 24905590
>[10563.037250] Buffer I/O error on device sda6, logical block 24905591
>[10563.040827] Buffer I/O error on device sda6, logical block 24905592
>[10563.044822] Buffer I/O error on device sda6, logical block 24905593
>[10563.048850] Buffer I/O error on device sda6, logical block 24905594
>[10563.052464] Buffer I/O error on device sda6, logical block 24905595
>[10563.058271] Buffer I/O error on device sda6, logical block 24905596
>[10563.063106] Buffer I/O error on device sda6, logical block 24905597
>[10563.066716] Buffer I/O error on device sda6, logical block 24905598
>[10563.070365] Buffer I/O error on device sda6, logical block 24905599
>[10563.073444] Buffer I/O error on device sda6, logical block 24905600
>[10563.076722] Buffer I/O error on device sda6, logical block 24905601
>[10563.080269] Buffer I/O error on device sda6, logical block 24905602
>[10563.083759] Buffer I/O error on device sda6, logical block 24905603
>[10563.087619] Buffer I/O error on device sda6, logical block 24905604
>[10563.091345] Buffer I/O error on device sda6, logical block 24905605
>[10563.095252] Buffer I/O error on device sda6, logical block 24905606
>[10563.098646] Buffer I/O error on device sda6, logical block 24905607
>[10563.102384] Buffer I/O error on device sda6, logical block 24905608
>[10563.106241] Buffer I/O error on device sda6, logical block 24905609
>[10563.109974] Buffer I/O error on device sda6, logical block 24905610
>[10563.113917] Buffer I/O error on device sda6, logical block 24905611
>[10563.117710] Buffer I/O error on device sda6, logical block 24905612
>[10563.120940] Buffer I/O error on device sda6, logical block 24905613
>[10563.124415] Buffer I/O error on device sda6, logical block 24905614
>[10563.128055] Buffer I/O error on device sda6, logical block 24905615
>[10563.131725] Buffer I/O error on device sda6, logical block 24905616
>[10563.135331] Buffer I/O error on device sda6, logical block 24905617
>[10563.138996] Buffer I/O error on device sda6, logical block 24905618
>[10563.142648] Buffer I/O error on device sda6, logical block 24905619
>[10563.146837] Buffer I/O error on device sda6, logical block 24905620
>[10563.150752] Buffer I/O error on device sda6, logical block 24905621
>[10563.154459] Buffer I/O error on device sda6, logical block 24905622
>[10563.158284] Buffer I/O error on device sda6, logical block 24905623
>[10563.161576] Buffer I/O error on device sda6, logical block 24905624
>[10563.165209] Buffer I/O error on device sda6, logical block 24905625
>[10563.168852] Buffer I/O error on device sda6, logical block 24905626
>[10563.172391] Buffer I/O error on device sda6, logical block 24905627
>[10563.176138] Buffer I/O error on device sda6, logical block 24905628
>[10563.179564] Buffer I/O error on device sda6, logical block 24905629
>[10563.182806] Buffer I/O error on device sda6, logical block 24905630
>[10563.186164] Buffer I/O error on device sda6, logical block 24905631
>[10563.189398] EXT4-fs warning (device sda6): ext4_end_bio:319: I/O error writing to inode 24252915 (offset 24248320 size 524288 starting block 34401056)
>[10563.189427] ata3: EH complete
>[10563.297588] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10563.301584] ata3.00: BMDMA2 stat 0x686d0009
>[10563.305520] ata3: SError: { Handshk UnrecFIS }
>[10563.309461] ata3.00: failed command: READ DMA
>[10563.313967] ata3.00: cmd c8/00:00:00:96:46/00:00:00:00:00/e9 tag 0 dma 131072 in
>[10563.313967]          res 51/04:df:21:96:46/00:00:10:00:00/e9 Emask 0x1 (device error)
>[10563.338169] ata3.00: status: { DRDY ERR }
>[10563.345302] ata3.00: error: { ABRT }
>[10563.355666] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10563.358645] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10563.358655] ata3.00: configured for UDMA/33
>[10563.358683] sd 2:0:0:0: [sda]  
>[10563.358686] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10563.358689] sd 2:0:0:0: [sda]  
>[10563.358692] Sense Key : Aborted Command [current] [descriptor]
>[10563.358697] Descriptor sense data with sense descriptors (in hex):
>[10563.358699]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10563.358714]         09 46 96 21 
>[10563.358720] sd 2:0:0:0: [sda]  
>[10563.358723] Add. Sense: No additional sense information
>[10563.358727] sd 2:0:0:0: [sda] CDB: 
>[10563.358729] Read(10): 28 00 09 46 96 00 00 01 00 00
>[10563.358741] end_request: I/O error, dev sda, sector 155620864
>[10563.366817] ata3: EH complete
>[10564.414897] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10564.423770] ata4.00: BMDMA2 stat 0x650001
>[10564.432698] ata4: SError: { Handshk UnrecFIS }
>[10564.441772] ata4.00: failed command: WRITE DMA EXT
>[10564.451281] ata4.00: cmd 35/00:00:80:73:e7/00:04:48:00:00/e0 tag 0 dma 524288 out
>[10564.451281]          res 51/04:a0:e0:74:e7/00:02:48:00:00/e0 Emask 0x1 (device error)
>[10564.492387] ata4.00: status: { DRDY ERR }
>[10564.503976] ata4.00: error: { ABRT }
>[10564.544373] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10564.547834] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10564.547847] ata4.00: configured for UDMA/33
>[10564.547874] sd 3:0:0:0: [sdb]  
>[10564.547878] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10564.547881] sd 3:0:0:0: [sdb]  
>[10564.547884] Sense Key : Aborted Command [current] [descriptor]
>[10564.547889] Descriptor sense data with sense descriptors (in hex):
>[10564.547891]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10564.547905]         48 e7 74 e0 
>[10564.547912] sd 3:0:0:0: [sdb]  
>[10564.547915] Add. Sense: No additional sense information
>[10564.547919] sd 3:0:0:0: [sdb] CDB: 
>[10564.547921] Write(10): 2a 00 48 e7 73 80 00 04 00 00
>[10564.547933] end_request: I/O error, dev sdb, sector 1223127936
>[10564.562561] Buffer I/O error on device sdb1, logical block 152890736
>[10564.575761] Buffer I/O error on device sdb1, logical block 152890737
>[10564.589498] Buffer I/O error on device sdb1, logical block 152890738
>[10564.603259] Buffer I/O error on device sdb1, logical block 152890739
>[10564.618446] Buffer I/O error on device sdb1, logical block 152890740
>[10564.632804] Buffer I/O error on device sdb1, logical block 152890741
>[10564.646687] Buffer I/O error on device sdb1, logical block 152890742
>[10564.659919] Buffer I/O error on device sdb1, logical block 152890743
>[10564.672831] Buffer I/O error on device sdb1, logical block 152890744
>[10564.684983] Buffer I/O error on device sdb1, logical block 152890745
>[10564.697500] Buffer I/O error on device sdb1, logical block 152890746
>[10564.709401] Buffer I/O error on device sdb1, logical block 152890747
>[10564.720897] Buffer I/O error on device sdb1, logical block 152890748
>[10564.731364] Buffer I/O error on device sdb1, logical block 152890749
>[10564.741861] Buffer I/O error on device sdb1, logical block 152890750
>[10564.751581] Buffer I/O error on device sdb1, logical block 152890751
>[10564.761083] Buffer I/O error on device sdb1, logical block 152890752
>[10564.770201] Buffer I/O error on device sdb1, logical block 152890753
>[10564.778951] Buffer I/O error on device sdb1, logical block 152890754
>[10564.787230] Buffer I/O error on device sdb1, logical block 152890755
>[10564.794768] Buffer I/O error on device sdb1, logical block 152890756
>[10564.802316] Buffer I/O error on device sdb1, logical block 152890757
>[10564.809426] Buffer I/O error on device sdb1, logical block 152890758
>[10564.816231] Buffer I/O error on device sdb1, logical block 152890759
>[10564.822222] Buffer I/O error on device sdb1, logical block 152890760
>[10564.827668] Buffer I/O error on device sdb1, logical block 152890761
>[10564.832452] Buffer I/O error on device sdb1, logical block 152890762
>[10564.836963] Buffer I/O error on device sdb1, logical block 152890763
>[10564.841361] Buffer I/O error on device sdb1, logical block 152890764
>[10564.845384] Buffer I/O error on device sdb1, logical block 152890765
>[10564.848836] Buffer I/O error on device sdb1, logical block 152890766
>[10564.852208] Buffer I/O error on device sdb1, logical block 152890767
>[10564.855724] Buffer I/O error on device sdb1, logical block 152890768
>[10564.858890] Buffer I/O error on device sdb1, logical block 152890769
>[10564.862465] Buffer I/O error on device sdb1, logical block 152890770
>[10564.866539] Buffer I/O error on device sdb1, logical block 152890771
>[10564.870308] Buffer I/O error on device sdb1, logical block 152890772
>[10564.873502] Buffer I/O error on device sdb1, logical block 152890773
>[10564.876625] Buffer I/O error on device sdb1, logical block 152890774
>[10564.879957] Buffer I/O error on device sdb1, logical block 152890775
>[10564.883441] Buffer I/O error on device sdb1, logical block 152890776
>[10564.886810] Buffer I/O error on device sdb1, logical block 152890777
>[10564.890225] Buffer I/O error on device sdb1, logical block 152890778
>[10564.893560] Buffer I/O error on device sdb1, logical block 152890779
>[10564.896917] Buffer I/O error on device sdb1, logical block 152890780
>[10564.900252] Buffer I/O error on device sdb1, logical block 152890781
>[10564.903691] Buffer I/O error on device sdb1, logical block 152890782
>[10564.907164] Buffer I/O error on device sdb1, logical block 152890783
>[10564.910511] Buffer I/O error on device sdb1, logical block 152890784
>[10564.913909] Buffer I/O error on device sdb1, logical block 152890785
>[10564.917481] Buffer I/O error on device sdb1, logical block 152890786
>[10564.920954] Buffer I/O error on device sdb1, logical block 152890787
>[10564.924391] Buffer I/O error on device sdb1, logical block 152890788
>[10564.927503] Buffer I/O error on device sdb1, logical block 152890789
>[10564.930827] Buffer I/O error on device sdb1, logical block 152890790
>[10564.934272] Buffer I/O error on device sdb1, logical block 152890791
>[10564.937827] Buffer I/O error on device sdb1, logical block 152890792
>[10564.941264] Buffer I/O error on device sdb1, logical block 152890793
>[10564.944697] Buffer I/O error on device sdb1, logical block 152890794
>[10564.948280] Buffer I/O error on device sdb1, logical block 152890795
>[10564.951656] Buffer I/O error on device sdb1, logical block 152890796
>[10564.955205] Buffer I/O error on device sdb1, logical block 152890797
>[10564.958691] Buffer I/O error on device sdb1, logical block 152890798
>[10564.961864] Buffer I/O error on device sdb1, logical block 152890799
>[10564.965076] Buffer I/O error on device sdb1, logical block 152890800
>[10564.968668] Buffer I/O error on device sdb1, logical block 152890801
>[10564.972197] Buffer I/O error on device sdb1, logical block 152890802
>[10564.975480] Buffer I/O error on device sdb1, logical block 152890803
>[10564.979333] Buffer I/O error on device sdb1, logical block 152890804
>[10564.983454] Buffer I/O error on device sdb1, logical block 152890805
>[10564.987256] Buffer I/O error on device sdb1, logical block 152890806
>[10564.990778] Buffer I/O error on device sdb1, logical block 152890807
>[10564.994236] Buffer I/O error on device sdb1, logical block 152890808
>[10564.997746] Buffer I/O error on device sdb1, logical block 152890809
>[10565.001353] Buffer I/O error on device sdb1, logical block 152890810
>[10565.004813] Buffer I/O error on device sdb1, logical block 152890811
>[10565.008393] Buffer I/O error on device sdb1, logical block 152890812
>[10565.011826] Buffer I/O error on device sdb1, logical block 152890813
>[10565.015281] Buffer I/O error on device sdb1, logical block 152890814
>[10565.018518] Buffer I/O error on device sdb1, logical block 152890815
>[10565.021646] Buffer I/O error on device sdb1, logical block 152890816
>[10565.024708] Buffer I/O error on device sdb1, logical block 152890817
>[10565.027812] Buffer I/O error on device sdb1, logical block 152890818
>[10565.031240] Buffer I/O error on device sdb1, logical block 152890819
>[10565.034625] Buffer I/O error on device sdb1, logical block 152890820
>[10565.037915] Buffer I/O error on device sdb1, logical block 152890821
>[10565.041316] Buffer I/O error on device sdb1, logical block 152890822
>[10565.044738] Buffer I/O error on device sdb1, logical block 152890823
>[10565.048109] Buffer I/O error on device sdb1, logical block 152890824
>[10565.051482] Buffer I/O error on device sdb1, logical block 152890825
>[10565.057821] Buffer I/O error on device sdb1, logical block 152890826
>[10565.061588] Buffer I/O error on device sdb1, logical block 152890827
>[10565.064981] Buffer I/O error on device sdb1, logical block 152890828
>[10565.068367] Buffer I/O error on device sdb1, logical block 152890829
>[10565.071673] Buffer I/O error on device sdb1, logical block 152890830
>[10565.075126] Buffer I/O error on device sdb1, logical block 152890831
>[10565.078497] Buffer I/O error on device sdb1, logical block 152890832
>[10565.081778] Buffer I/O error on device sdb1, logical block 152890833
>[10565.085220] Buffer I/O error on device sdb1, logical block 152890834
>[10565.088609] Buffer I/O error on device sdb1, logical block 152890835
>[10565.092059] Buffer I/O error on device sdb1, logical block 152890836
>[10565.095493] Buffer I/O error on device sdb1, logical block 152890837
>[10565.099157] Buffer I/O error on device sdb1, logical block 152890838
>[10565.102710] Buffer I/O error on device sdb1, logical block 152890839
>[10565.106096] Buffer I/O error on device sdb1, logical block 152890840
>[10565.109596] Buffer I/O error on device sdb1, logical block 152890841
>[10565.112978] Buffer I/O error on device sdb1, logical block 152890842
>[10565.116491] Buffer I/O error on device sdb1, logical block 152890843
>[10565.119827] Buffer I/O error on device sdb1, logical block 152890844
>[10565.123265] Buffer I/O error on device sdb1, logical block 152890845
>[10565.126607] Buffer I/O error on device sdb1, logical block 152890846
>[10565.130058] Buffer I/O error on device sdb1, logical block 152890847
>[10565.133534] Buffer I/O error on device sdb1, logical block 152890848
>[10565.136964] Buffer I/O error on device sdb1, logical block 152890849
>[10565.140525] Buffer I/O error on device sdb1, logical block 152890850
>[10565.143810] Buffer I/O error on device sdb1, logical block 152890851
>[10565.146767] Buffer I/O error on device sdb1, logical block 152890852
>[10565.149842] Buffer I/O error on device sdb1, logical block 152890853
>[10565.153352] Buffer I/O error on device sdb1, logical block 152890854
>[10565.156630] Buffer I/O error on device sdb1, logical block 152890855
>[10565.159954] Buffer I/O error on device sdb1, logical block 152890856
>[10565.163333] Buffer I/O error on device sdb1, logical block 152890857
>[10565.166722] Buffer I/O error on device sdb1, logical block 152890858
>[10565.170265] Buffer I/O error on device sdb1, logical block 152890859
>[10565.173551] Buffer I/O error on device sdb1, logical block 152890860
>[10565.176937] Buffer I/O error on device sdb1, logical block 152890861
>[10565.180232] Buffer I/O error on device sdb1, logical block 152890862
>[10565.183167] Buffer I/O error on device sdb1, logical block 152890863
>[10565.185938] EXT4-fs warning (device sdb1): ext4_end_bio:319: I/O error writing to inode 38015912 (offset 30867456 size 524288 starting block 152890992)
>[10565.185966] ata4: EH complete
>[10565.556208] JBD2: Detected IO errors while flushing file data on sda6-8
>[10567.061743] JBD2: Detected IO errors while flushing file data on sda6-8
>[10567.062784] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10572.306197] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10572.312675] JBD2: Detected IO errors while flushing file data on sda6-8
>[10577.384497] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10582.564337] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10587.557605] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10592.558291] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10607.567113] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10612.559310] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10617.573246] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10622.569215] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10638.063126] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10643.058916] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10653.075291] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10668.058874] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10674.419668] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10674.423615] ata3.00: BMDMA2 stat 0x696d0009
>[10674.428221] ata3: SError: { Handshk UnrecFIS }
>[10674.432758] ata3.00: failed command: READ DMA EXT
>[10674.437677] ata3.00: cmd 25/00:00:00:12:6d/00:02:09:00:00/e0 tag 0 dma 262144 in
>[10674.437677]          res 51/04:af:51:13:6d/00:00:09:00:00/e0 Emask 0x1 (device error)
>[10674.460401] ata3.00: status: { DRDY ERR }
>[10674.467730] ata3.00: error: { ABRT }
>[10674.479132] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10674.483045] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10674.483055] ata3.00: configured for UDMA/33
>[10674.483081] sd 2:0:0:0: [sda]  
>[10674.483084] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10674.483087] sd 2:0:0:0: [sda]  
>[10674.483090] Sense Key : Aborted Command [current] [descriptor]
>[10674.483095] Descriptor sense data with sense descriptors (in hex):
>[10674.483097]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10674.483111]         09 6d 13 51 
>[10674.483118] sd 2:0:0:0: [sda]  
>[10674.483121] Add. Sense: No additional sense information
>[10674.483124] sd 2:0:0:0: [sda] CDB: 
>[10674.483126] Read(10): 28 00 09 6d 12 00 00 02 00 00
>[10674.483139] end_request: I/O error, dev sda, sector 158142976
>[10674.491082] ata3: EH complete
>[10678.152549] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10683.063129] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10688.746046] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10688.754430] ata3.00: BMDMA2 stat 0x69650001
>[10688.763125] ata3: SError: { Handshk UnrecFIS }
>[10688.772069] ata3.00: failed command: WRITE DMA EXT
>[10688.781244] ata3.00: cmd 35/00:00:00:3c:a3/00:04:10:00:00/e0 tag 0 dma 524288 out
>[10688.781244]          res 51/04:90:70:3e:a3/00:01:10:00:00/e0 Emask 0x1 (device error)
>[10688.822485] ata3.00: status: { DRDY ERR }
>[10688.834029] ata3.00: error: { ABRT }
>[10688.845786] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10688.858188] ata4.00: BMDMA2 stat 0x6d0009
>[10688.870770] ata4: SError: { Handshk UnrecFIS }
>[10688.883631] ata4.00: failed command: READ DMA EXT
>[10688.896761] ata4.00: cmd 25/00:00:00:b3:81/00:01:2a:00:00/e0 tag 0 dma 131072 in
>[10688.896761]          res 51/04:af:51:b3:81/00:00:2a:00:00/e0 Emask 0x1 (device error)
>[10688.954365] ata4.00: status: { DRDY ERR }
>[10688.969401] ata4.00: error: { ABRT }
>[10688.984581] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10688.988240] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10688.988246] ata3.00: configured for UDMA/33
>[10688.988276] sd 2:0:0:0: [sda]  
>[10688.988279] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10688.988283] sd 2:0:0:0: [sda]  
>[10688.988285] Sense Key : Aborted Command [current] [descriptor]
>[10688.988290] Descriptor sense data with sense descriptors (in hex):
>[10688.988293]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10688.988307]         10 a3 3e 70 
>[10688.988314] sd 2:0:0:0: [sda]  
>[10688.988317] Add. Sense: No additional sense information
>[10688.988321] sd 2:0:0:0: [sda] CDB: 
>[10688.988323] Write(10): 2a 00 10 a3 3c 00 00 04 00 00
>[10688.988336] end_request: I/O error, dev sda, sector 279133184
>[10689.003662] Buffer I/O error on device sda6, logical block 25396096
>[10689.019115] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10689.019232] Buffer I/O error on device sda6, logical block 25396097
>[10689.034376] Buffer I/O error on device sda6, logical block 25396098
>[10689.048615] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10689.048627] ata4.00: configured for UDMA/33
>[10689.048669] sd 3:0:0:0: [sdb]  
>[10689.048674] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10689.048679] sd 3:0:0:0: [sdb]  
>[10689.048682] Sense Key : Aborted Command [current] [descriptor]
>[10689.048689] Descriptor sense data with sense descriptors (in hex):
>[10689.048693]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10689.048712]         2a 81 b3 51 
>[10689.048723] sd 3:0:0:0: [sdb]  
>[10689.048728] Add. Sense: No additional sense information
>[10689.048733] sd 3:0:0:0: [sdb] CDB: 
>[10689.048737] Read(10): 28 00 2a 81 b3 00 00 01 00 00
>[10689.048754] end_request: I/O error, dev sdb, sector 713143040
>[10689.048759] Buffer I/O error on device sda6, logical block 25396099
>[10689.048762] Buffer I/O error on device sda6, logical block 25396100
>[10689.048766] Buffer I/O error on device sda6, logical block 25396101
>[10689.048770] Buffer I/O error on device sda6, logical block 25396102
>[10689.048773] Buffer I/O error on device sda6, logical block 25396103
>[10689.048776] Buffer I/O error on device sda6, logical block 25396104
>[10689.048779] Buffer I/O error on device sda6, logical block 25396105
>[10689.048783] Buffer I/O error on device sda6, logical block 25396106
>[10689.048787] Buffer I/O error on device sda6, logical block 25396107
>[10689.048791] Buffer I/O error on device sda6, logical block 25396108
>[10689.048793] Buffer I/O error on device sda6, logical block 25396109
>[10689.048797] Buffer I/O error on device sda6, logical block 25396110
>[10689.048800] Buffer I/O error on device sda6, logical block 25396111
>[10689.048803] Buffer I/O error on device sda6, logical block 25396112
>[10689.048806] Buffer I/O error on device sda6, logical block 25396113
>[10689.048810] Buffer I/O error on device sda6, logical block 25396114
>[10689.048813] Buffer I/O error on device sda6, logical block 25396115
>[10689.048816] Buffer I/O error on device sda6, logical block 25396116
>[10689.048819] Buffer I/O error on device sda6, logical block 25396117
>[10689.048822] Buffer I/O error on device sda6, logical block 25396118
>[10689.048825] Buffer I/O error on device sda6, logical block 25396119
>[10689.048828] Buffer I/O error on device sda6, logical block 25396120
>[10689.048831] Buffer I/O error on device sda6, logical block 25396121
>[10689.048834] Buffer I/O error on device sda6, logical block 25396122
>[10689.048837] Buffer I/O error on device sda6, logical block 25396123
>[10689.048840] Buffer I/O error on device sda6, logical block 25396124
>[10689.048843] Buffer I/O error on device sda6, logical block 25396125
>[10689.048846] Buffer I/O error on device sda6, logical block 25396126
>[10689.048849] Buffer I/O error on device sda6, logical block 25396127
>[10689.048852] Buffer I/O error on device sda6, logical block 25396128
>[10689.048855] Buffer I/O error on device sda6, logical block 25396129
>[10689.048858] Buffer I/O error on device sda6, logical block 25396130
>[10689.048861] Buffer I/O error on device sda6, logical block 25396131
>[10689.048864] Buffer I/O error on device sda6, logical block 25396132
>[10689.048867] Buffer I/O error on device sda6, logical block 25396133
>[10689.048870] Buffer I/O error on device sda6, logical block 25396134
>[10689.048873] Buffer I/O error on device sda6, logical block 25396135
>[10689.048876] Buffer I/O error on device sda6, logical block 25396136
>[10689.048878] Buffer I/O error on device sda6, logical block 25396137
>[10689.048882] Buffer I/O error on device sda6, logical block 25396138
>[10689.048885] Buffer I/O error on device sda6, logical block 25396139
>[10689.048942] Buffer I/O error on device sda6, logical block 25396140
>[10689.048944] Buffer I/O error on device sda6, logical block 25396141
>[10689.048947] Buffer I/O error on device sda6, logical block 25396142
>[10689.048950] Buffer I/O error on device sda6, logical block 25396143
>[10689.048954] Buffer I/O error on device sda6, logical block 25396144
>[10689.048957] Buffer I/O error on device sda6, logical block 25396145
>[10689.048960] Buffer I/O error on device sda6, logical block 25396146
>[10689.048963] Buffer I/O error on device sda6, logical block 25396147
>[10689.048966] Buffer I/O error on device sda6, logical block 25396148
>[10689.048968] Buffer I/O error on device sda6, logical block 25396149
>[10689.048972] Buffer I/O error on device sda6, logical block 25396150
>[10689.048975] Buffer I/O error on device sda6, logical block 25396151
>[10689.048977] Buffer I/O error on device sda6, logical block 25396152
>[10689.048981] Buffer I/O error on device sda6, logical block 25396153
>[10689.048983] Buffer I/O error on device sda6, logical block 25396154
>[10689.048986] Buffer I/O error on device sda6, logical block 25396155
>[10689.048989] Buffer I/O error on device sda6, logical block 25396156
>[10689.048992] Buffer I/O error on device sda6, logical block 25396157
>[10689.048995] Buffer I/O error on device sda6, logical block 25396158
>[10689.049063] Buffer I/O error on device sda6, logical block 25396159
>[10689.049068] Buffer I/O error on device sda6, logical block 25396160
>[10689.049072] Buffer I/O error on device sda6, logical block 25396161
>[10689.049076] Buffer I/O error on device sda6, logical block 25396162
>[10689.049080] Buffer I/O error on device sda6, logical block 25396163
>[10689.049083] Buffer I/O error on device sda6, logical block 25396164
>[10689.049087] Buffer I/O error on device sda6, logical block 25396165
>[10689.049091] Buffer I/O error on device sda6, logical block 25396166
>[10689.049096] Buffer I/O error on device sda6, logical block 25396167
>[10689.049099] Buffer I/O error on device sda6, logical block 25396168
>[10689.049103] Buffer I/O error on device sda6, logical block 25396169
>[10689.049107] Buffer I/O error on device sda6, logical block 25396170
>[10689.049110] Buffer I/O error on device sda6, logical block 25396171
>[10689.049114] Buffer I/O error on device sda6, logical block 25396172
>[10689.049117] Buffer I/O error on device sda6, logical block 25396173
>[10689.049121] Buffer I/O error on device sda6, logical block 25396174
>[10689.049125] Buffer I/O error on device sda6, logical block 25396175
>[10689.049128] Buffer I/O error on device sda6, logical block 25396176
>[10689.049132] Buffer I/O error on device sda6, logical block 25396177
>[10689.049135] Buffer I/O error on device sda6, logical block 25396178
>[10689.049138] Buffer I/O error on device sda6, logical block 25396179
>[10689.049142] Buffer I/O error on device sda6, logical block 25396180
>[10689.049147] Buffer I/O error on device sda6, logical block 25396181
>[10689.049152] Buffer I/O error on device sda6, logical block 25396182
>[10689.049155] Buffer I/O error on device sda6, logical block 25396183
>[10689.049158] Buffer I/O error on device sda6, logical block 25396184
>[10689.049161] Buffer I/O error on device sda6, logical block 25396185
>[10689.049165] Buffer I/O error on device sda6, logical block 25396186
>[10689.049169] Buffer I/O error on device sda6, logical block 25396187
>[10689.049172] Buffer I/O error on device sda6, logical block 25396188
>[10689.049176] Buffer I/O error on device sda6, logical block 25396189
>[10689.049180] Buffer I/O error on device sda6, logical block 25396190
>[10689.049183] Buffer I/O error on device sda6, logical block 25396191
>[10689.049187] Buffer I/O error on device sda6, logical block 25396192
>[10689.049190] Buffer I/O error on device sda6, logical block 25396193
>[10689.049194] Buffer I/O error on device sda6, logical block 25396194
>[10689.049197] Buffer I/O error on device sda6, logical block 25396195
>[10689.049200] Buffer I/O error on device sda6, logical block 25396196
>[10689.049203] Buffer I/O error on device sda6, logical block 25396197
>[10689.049206] Buffer I/O error on device sda6, logical block 25396198
>[10689.049209] Buffer I/O error on device sda6, logical block 25396199
>[10689.049213] Buffer I/O error on device sda6, logical block 25396200
>[10689.049216] Buffer I/O error on device sda6, logical block 25396201
>[10689.049219] Buffer I/O error on device sda6, logical block 25396202
>[10689.049222] Buffer I/O error on device sda6, logical block 25396203
>[10689.049225] Buffer I/O error on device sda6, logical block 25396204
>[10689.049229] Buffer I/O error on device sda6, logical block 25396205
>[10689.049232] Buffer I/O error on device sda6, logical block 25396206
>[10689.049235] Buffer I/O error on device sda6, logical block 25396207
>[10689.049239] Buffer I/O error on device sda6, logical block 25396208
>[10689.049242] Buffer I/O error on device sda6, logical block 25396209
>[10689.049246] Buffer I/O error on device sda6, logical block 25396210
>[10689.049277] Buffer I/O error on device sda6, logical block 25396211
>[10689.049281] Buffer I/O error on device sda6, logical block 25396212
>[10689.049284] Buffer I/O error on device sda6, logical block 25396213
>[10689.049287] Buffer I/O error on device sda6, logical block 25396214
>[10689.049291] Buffer I/O error on device sda6, logical block 25396215
>[10689.049295] Buffer I/O error on device sda6, logical block 25396216
>[10689.049298] Buffer I/O error on device sda6, logical block 25396217
>[10689.049302] Buffer I/O error on device sda6, logical block 25396218
>[10689.049307] Buffer I/O error on device sda6, logical block 25396219
>[10689.049310] Buffer I/O error on device sda6, logical block 25396220
>[10689.049313] Buffer I/O error on device sda6, logical block 25396221
>[10689.049316] Buffer I/O error on device sda6, logical block 25396222
>[10689.049320] Buffer I/O error on device sda6, logical block 25396223
>[10689.049327] EXT4-fs warning (device sda6): ext4_end_bio:319: I/O error writing to inode 24253019 (offset 3670016 size 524288 starting block 34891648)
>[10689.049361] ata3: EH complete
>[10689.500233] ata4: EH complete
>[10690.154360] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10690.157158] ata4.00: BMDMA2 stat 0x6d0009
>[10690.160212] ata4: SError: { Handshk UnrecFIS }
>[10690.163487] ata4.00: failed command: READ DMA EXT
>[10690.167374] ata4.00: cmd 25/00:00:00:f0:81/00:01:2a:00:00/e0 tag 0 dma 131072 in
>[10690.167374]          res 51/04:1f:e1:f0:81/00:00:2a:00:00/e0 Emask 0x1 (device error)
>[10690.187367] ata4.00: status: { DRDY ERR }
>[10690.193772] ata4.00: error: { ABRT }
>[10690.221407] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10690.224714] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10690.224735] ata4.00: configured for UDMA/33
>[10690.224775] sd 3:0:0:0: [sdb]  
>[10690.224780] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10690.224785] sd 3:0:0:0: [sdb]  
>[10690.224789] Sense Key : Aborted Command [current] [descriptor]
>[10690.224797] Descriptor sense data with sense descriptors (in hex):
>[10690.224801]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10690.224823]         2a 81 f0 e1 
>[10690.224834] sd 3:0:0:0: [sdb]  
>[10690.224840] Add. Sense: No additional sense information
>[10690.224845] sd 3:0:0:0: [sdb] CDB: 
>[10690.224849] Read(10): 28 00 2a 81 f0 00 00 01 00 00
>[10690.224868] end_request: I/O error, dev sdb, sector 713158656
>[10690.232418] ata4: EH complete
>[10692.040146] JBD2: Detected IO errors while flushing file data on sda6-8
>[10693.061462] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10697.038791] JBD2: Detected IO errors while flushing file data on sda6-8
>[10698.064587] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10699.113563] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10699.121767] ata3.00: BMDMA2 stat 0x686d0009
>[10699.130414] ata3: SError: { Handshk UnrecFIS }
>[10699.139631] ata3.00: failed command: READ DMA EXT
>[10699.149154] ata3.00: cmd 25/00:00:00:34:cd/00:01:2e:00:00/e0 tag 0 dma 131072 in
>[10699.149154]          res 51/04:5f:a1:34:cd/00:00:2e:00:00/e0 Emask 0x1 (device error)
>[10699.189965] ata3.00: status: { DRDY ERR }
>[10699.202197] ata3.00: error: { ABRT }
>[10699.218211] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10699.222460] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10699.222477] ata3.00: configured for UDMA/33
>[10699.222514] sd 2:0:0:0: [sda]  
>[10699.222518] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10699.222641] sd 2:0:0:0: [sda]  
>[10699.222647] Sense Key : Aborted Command [current] [descriptor]
>[10699.222655] Descriptor sense data with sense descriptors (in hex):
>[10699.222659]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10699.222677]         2e cd 34 a1 
>[10699.222687] sd 2:0:0:0: [sda]  
>[10699.222692] Add. Sense: No additional sense information
>[10699.222697] sd 2:0:0:0: [sda] CDB: 
>[10699.222701] Read(10): 28 00 2e cd 34 00 00 01 00 00
>[10699.222720] end_request: I/O error, dev sda, sector 785200128
>[10699.235257] ata3: EH complete
>[10703.066780] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10706.992244] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10707.005617] ata3.00: BMDMA2 stat 0x686d0009
>[10707.019304] ata3: SError: { Handshk UnrecFIS }
>[10707.032770] ata3.00: failed command: READ DMA EXT
>[10707.046457] ata3.00: cmd 25/00:00:00:ad:84/00:01:12:00:00/e0 tag 0 dma 131072 in
>[10707.046457]          res 51/04:2f:d1:ad:84/00:00:12:00:00/e0 Emask 0x1 (device error)
>[10707.108113] ata3.00: status: { DRDY ERR }
>[10707.123590] ata3.00: error: { ABRT }
>[10707.141891] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10707.145646] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10707.145653] ata3.00: configured for UDMA/33
>[10707.145678] sd 2:0:0:0: [sda]  
>[10707.145681] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10707.145685] sd 2:0:0:0: [sda]  
>[10707.145687] Sense Key : Aborted Command [current] [descriptor]
>[10707.145692] Descriptor sense data with sense descriptors (in hex):
>[10707.145695]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10707.145709]         12 84 ad d1 
>[10707.145715] sd 2:0:0:0: [sda]  
>[10707.145719] Add. Sense: No additional sense information
>[10707.145722] sd 2:0:0:0: [sda] CDB: 
>[10707.145724] Read(10): 28 00 12 84 ad 00 00 01 00 00
>[10707.145737] end_request: I/O error, dev sda, sector 310684928
>[10707.161366] ata3: EH complete
>[10713.320387] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10718.261387] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10735.324536] JBD2: Detected IO errors while flushing file data on sdb1-8
>[10764.313818] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10764.329544] ata4.00: BMDMA2 stat 0x6d0009
>[10764.344804] ata4: SError: { Handshk UnrecFIS }
>[10764.360198] ata4.00: failed command: READ DMA EXT
>[10764.375505] ata4.00: cmd 25/00:00:00:ed:13/00:01:1b:00:00/e0 tag 0 dma 131072 in
>[10764.375505]          res 51/04:5f:a1:ed:13/00:00:1b:00:00/e0 Emask 0x1 (device error)
>[10764.436504] ata4.00: status: { DRDY ERR }
>[10764.452469] ata4.00: error: { ABRT }
>[10764.482629] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10764.486815] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10764.486832] ata4.00: configured for UDMA/33
>[10764.486875] sd 3:0:0:0: [sdb]  
>[10764.486882] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10764.486888] sd 3:0:0:0: [sdb]  
>[10764.486892] Sense Key : Aborted Command [current] [descriptor]
>[10764.486899] Descriptor sense data with sense descriptors (in hex):
>[10764.486903]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10764.486922]         1b 13 ed a1 
>[10764.486931] sd 3:0:0:0: [sdb]  
>[10764.486937] Add. Sense: No additional sense information
>[10764.486942] sd 3:0:0:0: [sdb] CDB: 
>[10764.486945] Read(10): 28 00 1b 13 ed 00 00 01 00 00
>[10764.486963] end_request: I/O error, dev sdb, sector 454290688
>[10764.502767] ata4: EH complete
>[10781.825571] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10781.841259] ata4.00: BMDMA2 stat 0x6d0009
>[10781.856548] ata4: SError: { Handshk UnrecFIS }
>[10781.871457] ata4.00: failed command: READ DMA
>[10781.886046] ata4.00: cmd c8/00:00:00:91:db/00:00:00:00:00/ee tag 0 dma 131072 in
>[10781.886046]          res 51/04:5f:a1:91:db/00:00:1b:00:00/ee Emask 0x1 (device error)
>[10781.946323] ata4.00: status: { DRDY ERR }
>[10781.961494] ata4.00: error: { ABRT }
>[10781.991870] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10781.995575] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10781.995581] ata4.00: configured for UDMA/33
>[10781.995604] sd 3:0:0:0: [sdb]  
>[10781.995608] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10781.995611] sd 3:0:0:0: [sdb]  
>[10781.995613] Sense Key : Aborted Command [current] [descriptor]
>[10781.995618] Descriptor sense data with sense descriptors (in hex):
>[10781.995621]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10781.995635]         0e db 91 a1 
>[10781.995641] sd 3:0:0:0: [sdb]  
>[10781.995644] Add. Sense: No additional sense information
>[10781.995647] sd 3:0:0:0: [sdb] CDB: 
>[10781.995649] Read(10): 28 00 0e db 91 00 00 01 00 00
>[10781.995662] end_request: I/O error, dev sdb, sector 249270528
>[10782.011122] ata4: EH complete
>[10782.141943] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10782.157581] ata4.00: BMDMA2 stat 0x6d0009
>[10782.172723] ata4: SError: { Handshk UnrecFIS }
>[10782.187563] ata4.00: failed command: READ DMA
>[10782.202161] ata4.00: cmd c8/00:00:00:a7:db/00:00:00:00:00/ee tag 0 dma 131072 in
>[10782.202161]          res 51/04:ef:11:a7:db/00:00:1b:00:00/ee Emask 0x1 (device error)
>[10782.262313] ata4.00: status: { DRDY ERR }
>[10782.277757] ata4.00: error: { ABRT }
>[10782.307204] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10782.313667] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10782.313673] ata4.00: configured for UDMA/33
>[10782.313697] sd 3:0:0:0: [sdb]  
>[10782.313701] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10782.313704] sd 3:0:0:0: [sdb]  
>[10782.313707] Sense Key : Aborted Command [current] [descriptor]
>[10782.313712] Descriptor sense data with sense descriptors (in hex):
>[10782.313714]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10782.313728]         0e db a7 11 
>[10782.313735] sd 3:0:0:0: [sdb]  
>[10782.313738] Add. Sense: No additional sense information
>[10782.313742] sd 3:0:0:0: [sdb] CDB: 
>[10782.313744] Read(10): 28 00 0e db a7 00 00 01 00 00
>[10782.313757] end_request: I/O error, dev sdb, sector 249276160
>[10782.329156] ata4: EH complete
>[10785.229771] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10785.245900] ata4.00: BMDMA2 stat 0x6d0009
>[10785.261888] ata4: SError: { Handshk UnrecFIS }
>[10785.277359] ata4.00: failed command: READ DMA EXT
>[10785.292654] ata4.00: cmd 25/00:00:00:68:fc/00:01:1b:00:00/e0 tag 0 dma 131072 in
>[10785.292654]          res 51/04:9f:61:68:fc/00:00:1b:00:00/e0 Emask 0x1 (device error)
>[10785.353399] ata4.00: status: { DRDY ERR }
>[10785.369293] ata4.00: error: { ABRT }
>[10785.402458] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10785.408804] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10785.408827] ata4.00: configured for UDMA/33
>[10785.408878] sd 3:0:0:0: [sdb]  
>[10785.408883] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10785.408888] sd 3:0:0:0: [sdb]  
>[10785.408892] Sense Key : Aborted Command [current] [descriptor]
>[10785.408901] Descriptor sense data with sense descriptors (in hex):
>[10785.408904]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10785.408923]         1b fc 68 61 
>[10785.408932] sd 3:0:0:0: [sdb]  
>[10785.408938] Add. Sense: No additional sense information
>[10785.408943] sd 3:0:0:0: [sdb] CDB: 
>[10785.408946] Read(10): 28 00 1b fc 68 00 00 01 00 00
>[10785.408964] end_request: I/O error, dev sdb, sector 469526528
>[10785.425515] ata4: EH complete
>[10789.482037] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10789.497578] ata4.00: BMDMA2 stat 0x6d0009
>[10789.512640] ata4: SError: { Handshk UnrecFIS }
>[10789.527364] ata4.00: failed command: READ DMA EXT
>[10789.541850] ata4.00: cmd 25/00:00:00:a3:bd/00:01:1b:00:00/e0 tag 0 dma 131072 in
>[10789.541850]          res 51/04:ef:11:a3:bd/00:00:1b:00:00/e0 Emask 0x1 (device error)
>[10789.601818] ata4.00: status: { DRDY ERR }
>[10789.616950] ata4.00: error: { ABRT }
>[10789.651170] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10789.654598] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10789.654604] ata4.00: configured for UDMA/33
>[10789.654630] sd 3:0:0:0: [sdb]  
>[10789.654633] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10789.654636] sd 3:0:0:0: [sdb]  
>[10789.654639] Sense Key : Aborted Command [current] [descriptor]
>[10789.654644] Descriptor sense data with sense descriptors (in hex):
>[10789.654646]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10789.654661]         1b bd a3 11 
>[10789.654667] sd 3:0:0:0: [sdb]  
>[10789.654670] Add. Sense: No additional sense information
>[10789.654674] sd 3:0:0:0: [sdb] CDB: 
>[10789.654676] Read(10): 28 00 1b bd a3 00 00 01 00 00
>[10789.654689] end_request: I/O error, dev sdb, sector 465412864
>[10789.670082] ata4: EH complete
>[10799.409088] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10799.425824] ata3.00: BMDMA2 stat 0x616d0009
>[10799.442305] ata3: SError: { Handshk UnrecFIS }
>[10799.458364] ata3.00: failed command: READ DMA EXT
>[10799.473757] ata3.00: cmd 25/00:00:00:95:83/00:01:35:00:00/e0 tag 0 dma 131072 in
>[10799.473757]          res 51/04:7f:81:95:83/00:00:35:00:00/e0 Emask 0x1 (device error)
>[10799.534557] ata3.00: status: { DRDY ERR }
>[10799.550471] ata3.00: error: { ABRT }
>[10799.570766] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10799.574781] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10799.574819] ata3.00: configured for UDMA/33
>[10799.574846] sd 2:0:0:0: [sda]  
>[10799.574849] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10799.574866] sd 2:0:0:0: [sda]  
>[10799.574869] Sense Key : Aborted Command [current] [descriptor]
>[10799.574874] Descriptor sense data with sense descriptors (in hex):
>[10799.574877]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10799.574891]         35 83 95 81 
>[10799.574897] sd 2:0:0:0: [sda]  
>[10799.574900] Add. Sense: No additional sense information
>[10799.574904] sd 2:0:0:0: [sda] CDB: 
>[10799.574906] Read(10): 28 00 35 83 95 00 00 01 00 00
>[10799.574918] end_request: I/O error, dev sda, sector 897815808
>[10799.590584] ata3: EH complete
>[10804.997480] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10805.013872] ata3.00: BMDMA2 stat 0x686d0009
>[10805.029301] ata3: SError: { Handshk UnrecFIS }
>[10805.044248] ata3.00: failed command: READ DMA EXT
>[10805.058946] ata3.00: cmd 25/00:00:00:8e:b2/00:01:20:00:00/e0 tag 0 dma 131072 in
>[10805.058946]          res 51/04:af:51:8e:b2/00:00:20:00:00/e0 Emask 0x1 (device error)
>[10805.119544] ata3.00: status: { DRDY ERR }
>[10805.136481] ata3.00: error: { ABRT }
>[10805.154591] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10805.158144] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10805.158151] ata3.00: configured for UDMA/33
>[10805.158175] sd 2:0:0:0: [sda]  
>[10805.158178] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10805.158181] sd 2:0:0:0: [sda]  
>[10805.158184] Sense Key : Aborted Command [current] [descriptor]
>[10805.158189] Descriptor sense data with sense descriptors (in hex):
>[10805.158191]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10805.158205]         20 b2 8e 51 
>[10805.158212] sd 2:0:0:0: [sda]  
>[10805.158215] Add. Sense: No additional sense information
>[10805.158218] sd 2:0:0:0: [sda] CDB: 
>[10805.158221] Read(10): 28 00 20 b2 8e 00 00 01 00 00
>[10805.158233] end_request: I/O error, dev sda, sector 548572672
>[10805.173728] ata3: EH complete
>[10805.494389] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10805.510615] ata4.00: BMDMA2 stat 0x6d0009
>[10805.526567] ata4: SError: { Handshk UnrecFIS }
>[10805.542067] ata4.00: failed command: READ DMA EXT
>[10805.557196] ata4.00: cmd 25/00:00:00:20:c6/00:01:1b:00:00/e0 tag 0 dma 131072 in
>[10805.557196]          res 51/04:af:51:20:c6/00:00:1b:00:00/e0 Emask 0x1 (device error)
>[10805.618267] ata4.00: status: { DRDY ERR }
>[10805.636720] ata4.00: error: { ABRT }
>[10805.666833] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10805.673179] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10805.673190] ata4.00: configured for UDMA/33
>[10805.673222] sd 3:0:0:0: [sdb]  
>[10805.673225] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10805.673229] sd 3:0:0:0: [sdb]  
>[10805.673231] Sense Key : Aborted Command [current] [descriptor]
>[10805.673237] Descriptor sense data with sense descriptors (in hex):
>[10805.673239]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10805.673253]         1b c6 20 51 
>[10805.673259] sd 3:0:0:0: [sdb]  
>[10805.673263] Add. Sense: No additional sense information
>[10805.673267] sd 3:0:0:0: [sdb] CDB: 
>[10805.673269] Read(10): 28 00 1b c6 20 00 00 01 00 00
>[10805.673317] end_request: I/O error, dev sdb, sector 465969152
>[10805.689285] ata4: EH complete
>[10874.641081] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10874.656904] ata4.00: BMDMA2 stat 0x6d0009
>[10874.672232] ata4: SError: { Handshk UnrecFIS }
>[10874.687258] ata4.00: failed command: READ DMA
>[10874.702301] ata4.00: cmd c8/00:00:00:51:8e/00:00:00:00:00/e6 tag 0 dma 131072 in
>[10874.702301]          res 51/04:8f:71:51:8e/00:00:1b:00:00/e6 Emask 0x1 (device error)
>[10874.762713] ata4.00: status: { DRDY ERR }
>[10874.778288] ata4.00: error: { ABRT }
>[10874.807943] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10874.811770] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10874.811793] ata4.00: configured for UDMA/33
>[10874.811849] sd 3:0:0:0: [sdb]  
>[10874.811854] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10874.811859] sd 3:0:0:0: [sdb]  
>[10874.811863] Sense Key : Aborted Command [current] [descriptor]
>[10874.811871] Descriptor sense data with sense descriptors (in hex):
>[10874.811874]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10874.811893]         06 8e 51 71 
>[10874.811903] sd 3:0:0:0: [sdb]  
>[10874.811908] Add. Sense: No additional sense information
>[10874.811913] sd 3:0:0:0: [sdb] CDB: 
>[10874.811916] Read(10): 28 00 06 8e 51 00 00 01 00 00
>[10874.811933] end_request: I/O error, dev sdb, sector 109990144
>[10874.828211] ata4: EH complete
>[10884.883761] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10884.899832] ata4.00: BMDMA2 stat 0x6d0009
>[10884.915800] ata4: SError: { Handshk UnrecFIS }
>[10884.931585] ata4.00: failed command: READ DMA
>[10884.946811] ata4.00: cmd c8/00:00:00:a3:a3/00:00:00:00:00/e8 tag 0 dma 131072 in
>[10884.946811]          res 51/04:bf:41:a3:a3/00:00:1b:00:00/e8 Emask 0x1 (device error)
>[10885.007481] ata4.00: status: { DRDY ERR }
>[10885.023674] ata4.00: error: { ABRT }
>[10885.056465] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10885.060228] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10885.060248] ata4.00: configured for UDMA/33
>[10885.060293] sd 3:0:0:0: [sdb]  
>[10885.060298] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10885.060303] sd 3:0:0:0: [sdb]  
>[10885.060307] Sense Key : Aborted Command [current] [descriptor]
>[10885.060314] Descriptor sense data with sense descriptors (in hex):
>[10885.060317]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10885.060334]         08 a3 a3 41 
>[10885.060342] sd 3:0:0:0: [sdb]  
>[10885.060346] Add. Sense: No additional sense information
>[10885.060350] sd 3:0:0:0: [sdb] CDB: 
>[10885.060353] Read(10): 28 00 08 a3 a3 00 00 01 00 00
>[10885.060369] end_request: I/O error, dev sdb, sector 144941824
>[10885.076533] ata4: EH complete
>[10888.263150] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10888.278854] ata4.00: BMDMA2 stat 0x6d0009
>[10888.293985] ata4: SError: { Handshk UnrecFIS }
>[10888.308836] ata4.00: failed command: READ DMA EXT
>[10888.323429] ata4.00: cmd 25/00:00:00:e6:86/00:01:28:00:00/e0 tag 0 dma 131072 in
>[10888.323429]          res 51/04:ef:11:e6:86/00:00:28:00:00/e0 Emask 0x1 (device error)
>[10888.383507] ata4.00: status: { DRDY ERR }
>[10888.398673] ata4.00: error: { ABRT }
>[10888.433683] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10888.436594] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10888.436600] ata4.00: configured for UDMA/33
>[10888.436625] sd 3:0:0:0: [sdb]  
>[10888.436628] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10888.436631] sd 3:0:0:0: [sdb]  
>[10888.436634] Sense Key : Aborted Command [current] [descriptor]
>[10888.436639] Descriptor sense data with sense descriptors (in hex):
>[10888.436641]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10888.436655]         28 86 e6 11 
>[10888.436662] sd 3:0:0:0: [sdb]  
>[10888.436665] Add. Sense: No additional sense information
>[10888.436669] sd 3:0:0:0: [sdb] CDB: 
>[10888.436671] Read(10): 28 00 28 86 e6 00 00 01 00 00
>[10888.436684] end_request: I/O error, dev sdb, sector 679929344
>[10888.452143] ata4: EH complete
>[10889.136971] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10889.153072] ata4.00: BMDMA2 stat 0x6d0009
>[10889.168594] ata4: SError: { Handshk UnrecFIS }
>[10889.183604] ata4.00: failed command: READ DMA EXT
>[10889.198526] ata4.00: cmd 25/00:00:00:83:87/00:01:28:00:00/e0 tag 0 dma 131072 in
>[10889.198526]          res 51/04:ef:11:83:87/00:00:28:00:00/e0 Emask 0x1 (device error)
>[10889.259154] ata4.00: status: { DRDY ERR }
>[10889.274446] ata4.00: error: { ABRT }
>[10889.313303] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10889.316594] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10889.316602] ata4.00: configured for UDMA/33
>[10889.316629] sd 3:0:0:0: [sdb]  
>[10889.316632] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10889.316636] sd 3:0:0:0: [sdb]  
>[10889.316638] Sense Key : Aborted Command [current] [descriptor]
>[10889.316643] Descriptor sense data with sense descriptors (in hex):
>[10889.316646]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10889.316660]         28 87 83 11 
>[10889.316666] sd 3:0:0:0: [sdb]  
>[10889.316669] Add. Sense: No additional sense information
>[10889.316673] sd 3:0:0:0: [sdb] CDB: 
>[10889.316675] Read(10): 28 00 28 87 83 00 00 01 00 00
>[10889.316688] end_request: I/O error, dev sdb, sector 679969536
>[10889.332104] ata4: EH complete
>[10892.454209] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10892.471506] ata4.00: BMDMA2 stat 0x6d0009
>[10892.487921] ata4: SError: { Handshk UnrecFIS }
>[10892.503605] ata4.00: failed command: READ DMA EXT
>[10892.518565] ata4.00: cmd 25/00:00:00:2d:06/00:01:1d:00:00/e0 tag 0 dma 131072 in
>[10892.518565]          res 51/04:1f:e1:2d:06/00:00:1d:00:00/e0 Emask 0x1 (device error)
>[10892.581988] ata4.00: status: { DRDY ERR }
>[10892.597536] ata4.00: error: { ABRT }
>[10892.634434] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10892.637926] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10892.637935] ata4.00: configured for UDMA/33
>[10892.637962] sd 3:0:0:0: [sdb]  
>[10892.637965] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10892.637969] sd 3:0:0:0: [sdb]  
>[10892.637971] Sense Key : Aborted Command [current] [descriptor]
>[10892.637976] Descriptor sense data with sense descriptors (in hex):
>[10892.637979]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10892.637993]         1d 06 2d e1 
>[10892.637999] sd 3:0:0:0: [sdb]  
>[10892.638021] Add. Sense: No additional sense information
>[10892.638039] sd 3:0:0:0: [sdb] CDB: 
>[10892.638048] Read(10): 28 00 1d 06 2d 00 00 01 00 00
>[10892.638066] end_request: I/O error, dev sdb, sector 486944000
>[10892.653853] ata4: EH complete
>[10904.983445] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10904.999313] ata4.00: BMDMA2 stat 0x6d0009
>[10905.015005] ata4: SError: { Handshk UnrecFIS }
>[10905.030278] ata4.00: failed command: READ DMA EXT
>[10905.045288] ata4.00: cmd 25/00:00:00:ed:0d/00:01:1d:00:00/e0 tag 0 dma 131072 in
>[10905.045288]          res 51/04:2f:d1:ed:0d/00:00:1d:00:00/e0 Emask 0x1 (device error)
>[10905.105822] ata4.00: status: { DRDY ERR }
>[10905.121367] ata4.00: error: { ABRT }
>[10905.154704] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10905.158711] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10905.158719] ata4.00: configured for UDMA/33
>[10905.158757] sd 3:0:0:0: [sdb]  
>[10905.158761] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10905.158765] sd 3:0:0:0: [sdb]  
>[10905.158768] Sense Key : Aborted Command [current] [descriptor]
>[10905.158773] Descriptor sense data with sense descriptors (in hex):
>[10905.158776]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10905.158790]         1d 0d ed d1 
>[10905.158796] sd 3:0:0:0: [sdb]  
>[10905.158800] Add. Sense: No additional sense information
>[10905.158803] sd 3:0:0:0: [sdb] CDB: 
>[10905.158805] Read(10): 28 00 1d 0d ed 00 00 01 00 00
>[10905.158818] end_request: I/O error, dev sdb, sector 487451904
>[10905.178238] ata4: EH complete
>[10905.370562] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10905.386206] ata4.00: BMDMA2 stat 0x6d0009
>[10905.401417] ata4: SError: { Handshk UnrecFIS }
>[10905.416408] ata4.00: failed command: READ DMA EXT
>[10905.431041] ata4.00: cmd 25/00:00:00:0f:11/00:01:1d:00:00/e0 tag 0 dma 131072 in
>[10905.431041]          res 51/04:1f:e1:0f:11/00:00:1d:00:00/e0 Emask 0x1 (device error)
>[10905.491123] ata4.00: status: { DRDY ERR }
>[10905.506327] ata4.00: error: { ABRT }
>[10905.544733] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10905.547579] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10905.547586] ata4.00: configured for UDMA/33
>[10905.547610] sd 3:0:0:0: [sdb]  
>[10905.547614] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10905.547617] sd 3:0:0:0: [sdb]  
>[10905.547620] Sense Key : Aborted Command [current] [descriptor]
>[10905.547625] Descriptor sense data with sense descriptors (in hex):
>[10905.547627]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10905.547642]         1d 11 0f e1 
>[10905.547648] sd 3:0:0:0: [sdb]  
>[10905.547652] Add. Sense: No additional sense information
>[10905.547655] sd 3:0:0:0: [sdb] CDB: 
>[10905.547657] Read(10): 28 00 1d 11 0f 00 00 01 00 00
>[10905.547671] end_request: I/O error, dev sdb, sector 487657216
>[10905.563082] ata4: EH complete
>[10908.674770] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10908.692665] ata4.00: BMDMA2 stat 0x6d0009
>[10908.709877] ata4: SError: { Handshk UnrecFIS }
>[10908.726761] ata4.00: failed command: READ DMA
>[10908.743186] ata4.00: cmd c8/00:00:00:31:31/00:00:00:00:00/e6 tag 0 dma 131072 in
>[10908.743186]          res 51/04:3f:c1:31:31/00:00:1d:00:00/e6 Emask 0x1 (device error)
>[10908.808665] ata4.00: status: { DRDY ERR }
>[10908.823862] ata4.00: error: { ABRT }
>[10908.855709] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10908.858583] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10908.858589] ata4.00: configured for UDMA/33
>[10908.858614] sd 3:0:0:0: [sdb]  
>[10908.858617] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10908.858620] sd 3:0:0:0: [sdb]  
>[10908.858623] Sense Key : Aborted Command [current] [descriptor]
>[10908.858628] Descriptor sense data with sense descriptors (in hex):
>[10908.858630]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10908.858644]         06 31 31 c1 
>[10908.858650] sd 3:0:0:0: [sdb]  
>[10908.858654] Add. Sense: No additional sense information
>[10908.858657] sd 3:0:0:0: [sdb] CDB: 
>[10908.858659] Read(10): 28 00 06 31 31 00 00 01 00 00
>[10908.858672] end_request: I/O error, dev sdb, sector 103887104
>[10908.874127] ata4: EH complete
>[10909.444483] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10909.460113] ata4.00: BMDMA2 stat 0x6d0009
>[10909.475170] ata4: SError: { Handshk UnrecFIS }
>[10909.490176] ata4.00: failed command: READ DMA
>[10909.504726] ata4.00: cmd c8/00:00:00:a7:34/00:00:00:00:00/e6 tag 0 dma 131072 in
>[10909.504726]          res 51/04:9f:61:a7:34/00:00:1d:00:00/e6 Emask 0x1 (device error)
>[10909.564464] ata4.00: status: { DRDY ERR }
>[10909.579565] ata4.00: error: { ABRT }
>[10909.619147] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10909.626299] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10909.626306] ata4.00: configured for UDMA/33
>[10909.626331] sd 3:0:0:0: [sdb]  
>[10909.626335] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10909.626338] sd 3:0:0:0: [sdb]  
>[10909.626341] Sense Key : Aborted Command [current] [descriptor]
>[10909.626346] Descriptor sense data with sense descriptors (in hex):
>[10909.626348]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10909.626362]         06 34 a7 61 
>[10909.626369] sd 3:0:0:0: [sdb]  
>[10909.626372] Add. Sense: No additional sense information
>[10909.626375] sd 3:0:0:0: [sdb] CDB: 
>[10909.626378] Read(10): 28 00 06 34 a7 00 00 01 00 00
>[10909.626390] end_request: I/O error, dev sdb, sector 104113920
>[10909.643423] ata4: EH complete
>[10955.705107] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10955.721256] ata4.00: BMDMA2 stat 0x6d0009
>[10955.737403] ata4: SError: { Handshk UnrecFIS }
>[10955.752841] ata4.00: failed command: READ DMA
>[10955.767876] ata4.00: cmd c8/00:00:00:d0:35/00:00:00:00:00/ef tag 0 dma 131072 in
>[10955.767876]          res 51/04:8f:71:d0:35/00:00:1d:00:00/ef Emask 0x1 (device error)
>[10955.828517] ata4.00: status: { DRDY ERR }
>[10955.844423] ata4.00: error: { ABRT }
>[10955.873664] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10955.876572] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10955.876579] ata4.00: configured for UDMA/33
>[10955.876604] sd 3:0:0:0: [sdb]  
>[10955.876607] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10955.876611] sd 3:0:0:0: [sdb]  
>[10955.876613] Sense Key : Aborted Command [current] [descriptor]
>[10955.876618] Descriptor sense data with sense descriptors (in hex):
>[10955.876621]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10955.876635]         0f 35 d0 71 
>[10955.876641] sd 3:0:0:0: [sdb]  
>[10955.876645] Add. Sense: No additional sense information
>[10955.876648] sd 3:0:0:0: [sdb] CDB: 
>[10955.876650] Read(10): 28 00 0f 35 d0 00 00 01 00 00
>[10955.876663] end_request: I/O error, dev sdb, sector 255184896
>[10955.891946] ata4: EH complete
>[10961.329407] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10961.345029] ata3.00: BMDMA2 stat 0x686d0009
>[10961.360193] ata3: SError: { Handshk UnrecFIS }
>[10961.374908] ata3.00: failed command: READ DMA EXT
>[10961.389423] ata3.00: cmd 25/00:00:00:6d:b7/00:01:1b:00:00/e0 tag 0 dma 131072 in
>[10961.389423]          res 51/04:ef:11:6d:b7/00:00:1b:00:00/e0 Emask 0x1 (device error)
>[10961.450197] ata3.00: status: { DRDY ERR }
>[10961.465645] ata3.00: error: { ABRT }
>[10961.484065] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10961.487552] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10961.487559] ata3.00: configured for UDMA/33
>[10961.487582] sd 2:0:0:0: [sda]  
>[10961.487585] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10961.487589] sd 2:0:0:0: [sda]  
>[10961.487591] Sense Key : Aborted Command [current] [descriptor]
>[10961.487596] Descriptor sense data with sense descriptors (in hex):
>[10961.487599]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10961.487613]         1b b7 6d 11 
>[10961.487620] sd 2:0:0:0: [sda]  
>[10961.487623] Add. Sense: No additional sense information
>[10961.487627] sd 2:0:0:0: [sda] CDB: 
>[10961.487629] Read(10): 28 00 1b b7 6d 00 00 01 00 00
>[10961.487641] end_request: I/O error, dev sda, sector 465005824
>[10961.503122] ata3: EH complete
>[10961.729847] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10961.745407] ata3.00: BMDMA2 stat 0x686d0009
>[10961.760705] ata3: SError: { Handshk UnrecFIS }
>[10961.775655] ata3.00: failed command: READ DMA EXT
>[10961.790361] ata3.00: cmd 25/00:00:00:1d:ba/00:01:1b:00:00/e0 tag 0 dma 131072 in
>[10961.790361]          res 51/04:0f:f1:1d:ba/00:00:1b:00:00/e0 Emask 0x1 (device error)
>[10961.850725] ata3.00: status: { DRDY ERR }
>[10961.866327] ata3.00: error: { ABRT }
>[10961.885463] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10961.889462] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10961.889468] ata3.00: configured for UDMA/33
>[10961.889492] sd 2:0:0:0: [sda]  
>[10961.889496] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10961.889499] sd 2:0:0:0: [sda]  
>[10961.889501] Sense Key : Aborted Command [current] [descriptor]
>[10961.889507] Descriptor sense data with sense descriptors (in hex):
>[10961.889509]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10961.889523]         1b ba 1d f1 
>[10961.889529] sd 2:0:0:0: [sda]  
>[10961.889533] Add. Sense: No additional sense information
>[10961.889536] sd 2:0:0:0: [sda] CDB: 
>[10961.889539] Read(10): 28 00 1b ba 1d 00 00 01 00 00
>[10961.889551] end_request: I/O error, dev sda, sector 465181952
>[10961.904970] ata3: EH complete
>[10962.104201] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10962.119799] ata3.00: BMDMA2 stat 0x686d0009
>[10962.136758] ata3: SError: { Handshk UnrecFIS }
>[10962.151539] ata3.00: failed command: READ DMA EXT
>[10962.166149] ata3.00: cmd 25/00:00:00:a1:ba/00:01:1b:00:00/e0 tag 0 dma 131072 in
>[10962.166149]          res 51/04:ef:11:a1:ba/00:00:1b:00:00/e0 Emask 0x1 (device error)
>[10962.226425] ata3.00: status: { DRDY ERR }
>[10962.241629] ata3.00: error: { ABRT }
>[10962.259856] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10962.263830] ata3.00: failed to get Identify Device Data, Emask 0x1
>[10962.263837] ata3.00: configured for UDMA/33
>[10962.263862] sd 2:0:0:0: [sda]  
>[10962.263866] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10962.263869] sd 2:0:0:0: [sda]  
>[10962.263871] Sense Key : Aborted Command [current] [descriptor]
>[10962.263877] Descriptor sense data with sense descriptors (in hex):
>[10962.263879]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10962.263893]         1b ba a1 11 
>[10962.263899] sd 2:0:0:0: [sda]  
>[10962.263903] Add. Sense: No additional sense information
>[10962.263906] sd 2:0:0:0: [sda] CDB: 
>[10962.263908] Read(10): 28 00 1b ba a1 00 00 01 00 00
>[10962.263921] end_request: I/O error, dev sda, sector 465215744
>[10962.279397] ata3: EH complete
>[10965.653608] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10965.669651] ata4.00: BMDMA2 stat 0x6d0009
>[10965.685446] ata4: SError: { Handshk UnrecFIS }
>[10965.700927] ata4.00: failed command: READ DMA
>[10965.715935] ata4.00: cmd c8/00:00:00:bc:91/00:00:00:00:00/ef tag 0 dma 131072 in
>[10965.715935]          res 51/04:4f:b1:bc:91/00:00:1d:00:00/ef Emask 0x1 (device error)
>[10965.776963] ata4.00: status: { DRDY ERR }
>[10965.792637] ata4.00: error: { ABRT }
>[10965.823281] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10965.827004] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10965.827090] ata4.00: configured for UDMA/33
>[10965.827155] sd 3:0:0:0: [sdb]  
>[10965.827161] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10965.827168] sd 3:0:0:0: [sdb]  
>[10965.827173] Sense Key : Aborted Command [current] [descriptor]
>[10965.827182] Descriptor sense data with sense descriptors (in hex):
>[10965.827186]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10965.827205]         0f 91 bc b1 
>[10965.827214] sd 3:0:0:0: [sdb]  
>[10965.827219] Add. Sense: No additional sense information
>[10965.827224] sd 3:0:0:0: [sdb] CDB: 
>[10965.827228] Read(10): 28 00 0f 91 bc 00 00 01 00 00
>[10965.827245] end_request: I/O error, dev sdb, sector 261209088
>[10965.843295] ata4: EH complete
>[10967.057001] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x0
>[10967.072967] ata4.00: BMDMA2 stat 0x6d0009
>[10967.088822] ata4: SError: { Handshk UnrecFIS }
>[10967.104414] ata4.00: failed command: READ DMA
>[10967.119224] ata4.00: cmd c8/00:00:00:b4:95/00:00:00:00:00/ef tag 0 dma 131072 in
>[10967.119224]          res 51/04:6f:91:b4:95/00:00:1d:00:00/ef Emask 0x1 (device error)
>[10967.181566] ata4.00: status: { DRDY ERR }
>[10967.197118] ata4.00: error: { ABRT }
>[10967.225771] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10967.229629] ata4.00: failed to get Identify Device Data, Emask 0x1
>[10967.229638] ata4.00: configured for UDMA/33
>[10967.229665] sd 3:0:0:0: [sdb]  
>[10967.229668] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[10967.229672] sd 3:0:0:0: [sdb]  
>[10967.229674] Sense Key : Aborted Command [current] [descriptor]
>[10967.229679] Descriptor sense data with sense descriptors (in hex):
>[10967.229682]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[10967.229696]         0f 95 b4 91 
>[10967.229703] sd 3:0:0:0: [sdb]  
>[10967.229706] Add. Sense: No additional sense information
>[10967.229710] sd 3:0:0:0: [sdb] CDB: 
>[10967.229712] Read(10): 28 00 0f 95 b4 00 00 01 00 00
>[10967.229725] end_request: I/O error, dev sdb, sector 261469184
>[10967.245473] ata4: EH complete
>[10986.631843] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10986.631843] Do you have a strange power saving mode enabled?
>[10986.631843] Dazed and confused, but trying to continue
>[10986.631843] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10986.631843] Do you have a strange power saving mode enabled?
>[10986.631843] Dazed and confused, but trying to continue
>[10986.631843] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[10986.631843] Do you have a strange power saving mode enabled?
>[10986.631843] Dazed and confused, but trying to continue
>[10986.631843] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10986.631843] Do you have a strange power saving mode enabled?
>[10986.631843] Dazed and confused, but trying to continue
>[10986.631843] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10986.631843] Do you have a strange power saving mode enabled?
>[10986.631843] Dazed and confused, but trying to continue
>[10986.631843] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[10986.631843] Do you have a strange power saving mode enabled?
>[10986.631843] Dazed and confused, but trying to continue
>[10986.631843] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[10986.631843] Do you have a strange power saving mode enabled?
>[10986.631843] Dazed and confused, but trying to continue
>[10986.631843] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[10986.631843] Do you have a strange power saving mode enabled?
>[10986.631843] Dazed and confused, but trying to continue
>[10987.025026] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[10987.033020] ------------[ cut here ]------------
>[10987.033020] WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0xac/0xd0()
>[10987.033020] Hardware name: Sun Fire(tm) V60 370-6037               
>[10987.033020] Watchdog detected hard LOCKUP on cpu 2
>[10987.033020] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[10987.033020] Pid: 193, comm: scsi_eh_3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1
>[10987.033020] Call Trace:
>[10987.033020]  [<c044d6a2>] warn_slowpath_common+0x72/0xa0
>[10987.033020]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[10987.033020]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[10987.033020]  [<c04ccb60>] ? touch_nmi_watchdog+0x70/0x70
>[10987.033020]  [<c044d773>] warn_slowpath_fmt+0x33/0x40
>[10987.033020]  [<c04ccc0c>] watchdog_overflow_callback+0xac/0xd0
>[10987.033020]  [<c04ff986>] __perf_event_overflow+0xa6/0x270
>[10987.033020]  [<c04ff986>] ? __perf_event_overflow+0xa6/0x270
>[10987.033020]  [<c042021f>] ? x86_perf_event_set_period+0x11f/0x230
>[10987.033020]  [<c05004f5>] perf_event_overflow+0x15/0x20
>[10987.033020]  [<c0422033>] p4_pmu_handle_irq+0x143/0x1f0
>[10987.033020]  [<c047191a>] ? hrtimer_forward+0x10a/0x250
>[10987.033020]  [<c098424b>] perf_event_nmi_handler+0x1b/0x20
>[10987.033020]  [<c09839c9>] nmi_handle.isra.0+0x39/0x60
>[10987.033020]  [<c07a6740>] ? ata_sff_wait_after_reset+0x110/0x110
>[10987.033020]  [<c0983ad8>] do_nmi+0xe8/0x3f0
>[10987.033020]  [<c0983a2d>] ? do_nmi+0x3d/0x3f0
>[10987.033020]  [<c07a6740>] ? ata_sff_wait_after_reset+0x110/0x110
>[10987.033020]  [<c0983107>] nmi_stack_correct+0x2f/0x34
>[10987.033020]  [<c07a6070>] ? ata_pci_sff_prepare_host+0xb0/0xb0
>[10987.033020]  [<c07a6740>] ? ata_sff_wait_after_reset+0x110/0x110
>[10987.033020]  [<c04500d8>] ? devkmsg_read+0x248/0x430
>[10987.033020]  [<c0693c22>] ? ioread8+0x32/0x40
>[10987.033020]  [<c07a5870>] ? ata_sff_check_status+0x10/0x20
>[10987.033020]  [<c07a60ae>] ata_sff_drain_fifo+0x3e/0x80
>[10987.033020]  [<c07a6a80>] ? ata_pio_sectors+0xa0/0xa0
>[10987.033020]  [<c07a6dc7>] ata_sff_error_handler+0x67/0xe0
>[10987.033020]  [<c07a00d8>] ? ata_eh_release+0x28/0x60
>[10987.033020]  [<c07a6e93>] ata_bmdma_error_handler+0x53/0xf0
>[10987.033020]  [<c07a4333>] ata_scsi_port_error_handler+0x433/0x790
>[10987.033020]  [<c07a8b76>] ? ata_sff_flush_pio_task+0x16/0x60
>[10987.033020]  [<c07a4705>] ata_scsi_error+0x75/0xa0
>[10987.033020]  [<c077c63f>] scsi_error_handler+0xdf/0x610
>[10987.033020]  [<c047e590>] ? default_wake_function+0x10/0x20
>[10987.033020]  [<c04761b5>] ? __wake_up_common+0x45/0x70
>[10987.033020]  [<c047931e>] ? complete+0x4e/0x60
>[10987.033020]  [<c077c560>] ? scsi_eh_get_sense+0x1b0/0x1b0
>[10987.033020]  [<c046d534>] kthread+0x94/0xa0
>[10987.033020]  [<c0989937>] ret_from_kernel_thread+0x1b/0x28
>[10987.033020]  [<c046d4a0>] ? kthread_create_on_node+0xc0/0xc0
>[10987.033020] ---[ end trace 8435b7bed5fce0ee ]---
>[11012.765621] BUG: soft lockup - CPU#3 stuck for 23s! [swapper/3:0]
>[11013.017459] BUG: soft lockup - CPU#1 stuck for 23s! [swapper/1:0]
>[11013.017459] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support
>[11012.765621] Modules linked in:
>[11013.017459]  lpc_ich
>[11012.765621]  nfsv3
>[11013.017459]  serio_raw
>[11012.765621]  nfs_acl
>[11013.017459]  i2c_core
>[11012.765621]  nfsv4
>[11013.017459]  mfd_core
>[11012.765621]  auth_rpcgss
>[11013.017459]  e7xxx_edac
>[11012.765621]  nfs
>[11013.017459]  edac_core
>[11012.765621]  dns_resolver
>[11013.017459]  microcode
>[11012.765621]  fscache
>[11013.017459]  binfmt_misc
>[11012.765621]  lockd
>[11013.017459]  uinput
>[11012.765621]  sunrpc
>[11013.017459]  sata_sil
>[11012.765621] 
>[11012.765621] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11012.765621] EIP: 0060:[<c098255f>] EFLAGS: 00000286 CPU: 3
>[11012.765621] EIP is at _raw_spin_unlock_irqrestore+0xf/0x20
>[11012.765621] EAX: 00000286 EBX: eec2ce40 ECX: 00000400 EDX: 00000286
>[11012.765621] ESI: f75e50c0 EDI: f75e5534 EBP: f7121f3c ESP: f7121f3c
>[11012.765621]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11012.765621] CR0: 8005003b CR2: 0a08989c CR3: 31718000 CR4: 000007f0
>[11012.765621] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11012.765621] DR6: ffff0ff0 DR7: 00000400
>[11012.765621] Process swapper/3 (pid: 0, ti=f7120000 task=f70ba5b0 task.ti=f70c2000)
>[11012.765621] Stack:
>[11013.017459]  ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[11013.017459] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11013.017459] EIP: 0060:[<c045cd45>] EFLAGS: 00000206 CPU: 1
>[11013.017459] EIP is at run_timer_softirq+0x125/0x210
>[11013.017459] EAX: 00a36856 EBX: f70d9fa4 ECX: 00a36857 EDX: f70d02b0
>[11013.017459] ESI: 00000001 EDI: f70d0000 EBP: f70d9fb8 ESP: f70d9f80
>[11012.765621]  f7121f68 c0483c73 0000000c c0cdc0c0 00000286 00000003 eec2ce40 eec2ce8c
>[11013.017459]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11013.017459] CR0: 8005003b CR2: 452a57a4 CR3: 36451000 CR4: 000007f0
>[11013.017459] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11013.017459] DR6: ffff0ff0 DR7: 00000400
>[11013.017459] Process swapper/1 (pid: 0, ti=f70d8000 task=f70b8c90 task.ti=f70b6000)
>[11013.017459] Stack:
>[11013.017459]  c0cdc0c0 00000003 00a45099 f7121f9c c04863bf 00000001
>[11012.765621]  00000001
>[11013.017459]  f70d4000
>[11012.765621]  00000001
>[11013.017459]  f75e50c0
>[11012.765621] 
>[11012.765621]  00000000 f70d0a14
>[11013.017459]  00000003
>[11012.765621]  f70d0814
>[11013.017459]  00000001
>[11012.765621]  00000001
>[11013.017459]  00000001
>[11012.765621]  00000001
>[11013.017459]  00000000
>[11012.765621]  f70d9fb8
>[11013.017459] 
>[11013.017459]  00000001 c0cdc0c0
>[11012.765621]  f70d9fa4
>[11013.017459]  00000003
>[11012.765621]  f70d9fa4
>[11013.017459]  00000001
>[11012.765621] 
>[11012.765621] Call Trace:
>[11012.765621]  f70d9fa4 00000082 00000001
>[11012.765621]  [<c0483c73>] update_shares+0xa3/0xd0
>[11012.765621]  c0bb8a04 f70d9ff8
>[11012.765621]  [<c04863bf>] rebalance_domains+0x3f/0x150
>[11013.017459]  c045596f
>[11012.765621]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[11012.765621]  00000000 00000000
>[11012.765621]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[11012.765621]  00000000 00000000
>[11012.765621]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[11012.765621]  00000000 00000000
>[11013.017459]  <IRQ> 
>[11012.765621]  00200040
>[11012.765621]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[11013.017459]  00000001
>[11013.017459] Call Trace:
>[11012.765621]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11013.017459]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[11012.765621]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[11013.017459]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[11012.765621]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11012.765621]  <IRQ> 
>[11012.765621]  [<c0419486>] ? default_idle+0x46/0x190
>[11013.017459]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[11012.765621]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[11013.017459]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11012.765621]  [<c09736e7>] ? start_secondary+0x250/0x255
>[11013.017459]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[11013.017459] Code: ce
>[11013.017459]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11013.017459]  38 d1
>[11013.017459]  [<c0419486>] ? default_idle+0x46/0x190
>[11013.017459]  74 10
>[11013.017459]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[11013.017459]  8d b4 26
>[11013.017459]  [<c09736e7>] ? start_secondary+0x250/0x255
>[11013.017459]  00 00 00
>[11012.765621] Code:
>[11013.017459]  00
>[11012.765621]  89
>[11013.017459]  f3
>[11012.765621]  f8
>[11013.017459]  90
>[11012.765621]  e8
>[11013.017459]  0f
>[11012.765621]  ff
>[11013.017459]  b6
>[11012.765621]  57
>[11013.017459]  10
>[11012.765621]  52
>[11013.017459]  38
>[11012.765621]  00
>[11013.017459]  ca
>[11012.765621]  8b
>[11013.017459]  75
>[11012.765621]  5d
>[11013.017459]  f7
>[11012.765621]  ec
>[11013.017459]  5d
>[11012.765621]  3b
>[11013.017459]  c3
>[11012.765621]  5d
>[11013.017459]  90
>[11012.765621]  e8
>[11013.017459]  8d
>[11012.765621]  75
>[11013.017459]  74
>[11012.765621]  8f
>[11013.017459]  26
>[11012.765621]  a1
>[11013.017459]  00
>[11012.765621]  40
>[11013.017459]  55
>[11012.765621]  8a
>[11013.017459]  89
>[11012.765621]  bb
>[11013.017459]  e5
>[11012.765621]  c0
>[11013.017459]  3e
>[11012.765621]  8b
>[11013.017459]  8d
>[11012.765621]  4f
>[11013.017459]  74
>[11012.765621]  08
>[11013.017459]  26
>[11012.765621]  39
>[11013.017459]  00
>[11012.765621]  c8
>[11013.017459]  80
>[11012.765621]  0f
>[11013.017459]  00
>[11012.765621]  89
>[11013.017459]  01
>[11012.765621]  3f
>[11013.017459]  89
>[11012.765621]  ff
>[11013.017459]  d0
>[11012.765621]  ff
>[11013.017459]  50
>[11012.765621]  ff
>[11013.017459]  9d
>[11012.765621]  c7
>[11013.017459]  <8d>
>[11012.765621]  47
>[11013.017459]  74
>[11012.765621]  04
>[11013.017459]  26
>[11012.765621]  00
>[11013.017459]  00
>[11012.765621]  00
>[11013.017459]  5d
>[11012.765621]  00
>[11013.017459]  c3
>[11012.765621]  00
>[11013.017459]  8d
>[11012.765621]  80
>[11013.017459]  74
>[11012.765621]  07
>[11013.017459]  26
>[11012.765621]  01
>[11013.017459]  00
>[11012.765621]  fb
>[11013.017459]  8d
>[11012.765621]  90
>[11013.017459]  bc
>[11012.765621]  <8d>
>[11013.017459]  27
>[11012.765621]  74
>[11013.017459]  00
>[11012.765621]  26
>[11013.017459]  00
>[11012.765621]  00
>[11013.017459]  00
>[11012.765621]  83
>[11013.017459]  00
>[11012.765621]  c4
>[11013.017459]  55
>[11012.765621]  2c
>[11013.017459]  89
>[11012.765621]  5b
>[11013.017459]  e5
>[11012.765621]  5e
>[11013.017459]  3e
>[11012.765621] 
>[11013.017459]  5f 5d c3 8d b4 26 00 00 00 00 8b 35
>[11040.289178] BUG: soft lockup - CPU#1 stuck for 22s! [swapper/1:0]
>[11040.316461] BUG: soft lockup - CPU#3 stuck for 22s! [swapper/3:0]
>[11040.316461] Modules linked in:
>[11040.289178] Modules linked in:
>[11040.316461]  nfsv3
>[11040.289178]  nfsv3
>[11040.316461]  nfs_acl
>[11040.289178]  nfs_acl
>[11040.316461]  nfsv4
>[11040.289178]  nfsv4
>[11040.316461]  auth_rpcgss
>[11040.289178]  auth_rpcgss
>[11040.316461]  nfs
>[11040.289178]  nfs
>[11040.316461]  dns_resolver
>[11040.289178]  dns_resolver
>[11040.316461]  fscache
>[11040.289178]  fscache
>[11040.316461]  lockd
>[11040.289178]  lockd
>[11040.316461]  sunrpc
>[11040.289178]  sunrpc
>[11040.316461]  ip6t_REJECT
>[11040.289178]  ip6t_REJECT
>[11040.316461]  ip6table_filter
>[11040.289178]  ip6table_filter
>[11040.316461]  ip6_tables
>[11040.289178]  ip6_tables
>[11040.316461]  iptable_raw
>[11040.289178]  iptable_raw
>[11040.316461]  e1000
>[11040.289178]  e1000
>[11040.316461]  i2c_i801
>[11040.289178]  i2c_i801
>[11040.316461]  iTCO_wdt
>[11040.289178]  iTCO_wdt
>[11040.316461]  iTCO_vendor_support
>[11040.289178]  iTCO_vendor_support
>[11040.316461]  lpc_ich
>[11040.289178]  lpc_ich
>[11040.316461]  serio_raw
>[11040.289178]  serio_raw
>[11040.316461]  i2c_core
>[11040.289178]  i2c_core
>[11040.316461]  mfd_core
>[11040.289178]  mfd_core
>[11040.316461]  e7xxx_edac
>[11040.289178]  e7xxx_edac
>[11040.316461]  edac_core
>[11040.289178]  edac_core
>[11040.316461]  microcode
>[11040.289178]  microcode
>[11040.316461]  binfmt_misc
>[11040.289178]  binfmt_misc
>[11040.316461]  uinput
>[11040.289178]  uinput
>[11040.316461]  sata_sil
>[11040.289178] 
>[11040.316461]  sata_sil
>[11040.289178] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11040.316461] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11040.289178] EIP: 0060:[<c068365c>] EFLAGS: 00000297 CPU: 1
>[11040.316461] EIP: 0060:[<c098255f>] EFLAGS: 00000286 CPU: 3
>[11040.289178] EIP is at cpumask_next_and+0x2c/0x40
>[11040.289178] EAX: 00000003 EBX: f7096c50 ECX: 00000000 EDX: ffffffff
>[11040.316461] EIP is at _raw_spin_unlock_irqrestore+0xf/0x20
>[11040.289178] ESI: f75c4930 EDI: f7096c50 EBP: f70d9dd8 ESP: f70d9dd0
>[11040.316461] EAX: 00000286 EBX: f16d0300 ECX: 00000400 EDX: 00000286
>[11040.289178]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11040.316461] ESI: f75e50c0 EDI: f75e5534 EBP: f7121f3c ESP: f7121f3c
>[11040.289178] CR0: 8005003b CR2: 452a57a4 CR3: 36451000 CR4: 000007f0
>[11040.316461]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11040.289178] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11040.316461] CR0: 8005003b CR2: 0a08989c CR3: 31718000 CR4: 000007f0
>[11040.289178] DR6: ffff0ff0 DR7: 00000400
>[11040.316461] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11040.289178] Process swapper/1 (pid: 0, ti=f70d8000 task=f70b8c90 task.ti=f70b6000)
>[11040.316461] DR6: ffff0ff0 DR7: 00000400
>[11040.289178] Stack:
>[11040.316461] Process swapper/3 (pid: 0, ti=f7120000 task=f70ba5b0 task.ti=f70c2000)
>[11040.316461] Stack:
>[11040.316461]  ffffffff f70d9f24
>[11040.289178]  f7121f68
>[11040.316461]  f70d9e50
>[11040.289178]  c0483c73
>[11040.316461]  c048554d
>[11040.289178]  0000000c
>[11040.316461]  f70d9f8c
>[11040.289178]  c0cdc0c0
>[11040.316461]  00000000
>[11040.289178]  00000286
>[11040.316461]  0001b5db
>[11040.289178]  00000003
>[11040.316461]  f70d9e20
>[11040.289178] 
>[11040.289178]  f16d0300 f16d034c
>[11040.316461] 
>[11040.316461]  ffffffff 00000000
>[11040.289178]  c0cdc0c0
>[11040.316461]  00000000
>[11040.289178]  00000003
>[11040.316461]  c0cdc0c0
>[11040.289178]  00a4cb8d
>[11040.316461]  f7096c40
>[11040.289178]  f7121f9c
>[11040.316461]  f70d9e80
>[11040.289178]  c04863bf
>[11040.316461]  f7096c50
>[11040.289178]  ffffff10
>[11040.316461]  00000000
>[11040.289178] 
>[11040.289178]  c045cd45
>[11040.316461]  ffffffff
>[11040.289178]  f75e50c0
>[11040.316461] 
>[11040.316461]  00000000 ffffffff
>[11040.289178]  00000003
>[11040.316461]  00000000
>[11040.289178]  00000001
>[11040.316461]  00000000
>[11040.289178]  00000001
>[11040.316461]  00000000
>[11040.289178]  00000000
>[11040.316461]  00000000
>[11040.289178]  00000001
>[11040.316461]  00000000
>[11040.289178] 
>[11040.289178] Call Trace:
>[11040.289178]  c0cdc0c0 00000003 00000001
>[11040.316461] 
>[11040.289178]  [<c048554d>] update_sd_lb_stats+0xcd/0x480
>[11040.316461] Call Trace:
>[11040.289178]  [<c048592b>] find_busiest_group+0x2b/0x420
>[11040.316461]  [<c0483c73>] update_shares+0xa3/0xd0
>[11040.289178]  [<c0485db8>] load_balance+0x98/0x660
>[11040.316461]  [<c04863bf>] rebalance_domains+0x3f/0x150
>[11040.289178]  [<c0486409>] rebalance_domains+0x89/0x150
>[11040.316461]  [<c045cd45>] ? run_timer_softirq+0x125/0x210
>[11040.289178]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[11040.316461]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[11040.289178]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[11040.316461]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[11040.289178]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[11040.316461]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[11040.289178]  <IRQ> 
>[11040.289178]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[11040.316461]  <IRQ> 
>[11040.316461]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[11040.289178]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11040.316461]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11040.289178]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[11040.316461]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[11040.289178]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11040.316461]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11040.289178]  [<c0419486>] ? default_idle+0x46/0x190
>[11040.316461]  [<c0419486>] ? default_idle+0x46/0x190
>[11040.289178]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[11040.316461]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[11040.289178]  [<c09736e7>] ? start_secondary+0x250/0x255
>[11040.316461]  [<c09736e7>] ? start_secondary+0x250/0x255
>[11040.316461] Code:
>[11040.289178] Code:
>[11040.316461]  89
>[11040.289178]  ce
>[11040.316461]  e5
>[11040.289178]  38
>[11040.316461]  56
>[11040.289178]  d1
>[11040.316461]  89
>[11040.289178]  74
>[11040.316461]  ce
>[11040.289178]  10
>[11040.316461]  53
>[11040.289178]  8d
>[11040.316461]  89
>[11040.289178]  b4
>[11040.316461]  d3
>[11040.289178]  26
>[11040.316461]  8d
>[11040.289178]  00
>[11040.316461]  b4
>[11040.289178]  00
>[11040.316461]  26
>[11040.289178]  00
>[11040.316461]  00
>[11040.289178]  00
>[11040.316461]  00
>[11040.289178]  f3
>[11040.316461]  00
>[11040.289178]  90
>[11040.316461]  00
>[11040.289178]  0f
>[11040.316461]  8d
>[11040.289178]  b6
>[11040.316461]  48
>[11040.289178]  10
>[11040.316461]  01
>[11040.289178]  38
>[11040.316461]  ba
>[11040.289178]  ca
>[11040.316461]  20
>[11040.289178]  75
>[11040.316461]  00
>[11040.289178]  f7
>[11040.316461]  00
>[11040.289178]  5d
>[11040.316461]  00
>[11040.289178]  c3
>[11040.316461]  89
>[11040.289178]  90
>[11040.316461]  d8
>[11040.289178]  8d
>[11040.316461]  e8
>[11040.289178]  74
>[11040.316461]  b1
>[11040.289178]  26
>[11040.316461]  f6
>[11040.289178]  00
>[11040.316461]  00
>[11040.289178]  55
>[11040.316461]  00
>[11040.289178]  89
>[11040.316461]  3b
>[11040.289178]  e5
>[11040.316461]  05
>[11040.289178]  3e
>[11040.316461]  a4
>[11040.289178]  8d
>[11040.316461]  2a
>[11040.289178]  74
>[11040.316461]  c3
>[11040.289178]  26
>[11040.316461]  c0
>[11040.289178]  00
>[11040.316461]  7d
>[11040.289178]  80
>[11040.316461]  09
>[11040.289178]  00
>[11040.316461]  0f
>[11040.289178]  01
>[11040.316461]  a3
>[11040.289178]  89
>[11040.316461]  06
>[11040.289178]  d0
>[11040.316461]  19
>[11040.289178]  50
>[11040.316461]  d2
>[11040.289178]  9d
>[11040.316461]  <85>
>[11040.289178]  <8d>
>[11040.316461]  d2
>[11040.289178]  74
>[11040.316461]  74
>[11040.289178]  26
>[11040.316461]  e0
>[11040.289178]  00
>[11040.316461]  5b
>[11040.289178]  5d
>[11040.316461]  5e
>[11040.289178]  c3
>[11040.316461]  5d
>[11040.289178]  8d
>[11040.316461]  c3
>[11040.289178]  74
>[11040.316461]  8d
>[11040.289178]  26
>[11040.316461]  b6
>[11040.289178]  00
>[11040.316461]  00
>[11040.289178]  8d
>[11040.316461]  00
>[11040.289178]  bc
>[11040.316461]  00
>[11040.289178]  27
>[11040.316461]  00
>[11040.289178]  00
>[11040.316461]  8d
>[11040.289178]  00
>[11040.316461]  bf
>[11040.289178]  00
>[11040.316461]  00
>[11040.289178]  00
>[11040.316461]  00
>[11040.289178]  55
>[11040.316461]  00
>[11040.289178]  89
>[11040.316461]  00
>[11040.289178]  e5
>[11040.316461]  55
>[11040.289178] 
>[11040.316461]  3e
>[11046.728697] INFO: rcu_sched detected stalls on CPUs/tasks: { 0} (detected by 1, t=60123 jiffies)
>[11046.728697] sending NMI to all CPUs:
>[10987.033020] NMI backtrace for cpu 2
>[10987.033020] Pid: 193, comm: scsi_eh_3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[10987.033020] EIP: 0060:[<c0693be3>] EFLAGS: 00000092 CPU: 2
>[10987.033020] EIP is at ioread16+0x33/0x40
>[10987.033020] EAX: f7e2ffff EBX: f6428000 ECX: c07a6070 EDX: f7e2a140
>[10987.033020] ESI: 000057ec EDI: c07a6740 EBP: f17c7e6c ESP: f17c7e50
>[10987.033020]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[10987.033020] CR0: 8005003b CR2: 452a57c0 CR3: 36451000 CR4: 000007f0
>[10987.033020] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[10987.033020] DR6: ffff0ff0 DR7: 00000400
>[10987.033020] Process scsi_eh_3 (pid: 193, ti=f17c6000 task=f17c9920 task.ti=f17c6000)
>[10987.033020] Stack:
>[10987.033020]  c07a60a3 f6428000 00000000 00000202 00000202 f6428000 c07a6a80 f17c7e90
>[10987.033020]  c07a6dc7 017c007b c07a00d8 f64280c4 00000202 f6428000 f64280c4 00000202
>[10987.033020]  f17c7ea4 c07a6e93 00000000 00000001 f64296d4 f17c7eec c07a4333 f17c7ec8
>[10987.033020] Call Trace:
>[10987.033020]  [<c07a60a3>] ? ata_sff_drain_fifo+0x33/0x80
>[10987.033020]  [<c07a6a80>] ? ata_pio_sectors+0xa0/0xa0
>[10987.033020]  [<c07a6dc7>] ata_sff_error_handler+0x67/0xe0
>[10987.033020]  [<c07a00d8>] ? ata_eh_release+0x28/0x60
>[10987.033020]  [<c07a6e93>] ata_bmdma_error_handler+0x53/0xf0
>[10987.033020]  [<c07a4333>] ata_scsi_port_error_handler+0x433/0x790
>[10987.033020]  [<c07a8b76>] ? ata_sff_flush_pio_task+0x16/0x60
>[10987.033020]  [<c07a4705>] ata_scsi_error+0x75/0xa0
>[10987.033020]  [<c077c63f>] scsi_error_handler+0xdf/0x610
>[10987.033020]  [<c047e590>] ? default_wake_function+0x10/0x20
>[10987.033020]  [<c04761b5>] ? __wake_up_common+0x45/0x70
>[10987.033020]  [<c047931e>] ? complete+0x4e/0x60
>[10987.033020]  [<c077c560>] ? scsi_eh_get_sense+0x1b0/0x1b0
>[10987.033020]  [<c046d534>] kthread+0x94/0xa0
>[10987.033020]  [<c0989937>] ret_from_kernel_thread+0x1b/0x28
>[10987.033020]  [<c046d4a0>] ? kthread_create_on_node+0xc0/0xc0
>[10987.033020] Code: 00 00 01 00 76 0a 0f b7 d0 66 ed 0f b7 c0 c3 90 55 ba 31 55 b2 c0 89 e5 e8 0b fe ff ff b8 ff ff 00 00 5d c3 8d 74 26 00 66 8b 00 <0f> b7 c0 c3 89 f6 8d bc 27 00 00 00 00 3d ff ff 03 00 77 29 3d
>[11046.728697] NMI backtrace for cpu 1
>[11046.728697] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11046.728697] EIP: 0060:[<c068d880>] EFLAGS: 00000006 CPU: 1
>[11046.728697] EIP is at __const_udelay+0x0/0x20
>[11046.728697] EAX: 00418958 EBX: 00002710 ECX: fffff000 EDX: fffff000
>[11046.728697] ESI: c0bd8a00 EDI: 00000001 EBP: f70d9df0 ESP: f70d9de4
>[11046.728697]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11046.728697] CR0: 8005003b CR2: 452a57a4 CR3: 36451000 CR4: 000007f0
>[11046.728697] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11046.728697] DR6: ffff0ff0 DR7: 00000400
>[11046.728697] Process swapper/1 (pid: 0, ti=f70d8000 task=f70b8c90 task.ti=f70b6000)
>[11046.728697] Stack:
>[11046.728697]  c0435a2a c0ae75ef f75c4ba0 f70d9e24 c04d5550 c0afef44 00000001 0000eadb
>[11046.728697]  00000000 00000083 00000001 00000001 c0bd8a00 f70b8c90 00000000 00000001
>[11046.728697]  f70d9e38 c045d6ab f75c4980 0e5cd73f 00000a0c f70d9e58 c04a1208 f70d9e58
>[11046.728697] Call Trace:
>[11046.728697]  [<c0435a2a>] ? arch_trigger_all_cpu_backtrace+0x5a/0x80
>[11046.728697]  [<c04d5550>] rcu_check_callbacks+0x4b0/0x540
>[11046.728697]  [<c045d6ab>] update_process_times+0x3b/0x70
>[11046.728697]  [<c04a1208>] tick_sched_timer+0x68/0xf0
>[11046.728697]  [<c0471370>] ? __remove_hrtimer+0x40/0xa0
>[11046.728697]  [<c047152d>] __run_hrtimer+0x6d/0x190
>[11046.728697]  [<c04a11a0>] ? tick_nohz_handler+0x100/0x100
>[11046.728697]  [<c04721bf>] hrtimer_interrupt+0xef/0x280
>[11046.728697]  [<c0480482>] ? sched_clock_local+0xb2/0x1a0
>[11046.728697]  [<c0434689>] smp_apic_timer_interrupt+0x59/0x90
>[11046.728697]  [<c0982de0>] apic_timer_interrupt+0x34/0x3c
>[11046.728697]  [<c098255f>] ? _raw_spin_unlock_irqrestore+0xf/0x20
>[11046.728697]  [<c0483c73>] update_shares+0xa3/0xd0
>[11046.728697]  [<c04863bf>] rebalance_domains+0x3f/0x150
>[11046.728697]  [<c045cd45>] ? run_timer_softirq+0x125/0x210
>[11046.728697]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[11046.728697]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[11046.728697]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[11046.728697]  <IRQ> 
>[11046.728697]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[11046.728697]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11046.728697]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[11046.728697]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11046.728697]  [<c0419486>] ? default_idle+0x46/0x190
>[11046.728697]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[11046.728697]  [<c09736e7>] ? start_secondary+0x250/0x255
>[11046.728697] Code: 00 00 00 00 8d bc 27 00 00 00 00 48 75 fd 48 5d c3 8d 76 00 8d bc 27 00 00 00 00 55 89 e5 3e 8d 74 26 00 ff 15 c0 fa bf c0 5d c3 <55> 89 e5 64 8b 15 dc bf cd c0 c1 e0 02 69 d2 fa 00 00 00 f7 e2
>[11046.942298] NMI backtrace for cpu 3
>[11046.942298] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11046.942298] EIP: 0060:[<c045cca4>] EFLAGS: 00000006 CPU: 3
>[11046.942298] EIP is at run_timer_softirq+0x84/0x210
>[11046.942298] EAX: f70d4534 EBX: f7121fa4 ECX: f7121fa4 EDX: f70d4520
>[11046.942298] ESI: 00000001 EDI: f70d4000 EBP: f7121fb8 ESP: f7121f80
>[11046.942298]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11046.942298] CR0: 8005003b CR2: 0a08989c CR3: 31718000 CR4: 000007f0
>[11046.942298] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11046.942298] DR6: ffff0ff0 DR7: 00000400
>[11046.942298] Process swapper/3 (pid: 0, ti=f7120000 task=f70ba5b0 task.ti=f70c2000)
>[11046.942298] Stack:
>[11046.942298]  00000001 00000001 00000000 f70d4a14 f70d4814 00000003 00000001 f7121fb8
>[11046.942298]  f7121fa4 f7121fa4 f7121fa4 00000082 00000001 c0bb8a04 f7121ff8 c045596f
>[11046.942298]  00000000 00000000 00000000 00000000 00000000 00000000 00200040 00000003
>[11046.942298] Call Trace:
>[11046.942298]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[11046.942298]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[11046.942298]  <IRQ> 
>[11046.942298]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[11046.942298]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11046.942298]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[11046.942298]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11046.942298]  [<c0419486>] ? default_idle+0x46/0x190
>[11046.942298]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[11046.942298]  [<c09736e7>] ? start_secondary+0x250/0x255
>[11046.942298] Code: cb 81 e3 ff 00 00 00 0f 84 1c 01 00 00 83 47 08 01 8d 14 df 8b 4a 14 8d 5d ec 8d 42 14 89 59 04 89 4d ec 8b 4a 18 89 4d f0 89 19 <89> 42 14 89 40 04 8d b6 00 00 00 00 8b 5d ec 3b 5d e8 74 71 8b
>[11051.582366] ------------[ cut here ]------------
>[11051.582366] WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0xac/0xd0()
>[11051.582366] Hardware name: Sun Fire(tm) V60 370-6037               
>[11051.582366] Watchdog detected hard LOCKUP on cpu 3
>[11051.582366] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[11051.582366] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1
>[11051.582366] Call Trace:
>[11051.582366]  [<c044d6a2>] warn_slowpath_common+0x72/0xa0
>[11051.582366]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[11051.582366]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[11051.582366]  [<c04ccb60>] ? touch_nmi_watchdog+0x70/0x70
>[11051.582366]  [<c044d773>] warn_slowpath_fmt+0x33/0x40
>[11051.582366]  [<c04ccc0c>] watchdog_overflow_callback+0xac/0xd0
>[11051.582366]  [<c04ff986>] __perf_event_overflow+0xa6/0x270
>[11051.582366]  [<c042021f>] ? x86_perf_event_set_period+0x11f/0x230
>[11051.582366]  [<c05004f5>] perf_event_overflow+0x15/0x20
>[11051.582366]  [<c0422033>] p4_pmu_handle_irq+0x143/0x1f0
>[11051.582366]  [<c04184d8>] ? sched_clock+0x8/0x10
>[11051.582366]  [<c098424b>] perf_event_nmi_handler+0x1b/0x20
>[11051.582366]  [<c09839c9>] nmi_handle.isra.0+0x39/0x60
>[11051.582366]  [<c0983bcf>] do_nmi+0x1df/0x3f0
>[11051.582366]  [<c0455bb7>] ? irq_exit+0x67/0xb0
>[11051.582366]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11051.582366]  [<c0983107>] nmi_stack_correct+0x2f/0x34
>[11051.582366]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11051.582366]  [<c0419486>] default_idle+0x46/0x190
>[11051.582366]  [<c041a086>] cpu_idle+0xb6/0xe0
>[11051.582366]  [<c09736e7>] start_secondary+0x250/0x255
>[11051.582366] ---[ end trace 8435b7bed5fce0ef ]---
>[11046.728697] ------------[ cut here ]------------
>[11046.728697] WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0xac/0xd0()
>[11046.728697] Hardware name: Sun Fire(tm) V60 370-6037               
>[11046.728697] Watchdog detected hard LOCKUP on cpu 1
>[11046.728697] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[11046.728697] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1
>[11046.728697] Call Trace:
>[11046.728697]  [<c044d6a2>] warn_slowpath_common+0x72/0xa0
>[11046.728697]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[11046.728697]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[11046.728697]  [<c04ccb60>] ? touch_nmi_watchdog+0x70/0x70
>[11046.728697]  [<c044d773>] warn_slowpath_fmt+0x33/0x40
>[11046.728697]  [<c04ccc0c>] watchdog_overflow_callback+0xac/0xd0
>[11046.728697]  [<c04ff986>] __perf_event_overflow+0xa6/0x270
>[11046.728697]  [<c042021f>] ? x86_perf_event_set_period+0x11f/0x230
>[11046.728697]  [<c05004f5>] perf_event_overflow+0x15/0x20
>[11046.728697]  [<c0422033>] p4_pmu_handle_irq+0x143/0x1f0
>[11046.728697]  [<c042207c>] ? p4_pmu_handle_irq+0x18c/0x1f0
>[11046.728697]  [<c098424b>] perf_event_nmi_handler+0x1b/0x20
>[11046.728697]  [<c09839c9>] nmi_handle.isra.0+0x39/0x60
>[11046.728697]  [<c0983ad8>] do_nmi+0xe8/0x3f0
>[11046.728697]  [<c0983107>] nmi_stack_correct+0x2f/0x34
>[11046.728697]  [<c068d944>] ? delay_tsc+0x44/0x60
>[11046.728697]  [<c068d89e>] __const_udelay+0x1e/0x20
>[11046.728697]  [<c0435a2a>] arch_trigger_all_cpu_backtrace+0x5a/0x80
>[11046.728697]  [<c04d5550>] rcu_check_callbacks+0x4b0/0x540
>[11046.728697]  [<c045d6ab>] update_process_times+0x3b/0x70
>[11046.728697]  [<c04a1208>] tick_sched_timer+0x68/0xf0
>[11046.728697]  [<c0471370>] ? __remove_hrtimer+0x40/0xa0
>[11046.728697]  [<c047152d>] __run_hrtimer+0x6d/0x190
>[11046.728697]  [<c04a11a0>] ? tick_nohz_handler+0x100/0x100
>[11046.728697]  [<c04721bf>] hrtimer_interrupt+0xef/0x280
>[11046.728697]  [<c0480482>] ? sched_clock_local+0xb2/0x1a0
>[11046.728697]  [<c0434689>] smp_apic_timer_interrupt+0x59/0x90
>[11046.728697]  [<c0982de0>] apic_timer_interrupt+0x34/0x3c
>[11046.728697]  [<c098255f>] ? _raw_spin_unlock_irqrestore+0xf/0x20
>[11046.728697]  [<c0483c73>] update_shares+0xa3/0xd0
>[11046.728697]  [<c04863bf>] rebalance_domains+0x3f/0x150
>[11046.728697]  [<c045cd45>] ? run_timer_softirq+0x125/0x210
>[11046.728697]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[11046.728697]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[11046.728697]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[11046.728697]  <IRQ>  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[11046.728697]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11046.728697]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[11046.728697]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11046.728697]  [<c0419486>] ? default_idle+0x46/0x190
>[11046.728697]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[11046.728697]  [<c09736e7>] ? start_secondary+0x250/0x255
>[11046.728697] ---[ end trace 8435b7bed5fce0f0 ]---
>[11076.238351] BUG: soft lockup - CPU#1 stuck for 24s! [swapper/1:0]
>[11076.440512] BUG: soft lockup - CPU#3 stuck for 22s! [swapper/3:0]
>[11076.440512] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt
>[11076.238351] Modules linked in:
>[11076.440512]  iTCO_vendor_support
>[11076.238351]  nfsv3
>[11076.440512]  lpc_ich
>[11076.238351]  nfs_acl
>[11076.440512]  serio_raw
>[11076.238351]  nfsv4
>[11076.440512]  i2c_core
>[11076.238351]  auth_rpcgss
>[11076.440512]  mfd_core
>[11076.238351]  nfs
>[11076.440512]  e7xxx_edac
>[11076.238351]  dns_resolver
>[11076.440512]  edac_core
>[11076.238351]  fscache
>[11076.440512]  microcode
>[11076.238351]  lockd
>[11076.440512]  binfmt_misc
>[11076.238351]  sunrpc
>[11076.440512]  uinput
>[11076.238351]  ip6t_REJECT
>[11076.440512]  sata_sil
>[11076.238351] 
>[11076.238351] Pid: 0, comm: swapper/1 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11076.238351] EIP: 0060:[<c0486094>] EFLAGS: 00000292 CPU: 1
>[11076.238351] EIP is at load_balance+0x374/0x660
>[11076.238351] EAX: 00000292 EBX: ef9a8cd8 ECX: 00000003 EDX: 00000001
>[11076.238351] ESI: f75e55ac EDI: 000000d1 EBP: f70d9f68 ESP: f70d9edc
>[11076.238351]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11076.238351] CR0: 8005003b CR2: 452a57a4 CR3: 36451000 CR4: 000007f0
>[11076.238351] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11076.238351] DR6: ffff0ff0 DR7: 00000400
>[11076.238351] Process swapper/1 (pid: 0, ti=f70d8000 task=f70b8c90 task.ti=f70b6000)
>[11076.238351] Stack:
>[11076.238351]  ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput
>[11076.440512]  00000003
>[11076.238351]  sata_sil
>[11076.440512] 
>[11076.440512] Pid: 0, comm: swapper/3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11076.440512] EIP: 0060:[<c0483c20>] EFLAGS: 00000212 CPU: 3
>[11076.440512] EIP is at update_shares+0x50/0xd0
>[11076.440512] EAX: f64c3200 EBX: eec28180 ECX: 0000000c EDX: 00000286
>[11076.440512] ESI: f75e50c0 EDI: f75e5534 EBP: f7121f68 ESP: f7121f44
>[11076.238351]  f70d9ef8 c043468e 00000000 00000001 f75c4930 00000001 c0cdc0c0
>[11076.440512]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11076.440512] CR0: 8005003b CR2: 0a08989c CR3: 31718000 CR4: 000007f0
>[11076.440512] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11076.440512] DR6: ffff0ff0 DR7: 00000400
>[11076.440512] Process swapper/3 (pid: 0, ti=f7120000 task=f70ba5b0 task.ti=f70c2000)
>[11076.440512] Stack:
>[11076.440512]  c0cdc0c0 f70ad100 00000001 00000003 00000292 00000003
>[11076.238351]  0000000c
>[11076.440512]  f75e50c0
>[11076.238351]  c0cdc0c0
>[11076.440512]  f75e50c0
>[11076.238351] 
>[11076.238351]  00000286 00000003
>[11076.440512]  ef9a8c90
>[11076.238351]  efc4d000
>[11076.440512]  f70d9f24
>[11076.238351]  eec281cc
>[11076.440512]  f70ad100
>[11076.238351]  c0cdc0c0
>[11076.440512]  f75e50c0
>[11076.238351]  00000003
>[11076.440512] 
>[11076.440512]  00000003 00000001
>[11076.238351]  00a558d1
>[11076.440512]  f75c90c0
>[11076.238351]  f7121f9c
>[11076.440512]  f7096bd0
>[11076.238351] 
>[11076.238351] Call Trace:
>[11076.238351]  c04863bf 00000001 f70d4000
>[11076.238351]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11076.238351]  f75e50c0 00000003 00000001
>[11076.440512] 
>[11076.238351]  [<c0486409>] rebalance_domains+0x89/0x150
>[11076.238351]  00000001
>[11076.238351]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[11076.238351]  00000000 00000001
>[11076.238351]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[11076.238351]  c0cdc0c0 00000003
>[11076.238351]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[11076.238351]  00000001 f7121fb8
>[11076.440512]  <IRQ> 
>[11076.238351]  c048650a
>[11076.440512] 
>[11076.238351]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[11076.440512] Call Trace:
>[11076.238351]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11076.440512]  [<c04863bf>] rebalance_domains+0x3f/0x150
>[11076.238351]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[11076.440512]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[11076.238351]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11076.440512]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[11076.238351]  [<c0419486>] ? default_idle+0x46/0x190
>[11076.440512]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[11076.238351]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[11076.238351]  <IRQ> 
>[11076.238351]  [<c09736e7>] ? start_secondary+0x250/0x255
>[11076.440512]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[11076.440512] Code:
>[11076.440512]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[11076.440512]  45 bc
>[11076.440512]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[11076.440512]  8b 55
>[11076.440512]  [<c043e474>] ? native_safe_halt+0x4/0x10
>[11076.440512]  d8 8b
>[11076.440512]  [<c0419486>] ? default_idle+0x46/0x190
>[11076.440512]  4d a8 01
>[11076.440512]  [<c041a086>] ? cpu_idle+0xb6/0xe0
>[11076.440512]  8c 90 84
>[11076.440512]  [<c09736e7>] ? start_secondary+0x250/0x255
>[11076.440512]  00 00
>[11076.238351] Code:
>[11076.440512]  00
>[11076.238351]  1f
>[11076.440512]  8b
>[11076.238351]  c3
>[11076.440512]  45
>[11076.238351]  c0
>[11076.440512]  a8
>[11076.238351]  c1
>[11076.440512]  01
>[11076.238351]  e6
>[11076.440512]  45
>[11076.238351]  02
>[11076.440512]  a0
>[11076.238351]  8b
>[11076.440512]  8b
>[11076.238351]  93
>[11076.440512]  45
>[11076.238351]  74
>[11076.440512]  cc
>[11076.238351]  04
>[11076.440512]  80
>[11076.238351]  00
>[11076.440512]  00
>[11076.238351]  00
>[11076.440512]  01
>[11076.238351]  8d
>[11076.440512]  3b
>[11076.238351]  bb
>[11076.440512]  45
>[11076.238351]  74
>[11076.440512]  ac
>[11076.238351]  04
>[11076.440512]  74
>[11076.238351]  00
>[11076.440512]  06
>[11076.238351]  00
>[11076.440512]  8b
>[11076.238351]  89
>[11076.440512]  45
>[11076.238351]  55
>[11076.440512]  ac
>[11076.238351]  f0
>[11076.440512]  80
>[11076.238351]  8b
>[11076.440512]  00
>[11076.238351]  55
>[11076.440512]  01
>[11076.238351]  f0
>[11076.440512]  8b
>[11076.238351]  39
>[11076.440512]  45
>[11076.238351]  d7
>[11076.440512]  a4
>[11076.238351]  8d
>[11076.440512]  50
>[11076.238351]  4a
>[11076.440512]  9d
>[11076.238351]  b4
>[11076.440512]  <8d>
>[11076.238351]  74
>[11076.440512]  74
>[11076.238351]  72
>[11076.440512]  26
>[11076.238351]  89
>[11076.440512]  00
>[11076.238351]  cb
>[11076.440512]  8b
>[11076.238351]  89
>[11076.440512]  55
>[11076.238351]  f1
>[11076.440512]  e4
>[11076.238351]  3e
>[11076.440512]  f6
>[11076.238351]  8d
>[11076.440512]  c2
>[11076.238351]  74
>[11076.440512]  02
>[11076.238351]  26
>[11076.440512]  74
>[11076.238351]  00
>[11076.440512]  60
>[11076.238351]  8b
>[11076.440512]  83
>[11076.238351]  43
>[11076.440512]  e2
>[11076.238351]  54
>[11076.440512]  fd
>[11076.238351]  <8b>
>[11076.440512]  89
>[11076.238351]  50
>[11076.440512]  55
>[11076.238351]  20
>[11076.440512]  e4
>[11076.238351]  8b
>[11076.440512]  e9
>[11076.238351]  14
>[11076.440512]  8d
>[11076.238351]  0a
>[11076.440512]  fe
>[11076.238351] 
>[11076.440512]  85 d2 74 4c 8b 40 24 8b 55 e8 8b 75 e0 8b 04
>[10987.033020] ata4: drained 65536 bytes to clear DRQ
>[11100.665630] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x6 frozen
>[11100.665633] ata4: SError: { Handshk UnrecFIS }
>[11100.665637] ata4.00: failed command: READ DMA EXT
>[11100.665646] ata4.00: cmd 25/00:00:00:f0:cf/00:01:17:00:00/e0 tag 0 dma 131072 in
>[11100.665646]          res ff/ff:ff:ff:ff:ff/ff:ff:ff:ff:ff/ff Emask 0x2 (HSM violation)
>[11100.665648] ata4.00: status: { Busy }
>[11100.665650] ata4.00: error: { ICRC UNC IDNF ABRT }
>[11100.665661] ata4: hard resetting link
>[11100.669424] ata3: lost interrupt (Status 0x50)
>[11100.669452] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x2400000 action 0x6 frozen
>[11100.669455] ata3: SError: { Handshk UnrecFIS }
>[11100.669460] ata3.00: failed command: READ DMA EXT
>[11100.669468] ata3.00: cmd 25/00:00:00:ba:71/00:01:26:00:00/e0 tag 0 dma 131072 in
>[11100.669468]          res 40/00:ef:11:a1:ba/00:00:1b:00:00/e0 Emask 0x4 (timeout)
>[11100.669470] ata3.00: status: { DRDY }
>[11100.669481] ata3: hard resetting link
>[11100.662685] Do you have a strange power saving mode enabled?
>[11100.662685] Dazed and confused, but trying to continue
>[11100.662685] NMI backtrace for cpu 0
>[11100.662685] Pid: 1652, comm: java Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11100.662685] EIP: 0060:[<c0982864>] EFLAGS: 00000082 CPU: 0
>[11100.662685] EIP is at _raw_spin_lock_irqsave+0x24/0x40
>[11100.662685] EAX: 0000e0e0 EBX: c0bc9c38 ECX: c0bc9c38 EDX: 00000001
>[11100.662685] ESI: 00000092 EDI: 00000001 EBP: f0411ba0 ESP: f0411b98
>[11100.662685]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11100.662685] CR0: 80050033 CR2: b67f8500 CR3: 36451000 CR4: 000007f0
>[11100.662685] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11100.662685] DR6: ffff0ff0 DR7: 00000400
>[11100.662685] Process java (pid: 1652, ti=f0410000 task=f17ce480 task.ti=f0410000)
>[11100.662685] Stack:
>[11100.662685]  c0bc9c38 00000000 f0411bc0 c0477320 00000001 f0411bbc 00000001 00000001
>[11100.662685]  00000000 00000000 f0411bd4 c0450b9b 00000000 00000000 f17ce480 f0411be8
>[11100.662685]  c045d6b0 f75b6980 1dc6713a 000009fe f0411c08 c04a1208 f0411c08 c0471370
>[11100.662685] Call Trace:
>[11100.662685]  [<c0477320>] __wake_up+0x20/0x60
>[11100.662685]  [<c0450b9b>] printk_tick+0x5b/0x80
>[11100.662685]  [<c045d6b0>] update_process_times+0x40/0x70
>[11100.662685]  [<c04a1208>] tick_sched_timer+0x68/0xf0
>[11100.662685]  [<c0471370>] ? __remove_hrtimer+0x40/0xa0
>[11100.662685]  [<c047152d>] __run_hrtimer+0x6d/0x190
>[11100.662685]  [<c04a11a0>] ? tick_nohz_handler+0x100/0x100
>[11100.662685]  [<c04721bf>] hrtimer_interrupt+0xef/0x280
>[11100.662685]  [<c0434689>] smp_apic_timer_interrupt+0x59/0x90
>[11100.662685]  [<c0982de0>] apic_timer_interrupt+0x34/0x3c
>[11100.662685]  [<c08c0ed0>] ? ip_queue_xmit+0x230/0x3b0
>[11100.662685]  [<c0549b5d>] ? __kmalloc_track_caller+0x2d/0x170
>[11100.662685]  [<c08d738b>] tcp_transmit_skb+0x3cb/0x850
>[11100.662685]  [<c08d9b50>] tcp_send_ack+0xd0/0x120
>[11100.662685]  [<c08ca989>] tcp_cleanup_rbuf+0x79/0x120
>[11100.662685]  [<c08cb87c>] tcp_recvmsg+0x38c/0xc50
>[11100.662685]  [<c08eeb49>] inet_recvmsg+0x79/0x90
>[11100.662685]  [<c087acc4>] sock_aio_read+0x114/0x150
>[11100.662685]  [<c0558f07>] do_sync_read+0x97/0xd0
>[11100.662685]  [<c05597fd>] vfs_read+0x13d/0x160
>[11100.662685]  [<c04a53a0>] ? sys_futex+0x80/0x130
>[11100.662685]  [<c0559867>] sys_read+0x47/0x80
>[11100.662685]  [<c09899cd>] sysenter_do_call+0x12/0x28
>[11100.662685]  [<c0980000>] ? check_tnode.part.14+0x3/0x14
>[11100.662685] Code: 89 c8 5b 5d c3 66 90 55 89 e5 56 53 3e 8d 74 26 00 89 c1 9c 58 8d 74 26 00 89 c6 fa 90 8d 74 26 00 b8 00 01 00 00 f0 66 0f c1 01 <0f> b6 dc 38 c3 74 0e 90 8d 74 26 00 f3 90 0f b6 11 38 da 75 f7
>[11101.947050] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
>[11101.974348] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11101.977592] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11101.977598] ata4.00: configured for UDMA/33
>[11101.977623] sd 3:0:0:0: [sdb]  
>[11101.977627] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11101.977630] sd 3:0:0:0: [sdb]  
>[11101.977633] Sense Key : Aborted Command [current] [descriptor]
>[11101.977638] Descriptor sense data with sense descriptors (in hex):
>[11101.977640]         72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 ff ff 
>[11101.977654]         ff ff ff ff 
>[11101.977661] sd 3:0:0:0: [sdb]  
>[11101.977667] Add. Sense: Scsi parity error
>[11101.977670] sd 3:0:0:0: [sdb] CDB: 
>[11101.977673] Read(10): 28 00 17 cf f0 00 00 01 00 00
>[11101.977685] end_request: I/O error, dev sdb, sector 399503360
>[11101.993290] ata4: EH complete
>[11102.138052] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
>[11102.156384] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11102.159561] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11102.159567] ata3.00: configured for UDMA/33
>[11102.159593] sd 2:0:0:0: [sda]  
>[11102.159597] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11102.159600] sd 2:0:0:0: [sda]  
>[11102.159602] Sense Key : Aborted Command [current] [descriptor]
>[11102.159607] Descriptor sense data with sense descriptors (in hex):
>[11102.159610]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11102.159624]         1b ba a1 11 
>[11102.159631] sd 2:0:0:0: [sda]  
>[11102.159634] Add. Sense: No additional sense information
>[11102.159637] sd 2:0:0:0: [sda] CDB: 
>[11102.159639] Read(10): 28 00 26 71 ba 00 00 01 00 00
>[11102.159652] end_request: I/O error, dev sda, sector 644987392
>[11102.175488] ata3: EH complete
>[11112.855489] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11112.870616] ata3.00: BMDMA2 stat 0x686d0009
>[11112.885716] ata3.00: failed command: READ DMA
>[11112.900739] ata3.00: cmd c8/00:00:00:7e:30/00:00:00:00:00/e5 tag 0 dma 131072 in
>[11112.900739]          res 51/04:ef:11:7e:30/00:00:1b:00:00/e5 Emask 0x1 (device error)
>[11112.959851] ata3.00: status: { DRDY ERR }
>[11112.974792] ata3.00: error: { ABRT }
>[11112.993007] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11112.997014] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11112.997021] ata3.00: configured for UDMA/33
>[11112.997046] sd 2:0:0:0: [sda]  
>[11112.997050] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11112.997053] sd 2:0:0:0: [sda]  
>[11112.997056] Sense Key : Aborted Command [current] [descriptor]
>[11112.997061] Descriptor sense data with sense descriptors (in hex):
>[11112.997063]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11112.997077]         05 30 7e 11 
>[11112.997084] sd 2:0:0:0: [sda]  
>[11112.997087] Add. Sense: No additional sense information
>[11112.997091] sd 2:0:0:0: [sda] CDB: 
>[11112.997093] Read(10): 28 00 05 30 7e 00 00 01 00 00
>[11112.997106] end_request: I/O error, dev sda, sector 87064064
>[11113.012173] ata3: EH complete
>[11141.886630] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11141.901771] ata3.00: BMDMA2 stat 0x686d0009
>[11141.916782] ata3.00: failed command: READ DMA EXT
>[11141.931637] ata3.00: cmd 25/00:00:00:6f:2e/00:01:15:00:00/e0 tag 0 dma 131072 in
>[11141.931637]          res 51/04:9f:61:6f:2e/00:00:15:00:00/e0 Emask 0x1 (device error)
>[11141.990629] ata3.00: status: { DRDY ERR }
>[11142.005499] ata3.00: error: { ABRT }
>[11142.024727] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11142.027687] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11142.027703] ata3.00: configured for UDMA/33
>[11142.027742] sd 2:0:0:0: [sda]  
>[11142.027747] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11142.027752] sd 2:0:0:0: [sda]  
>[11142.027756] Sense Key : Aborted Command [current] [descriptor]
>[11142.027764] Descriptor sense data with sense descriptors (in hex):
>[11142.027768]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11142.027792]         15 2e 6f 61 
>[11142.027803] sd 2:0:0:0: [sda]  
>[11142.027808] Add. Sense: No additional sense information
>[11142.027814] sd 2:0:0:0: [sda] CDB: 
>[11142.027817] Read(10): 28 00 15 2e 6f 00 00 01 00 00
>[11142.027839] end_request: I/O error, dev sda, sector 355364608
>[11142.042962] ata3: EH complete
>[11350.433124] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11350.447984] ata4.00: BMDMA2 stat 0x6d0009
>[11350.462445] ata4.00: failed command: READ DMA EXT
>[11350.476977] ata4.00: cmd 25/00:00:00:07:63/00:01:38:00:00/e0 tag 0 dma 131072 in
>[11350.476977]          res 51/04:cf:31:07:63/00:00:38:00:00/e0 Emask 0x1 (device error)
>[11350.536959] ata4.00: status: { DRDY ERR }
>[11350.551908] ata4.00: error: { ABRT }
>[11350.590272] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11350.593588] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11350.593594] ata4.00: configured for UDMA/33
>[11350.593615] sd 3:0:0:0: [sdb]  
>[11350.593618] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11350.593622] sd 3:0:0:0: [sdb]  
>[11350.593624] Sense Key : Aborted Command [current] [descriptor]
>[11350.593629] Descriptor sense data with sense descriptors (in hex):
>[11350.593632]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11350.593646]         38 63 07 31 
>[11350.593653] sd 3:0:0:0: [sdb]  
>[11350.593657] Add. Sense: No additional sense information
>[11350.593660] sd 3:0:0:0: [sdb] CDB: 
>[11350.593662] Read(10): 28 00 38 63 07 00 00 01 00 00
>[11350.593675] end_request: I/O error, dev sdb, sector 946013952
>[11350.608840] ata4: EH complete
>[11470.049031] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[11470.049031] Do you have a strange power saving mode enabled?
>[11470.049031] Dazed and confused, but trying to continue
>[11470.098523] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
>[11470.114045] ata3.00: failed command: READ DMA EXT
>[11470.129051] ata3.00: cmd 25/00:00:00:69:92/00:01:36:00:00/e0 tag 0 dma 131072 in
>[11470.129051]          res 40/00:9f:61:6f:2e/00:00:15:00:00/e0 Emask 0x4 (timeout)
>[11470.190143] ata3.00: status: { DRDY }
>[11470.205651] ata3: hard resetting link
>[11470.510048] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
>[11470.527753] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11470.530554] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11470.530560] ata3.00: configured for UDMA/33
>[11470.530583] sd 2:0:0:0: [sda]  
>[11470.530587] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11470.530590] sd 2:0:0:0: [sda]  
>[11470.530593] Sense Key : Aborted Command [current] [descriptor]
>[11470.530598] Descriptor sense data with sense descriptors (in hex):
>[11470.530600]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11470.530614]         15 2e 6f 61 
>[11470.530621] sd 2:0:0:0: [sda]  
>[11470.530624] Add. Sense: No additional sense information
>[11470.530628] sd 2:0:0:0: [sda] CDB: 
>[11470.530630] Read(10): 28 00 36 92 69 00 00 01 00 00
>[11470.530642] end_request: I/O error, dev sda, sector 915564800
>[11470.546347] ata3: EH complete
>[11476.209349] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11476.226417] ata4.00: BMDMA2 stat 0x6d0009
>[11476.243152] ata4.00: failed command: READ DMA EXT
>[11476.259510] ata4.00: cmd 25/00:00:00:20:0c/00:01:14:00:00/e0 tag 0 dma 131072 in
>[11476.259510]          res 51/04:af:51:20:0c/00:00:14:00:00/e0 Emask 0x1 (device error)
>[11476.329267] ata4.00: status: { DRDY ERR }
>[11476.346598] ata4.00: error: { ABRT }
>[11476.366804] ata4.00: n_sectors mismatch 1465149168 != 458752
>[11476.366813] ata4.00: old n_sectors matches native, probably late HPA lock, will try to unlock HPA
>[11476.366819] ata4.00: revalidation failed (errno=-5)
>[11476.382590] ata4: hard resetting link
>[11476.689097] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
>[11476.707373] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11476.710804] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11476.710812] ata4.00: configured for UDMA/33
>[11476.710865] sd 3:0:0:0: [sdb]  
>[11476.710870] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11476.710874] sd 3:0:0:0: [sdb]  
>[11476.710877] Sense Key : Aborted Command [current] [descriptor]
>[11476.710883] Descriptor sense data with sense descriptors (in hex):
>[11476.710886]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11476.710901]         14 0c 20 51 
>[11476.710907] sd 3:0:0:0: [sdb]  
>[11476.710912] Add. Sense: No additional sense information
>[11476.710916] sd 3:0:0:0: [sdb] CDB: 
>[11476.710918] Read(10): 28 00 14 0c 20 00 00 01 00 00
>[11476.710932] end_request: I/O error, dev sdb, sector 336338944
>[11476.731442] ata4: EH complete
>[11502.130401] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11502.146811] ata4.00: BMDMA2 stat 0x6d0009
>[11502.162252] ata4.00: failed command: READ DMA EXT
>[11502.177276] ata4.00: cmd 25/00:00:00:de:44/00:01:57:00:00/e0 tag 0 dma 131072 in
>[11502.177276]          res 51/04:ef:11:de:44/00:00:57:00:00/e0 Emask 0x1 (device error)
>[11502.238939] ata4.00: status: { DRDY ERR }
>[11502.254747] ata4.00: error: { ABRT }
>[11502.299109] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11502.306162] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11502.306168] ata4.00: configured for UDMA/33
>[11502.306192] sd 3:0:0:0: [sdb]  
>[11502.306196] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11502.306199] sd 3:0:0:0: [sdb]  
>[11502.306201] Sense Key : Aborted Command [current] [descriptor]
>[11502.306207] Descriptor sense data with sense descriptors (in hex):
>[11502.306209]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11502.306223]         57 44 de 11 
>[11502.306229] sd 3:0:0:0: [sdb]  
>[11502.306233] Add. Sense: No additional sense information
>[11502.306236] sd 3:0:0:0: [sdb] CDB: 
>[11502.306238] Read(10): 28 00 57 44 de 00 00 01 00 00
>[11502.306252] end_request: I/O error, dev sdb, sector 1464131072
>[11502.321873] ata4: EH complete
>[11507.987881] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11508.003867] ata4.00: BMDMA2 stat 0x6d0009
>[11508.019340] ata4.00: failed command: READ DMA EXT
>[11508.034640] ata4.00: cmd 25/00:00:00:db:e9/00:01:20:00:00/e0 tag 0 dma 131072 in
>[11508.034640]          res 51/04:3f:c1:db:e9/00:00:20:00:00/e0 Emask 0x1 (device error)
>[11508.095236] ata4.00: status: { DRDY ERR }
>[11508.110807] ata4.00: error: { ABRT }
>[11508.141099] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11508.144629] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11508.144639] ata4.00: configured for UDMA/33
>[11508.144667] sd 3:0:0:0: [sdb]  
>[11508.144670] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11508.144674] sd 3:0:0:0: [sdb]  
>[11508.144676] Sense Key : Aborted Command [current] [descriptor]
>[11508.144682] Descriptor sense data with sense descriptors (in hex):
>[11508.144684]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11508.144698]         20 e9 db c1 
>[11508.144705] sd 3:0:0:0: [sdb]  
>[11508.144708] Add. Sense: No additional sense information
>[11508.144712] sd 3:0:0:0: [sdb] CDB: 
>[11508.144714] Read(10): 28 00 20 e9 db 00 00 01 00 00
>[11508.144727] end_request: I/O error, dev sdb, sector 552196864
>[11508.160209] ata4: EH complete
>[11511.614506] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11511.630211] ata4.00: BMDMA2 stat 0x6d0009
>[11511.645593] ata4.00: failed command: READ DMA EXT
>[11511.660601] ata4.00: cmd 25/00:00:00:d9:bd/00:01:35:00:00/e0 tag 0 dma 131072 in
>[11511.660601]          res 51/04:ef:11:d9:bd/00:00:35:00:00/e0 Emask 0x1 (device error)
>[11511.719987] ata4.00: status: { DRDY ERR }
>[11511.735780] ata4.00: error: { ABRT }
>[11511.775763] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11511.779598] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11511.779606] ata4.00: configured for UDMA/33
>[11511.779630] sd 3:0:0:0: [sdb]  
>[11511.779634] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11511.779637] sd 3:0:0:0: [sdb]  
>[11511.779640] Sense Key : Aborted Command [current] [descriptor]
>[11511.779645] Descriptor sense data with sense descriptors (in hex):
>[11511.779647]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11511.779661]         35 bd d9 11 
>[11511.779668] sd 3:0:0:0: [sdb]  
>[11511.779671] Add. Sense: No additional sense information
>[11511.779674] sd 3:0:0:0: [sdb] CDB: 
>[11511.779676] Read(10): 28 00 35 bd d9 00 00 01 00 00
>[11511.779689] end_request: I/O error, dev sdb, sector 901634304
>[11511.795096] ata4: EH complete
>[11556.028997] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[11556.068464] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
>[11556.068470] ata3.00: failed command: READ DMA EXT
>[11556.068476] ata3.00: cmd 25/00:00:00:81:83/00:01:19:00:00/e0 tag 0 dma 131072 in
>[11556.068476]          res 40/00:9f:61:6f:2e/00:00:15:00:00/e0 Emask 0x4 (timeout)
>[11556.068478] ata3.00: status: { DRDY }
>[11556.068487] ata3: hard resetting link
>[11556.028997] Do you have a strange power saving mode enabled?
>[11556.028997] Dazed and confused, but trying to continue
>[11556.373050] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
>[11556.390667] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11556.393570] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11556.393576] ata3.00: configured for UDMA/33
>[11556.393600] sd 2:0:0:0: [sda]  
>[11556.393604] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11556.393607] sd 2:0:0:0: [sda]  
>[11556.393609] Sense Key : Aborted Command [current] [descriptor]
>[11556.393614] Descriptor sense data with sense descriptors (in hex):
>[11556.393617]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11556.393631]         15 2e 6f 61 
>[11556.393637] sd 2:0:0:0: [sda]  
>[11556.393641] Add. Sense: No additional sense information
>[11556.393644] sd 2:0:0:0: [sda] CDB: 
>[11556.393646] Read(10): 28 00 19 83 81 00 00 01 00 00
>[11556.393659] end_request: I/O error, dev sda, sector 428048640
>[11556.409384] ata3: EH complete
>[11584.611812] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11584.628320] ata4.00: BMDMA2 stat 0x6d0009
>[11584.644210] ata4.00: failed command: READ DMA EXT
>[11584.659581] ata4.00: cmd 25/00:00:00:fc:5c/00:01:4d:00:00/e0 tag 0 dma 131072 in
>[11584.659581]          res 51/04:7f:81:fc:5c/00:00:4d:00:00/e0 Emask 0x1 (device error)
>[11584.721075] ata4.00: status: { DRDY ERR }
>[11584.736868] ata4.00: error: { ABRT }
>[11584.775589] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11584.778590] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11584.778597] ata4.00: configured for UDMA/33
>[11584.778622] sd 3:0:0:0: [sdb]  
>[11584.778625] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11584.778629] sd 3:0:0:0: [sdb]  
>[11584.778631] Sense Key : Aborted Command [current] [descriptor]
>[11584.778636] Descriptor sense data with sense descriptors (in hex):
>[11584.778639]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11584.778653]         4d 5c fc 81 
>[11584.778659] sd 3:0:0:0: [sdb]  
>[11584.778663] Add. Sense: No additional sense information
>[11584.778666] sd 3:0:0:0: [sdb] CDB: 
>[11584.778668] Read(10): 28 00 4d 5c fc 00 00 01 00 00
>[11584.778681] end_request: I/O error, dev sdb, sector 1297939456
>[11584.794507] ata4: EH complete
>[11584.954147] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11584.970414] ata4.00: BMDMA2 stat 0x6d0009
>[11584.986396] ata4.00: failed command: READ DMA EXT
>[11585.001915] ata4.00: cmd 25/00:00:00:8e:5d/00:01:4d:00:00/e0 tag 0 dma 131072 in
>[11585.001915]          res 51/04:ef:11:8e:5d/00:00:4d:00:00/e0 Emask 0x1 (device error)
>[11585.064802] ata4.00: status: { DRDY ERR }
>[11585.081417] ata4.00: error: { ABRT }
>[11585.124082] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11585.127577] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11585.127584] ata4.00: configured for UDMA/33
>[11585.127610] sd 3:0:0:0: [sdb]  
>[11585.127614] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11585.127617] sd 3:0:0:0: [sdb]  
>[11585.127620] Sense Key : Aborted Command [current] [descriptor]
>[11585.127625] Descriptor sense data with sense descriptors (in hex):
>[11585.127627]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11585.127641]         4d 5d 8e 11 
>[11585.127648] sd 3:0:0:0: [sdb]  
>[11585.127651] Add. Sense: No additional sense information
>[11585.127655] sd 3:0:0:0: [sdb] CDB: 
>[11585.127657] Read(10): 28 00 4d 5d 8e 00 00 01 00 00
>[11585.127670] end_request: I/O error, dev sdb, sector 1297976832
>[11585.143303] ata4: EH complete
>[11593.170242] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11593.186242] ata3.00: BMDMA2 stat 0x686c0009
>[11593.202239] ata3.00: failed command: READ DMA EXT
>[11593.217906] ata3.00: cmd 25/00:00:38:13:89/00:01:1f:00:00/e0 tag 0 dma 131072 in
>[11593.217906]          res 51/04:00:37:14:89/00:00:1f:00:00/e0 Emask 0x1 (device error)
>[11593.279209] ata3.00: status: { DRDY ERR }
>[11593.295053] ata3.00: error: { ABRT }
>[11593.313880] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11593.317644] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11593.317655] ata3.00: configured for UDMA/33
>[11593.317694] sd 2:0:0:0: [sda]  
>[11593.317698] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11593.317701] sd 2:0:0:0: [sda]  
>[11593.317703] Sense Key : Aborted Command [current] [descriptor]
>[11593.317708] Descriptor sense data with sense descriptors (in hex):
>[11593.317711]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11593.317725]         1f 89 14 37 
>[11593.317731] sd 2:0:0:0: [sda]  
>[11593.317735] Add. Sense: No additional sense information
>[11593.317738] sd 2:0:0:0: [sda] CDB: 
>[11593.317740] Read(10): 28 00 1f 89 13 38 00 01 00 00
>[11593.317753] end_request: I/O error, dev sda, sector 529077048
>[11593.333997] ata3: EH complete
>[11616.662071] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11616.677624] ata3.00: BMDMA2 stat 0x686d0009
>[11616.693209] ata3.00: failed command: READ DMA EXT
>[11616.708604] ata3.00: cmd 25/00:00:00:4a:0c/00:01:26:00:00/e0 tag 0 dma 131072 in
>[11616.708604]          res 51/04:df:21:4a:0c/00:00:26:00:00/e0 Emask 0x1 (device error)
>[11616.769339] ata3.00: status: { DRDY ERR }
>[11616.785265] ata3.00: error: { ABRT }
>[11616.804637] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11616.807671] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11616.807677] ata3.00: configured for UDMA/33
>[11616.807701] sd 2:0:0:0: [sda]  
>[11616.807705] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11616.807708] sd 2:0:0:0: [sda]  
>[11616.807710] Sense Key : Aborted Command [current] [descriptor]
>[11616.807715] Descriptor sense data with sense descriptors (in hex):
>[11616.807717]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11616.807732]         26 0c 4a 21 
>[11616.807738] sd 2:0:0:0: [sda]  
>[11616.807741] Add. Sense: No additional sense information
>[11616.807744] sd 2:0:0:0: [sda] CDB: 
>[11616.807746] Read(10): 28 00 26 0c 4a 00 00 01 00 00
>[11616.807759] end_request: I/O error, dev sda, sector 638339584
>[11616.823061] ata3: EH complete
>[11650.106481] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11650.123164] ata3.00: BMDMA2 stat 0x696d0009
>[11650.139666] ata3.00: failed command: READ DMA
>[11650.155610] ata3.00: cmd c8/00:00:00:4d:12/00:00:00:00:00/ed tag 0 dma 131072 in
>[11650.155610]          res 51/04:8f:71:4d:12/00:00:26:00:00/ed Emask 0x1 (device error)
>[11650.217288] ata3.00: status: { DRDY ERR }
>[11650.233369] ata3.00: error: { ABRT }
>[11650.253193] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11650.257164] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11650.257170] ata3.00: configured for UDMA/33
>[11650.257194] sd 2:0:0:0: [sda]  
>[11650.257198] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11650.257201] sd 2:0:0:0: [sda]  
>[11650.257203] Sense Key : Aborted Command [current] [descriptor]
>[11650.257208] Descriptor sense data with sense descriptors (in hex):
>[11650.257211]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11650.257225]         0d 12 4d 71 
>[11650.257231] sd 2:0:0:0: [sda]  
>[11650.257235] Add. Sense: No additional sense information
>[11650.257238] sd 2:0:0:0: [sda] CDB: 
>[11650.257240] Read(10): 28 00 0d 12 4d 00 00 01 00 00
>[11650.257253] end_request: I/O error, dev sda, sector 219303168
>[11650.272596] ata3: EH complete
>[11775.502494] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11775.518146] ata3.00: BMDMA2 stat 0x686d0009
>[11775.533979] ata3.00: failed command: READ DMA EXT
>[11775.549425] ata3.00: cmd 25/00:00:00:5b:bc/00:01:16:00:00/e0 tag 0 dma 131072 in
>[11775.549425]          res 51/04:ef:11:5b:bc/00:00:16:00:00/e0 Emask 0x1 (device error)
>[11775.610070] ata3.00: status: { DRDY ERR }
>[11775.626406] ata3.00: error: { ABRT }
>[11775.647115] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11775.650572] ata3.00: failed to get Identify Device Data, Emask 0x1
>[11775.650582] ata3.00: configured for UDMA/33
>[11775.650611] sd 2:0:0:0: [sda]  
>[11775.650615] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11775.650618] sd 2:0:0:0: [sda]  
>[11775.650621] Sense Key : Aborted Command [current] [descriptor]
>[11775.650626] Descriptor sense data with sense descriptors (in hex):
>[11775.650628]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11775.650643]         16 bc 5b 11 
>[11775.650649] sd 2:0:0:0: [sda]  
>[11775.650652] Add. Sense: No additional sense information
>[11775.650655] sd 2:0:0:0: [sda] CDB: 
>[11775.650658] Read(10): 28 00 16 bc 5b 00 00 01 00 00
>[11775.650670] end_request: I/O error, dev sda, sector 381442816
>[11775.665949] ata3: EH complete
>[11802.030274] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11802.050963] ata4.00: BMDMA2 stat 0x6d0009
>[11802.068397] ata4.00: failed command: READ DMA EXT
>[11802.085102] ata4.00: cmd 25/00:00:00:7d:b2/00:01:53:00:00/e0 tag 0 dma 131072 in
>[11802.085102]          res 51/04:ef:11:7d:b2/00:00:53:00:00/e0 Emask 0x1 (device error)
>[11802.151089] ata4.00: status: { DRDY ERR }
>[11802.169207] ata4.00: error: { ABRT }
>[11802.215156] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11802.218843] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11802.218859] ata4.00: configured for UDMA/33
>[11802.218902] sd 3:0:0:0: [sdb]  
>[11802.218970] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11802.218987] sd 3:0:0:0: [sdb]  
>[11802.218992] Sense Key : Aborted Command [current] [descriptor]
>[11802.219034] Descriptor sense data with sense descriptors (in hex):
>[11802.219039]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11802.219059]         53 b2 7d 11 
>[11802.219070] sd 3:0:0:0: [sdb]  
>[11802.219077] Add. Sense: No additional sense information
>[11802.219083] sd 3:0:0:0: [sdb] CDB: 
>[11802.219087] Read(10): 28 00 53 b2 7d 00 00 01 00 00
>[11802.219105] end_request: I/O error, dev sdb, sector 1404206336
>[11802.235069] ata4: EH complete
>[11840.467230] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
>[11840.482629] ata4.00: BMDMA2 stat 0x6d0009
>[11840.497959] ata4.00: failed command: READ DMA EXT
>[11840.513059] ata4.00: cmd 25/00:00:00:d4:98/00:01:3f:00:00/e0 tag 0 dma 131072 in
>[11840.513059]          res 51/04:1f:e1:d4:98/00:00:3f:00:00/e0 Emask 0x1 (device error)
>[11840.573158] ata4.00: status: { DRDY ERR }
>[11840.588775] ata4.00: error: { ABRT }
>[11840.627691] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11840.630591] ata4.00: failed to get Identify Device Data, Emask 0x1
>[11840.630597] ata4.00: configured for UDMA/33
>[11840.630625] sd 3:0:0:0: [sdb]  
>[11840.630629] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[11840.630632] sd 3:0:0:0: [sdb]  
>[11840.630634] Sense Key : Aborted Command [current] [descriptor]
>[11840.630639] Descriptor sense data with sense descriptors (in hex):
>[11840.630641]         72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00 
>[11840.630656]         3f 98 d4 e1 
>[11840.630662] sd 3:0:0:0: [sdb]  
>[11840.630665] Add. Sense: No additional sense information
>[11840.630669] sd 3:0:0:0: [sdb] CDB: 
>[11840.630671] Read(10): 28 00 3f 98 d4 00 00 01 00 00
>[11840.630683] end_request: I/O error, dev sdb, sector 1066980352
>[11840.646532] ata4: EH complete
>[11958.265644] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[11958.265644] Do you have a strange power saving mode enabled?
>[11958.265644] Dazed and confused, but trying to continue
>[11958.265644] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[11958.265644] Do you have a strange power saving mode enabled?
>[11958.265644] Dazed and confused, but trying to continue
>[11958.265644] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[11958.265644] Do you have a strange power saving mode enabled?
>[11958.265644] Dazed and confused, but trying to continue
>[11958.265644] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[11958.265644] Do you have a strange power saving mode enabled?
>[11958.265644] Dazed and confused, but trying to continue
>[11958.265644] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[11958.265644] Do you have a strange power saving mode enabled?
>[11958.265644] Dazed and confused, but trying to continue
>[11958.265644] Uhhuh. NMI received for unknown reason 21 on CPU 0.
>[11958.265644] Do you have a strange power saving mode enabled?
>[11958.265644] Dazed and confused, but trying to continue
>[11958.265644] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[11958.265644] Do you have a strange power saving mode enabled?
>[11958.265644] Dazed and confused, but trying to continue
>[11958.611023] Uhhuh. NMI received for unknown reason 31 on CPU 0.
>[11958.617342] ------------[ cut here ]------------
>[11958.617342] WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0xac/0xd0()
>[11958.617342] Hardware name: Sun Fire(tm) V60 370-6037               
>[11958.617342] Watchdog detected hard LOCKUP on cpu 1
>[11958.617342] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[11958.617342] Pid: 193, comm: scsi_eh_3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1
>[11958.617342] Call Trace:
>[11958.617342]  [<c044d6a2>] warn_slowpath_common+0x72/0xa0
>[11958.617342]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[11958.617342]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[11958.617342]  [<c04ccb60>] ? touch_nmi_watchdog+0x70/0x70
>[11958.617342]  [<c044d773>] warn_slowpath_fmt+0x33/0x40
>[11958.617342]  [<c04ccc0c>] watchdog_overflow_callback+0xac/0xd0
>[11958.617342]  [<c04ff986>] __perf_event_overflow+0xa6/0x270
>[11958.617342]  [<c042021f>] ? x86_perf_event_set_period+0x11f/0x230
>[11958.617342]  [<c05004f5>] perf_event_overflow+0x15/0x20
>[11958.617342]  [<c0422033>] p4_pmu_handle_irq+0x143/0x1f0
>[11958.617342]  [<c047191a>] ? hrtimer_forward+0x10a/0x250
>[11958.617342]  [<c098424b>] perf_event_nmi_handler+0x1b/0x20
>[11958.617342]  [<c09839c9>] nmi_handle.isra.0+0x39/0x60
>[11958.617342]  [<c07a6740>] ? ata_sff_wait_after_reset+0x110/0x110
>[11958.617342]  [<c0983bcf>] do_nmi+0x1df/0x3f0
>[11958.617342]  [<c07a6740>] ? ata_sff_wait_after_reset+0x110/0x110
>[11958.617342]  [<c0983107>] nmi_stack_correct+0x2f/0x34
>[11958.617342]  [<c07a6070>] ? ata_pci_sff_prepare_host+0xb0/0xb0
>[11958.617342]  [<c07a6740>] ? ata_sff_wait_after_reset+0x110/0x110
>[11958.617342]  [<c045007b>] ? devkmsg_read+0x1eb/0x430
>[11958.617342]  [<c04300d8>] ? perf_ibs_init+0x1d8/0x280
>[11958.617342]  [<c04800e0>] ? sched_rt_handler+0xb0/0x170
>[11958.617342]  [<c0693be3>] ? ioread16+0x33/0x40
>[11958.617342]  [<c07a60a3>] ? ata_sff_drain_fifo+0x33/0x80
>[11958.617342]  [<c07a6a80>] ? ata_pio_sectors+0xa0/0xa0
>[11958.617342]  [<c07a6dc7>] ata_sff_error_handler+0x67/0xe0
>[11958.617342]  [<c07a00d8>] ? ata_eh_release+0x28/0x60
>[11958.617342]  [<c07a6e93>] ata_bmdma_error_handler+0x53/0xf0
>[11958.617342]  [<c07a4333>] ata_scsi_port_error_handler+0x433/0x790
>[11958.617342]  [<c07a8b76>] ? ata_sff_flush_pio_task+0x16/0x60
>[11958.617342]  [<c07a4705>] ata_scsi_error+0x75/0xa0
>[11958.617342]  [<c077c63f>] scsi_error_handler+0xdf/0x610
>[11958.617342]  [<c047e590>] ? default_wake_function+0x10/0x20
>[11958.617342]  [<c04761b5>] ? __wake_up_common+0x45/0x70
>[11958.617342]  [<c047931e>] ? complete+0x4e/0x60
>[11958.617342]  [<c077c560>] ? scsi_eh_get_sense+0x1b0/0x1b0
>[11958.617342]  [<c046d534>] kthread+0x94/0xa0
>[11958.617342]  [<c0989937>] ret_from_kernel_thread+0x1b/0x28
>[11958.617342]  [<c046d4a0>] ? kthread_create_on_node+0xc0/0xc0
>[11958.617342] ---[ end trace 8435b7bed5fce0f1 ]---
>[11964.213320] Do you have a strange power saving mode enabled?
>[11992.176579] BUG: soft lockup - CPU#2 stuck for 22s! [java:32079]
>[11992.176579] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[11992.176579] Pid: 32079, comm: java Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11992.176579] EIP: 0060:[<c04a5f9a>] EFLAGS: 00000202 CPU: 2
>[11992.176579] EIP is at generic_exec_single+0x5a/0x80
>[11992.176579] EAX: 00000202 EBX: f75c9700 ECX: 00000001 EDX: fffff000
>[11992.176579] ESI: f07c1e2c EDI: f75c9708 EBP: f07c1e20 ESP: f07c1e04
>[11992.176579]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11992.176579] CR0: 80050033 CR2: 65262fd8 CR3: 31718000 CR4: 000007f0
>[11992.176579] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11992.176579] DR6: ffff0ff0 DR7: 00000400
>[11992.176579] Process java (pid: 32079, ti=f07c0000 task=f70ef110 task.ti=f07c0000)
>[11992.176579] Stack:
>[11992.176579]  00000001 f75c9700 00000286 00000001 00000001 f07c1e2c c0445090 f07c1e4c
>[11992.176579]  c04a606c f07c1e80 f75c9700 f75c9700 c0445090 f07c1e80 00000001 00000001
>[11992.176579]  00000002 f65bab64 f07c1e74 c04a63a4 00000001 f67d9268 015baa00 f07c1e80
>[11992.176579] Call Trace:
>[11992.176579]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[11992.176579]  [<c04a606c>] smp_call_function_single+0xac/0x150
>[11992.176579]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[11992.176579]  [<c04a63a4>] smp_call_function_many+0x1b4/0x210
>[11992.176579]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[11992.176579]  [<c044516b>] native_flush_tlb_others+0x2b/0x30
>[11992.176579]  [<c0445228>] flush_tlb_mm_range+0x58/0x1f0
>[11992.176579]  [<c05290ec>] tlb_flush_mmu+0x8c/0x90
>[11992.176579]  [<c0529107>] tlb_finish_mmu+0x17/0x40
>[11992.176579]  [<c052b263>] zap_page_range+0xa3/0xd0
>[11992.176579]  [<c05280de>] sys_madvise+0x20e/0x640
>[11992.176579]  [<c045f2d5>] ? __set_task_blocked+0x35/0x80
>[11992.176579]  [<c0498f07>] ? getnstimeofday+0x47/0x130
>[11992.176579]  [<c04bf65c>] ? __audit_syscall_entry+0xbc/0x290
>[11992.176579]  [<c04bfb86>] ? __audit_syscall_exit+0x356/0x3b0
>[11992.176579]  [<c09899cd>] sysenter_do_call+0x12/0x28
>[11992.176579] Code: 89 d9 89 45 ec 8b 03 89 45 e8 89 f0 e8 a0 01 1f 00 8b 55 ec 89 f8 e8 c6 c5 4d 00 3b 5d e8 74 19 8b 4d e4 85 c9 75 04 eb 08 f3 90 <f6> 46 10 01 75 f8 83 c4 10 5b 5e 5f 5d c3 8b 45 f0 ff 15 08 7d
>[12018.587660] INFO: rcu_sched detected stalls on CPUs/tasks: { 0 1} (detected by 3, t=60033 jiffies)
>[12018.587660] sending NMI to all CPUs:
>[12018.874165] NMI backtrace for cpu 2
>[12018.874165] Pid: 32079, comm: java Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[12018.874165] EIP: 0060:[<c085b211>] EFLAGS: 00000046 CPU: 2
>[12018.874165] EIP is at acpi_pm_read+0x11/0x20
>[12018.874165] EAX: 003e1e77 EBX: c0c159c0 ECX: f70fbf80 EDX: 00000000
>[12018.874165] ESI: f75d27b0 EDI: 0106fb9c EBP: f70fbe9c ESP: f70fbe9c
>[12018.874165]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[12018.874165] CR0: 80050033 CR2: 65262fd8 CR3: 31718000 CR4: 000007f0
>[12018.874165] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[12018.874165] DR6: ffff0ff0 DR7: 00000400
>[12018.874165] Process java (pid: 32079, ti=f70fa000 task=f70ef110 task.ti=f07c0000)
>[12018.874165] Stack:
>[12018.874165]  f70fbeb8 c04990be 00002ef1 00000000 f75d2980 f75d27b0 00000001 f70fbed8
>[12018.874165]  c04a11c5 f70fbed8 c0471370 f70fbf80 f75d2980 f75d27b0 00000001 f70fbf08
>[12018.874165]  c047152d 00000000 f75d2838 00000017 c04a11a0 f75d27b0 f75d2780 f70fbf4c
>[12018.874165] Call Trace:
>[12018.874165]  [<c04990be>] ktime_get+0x5e/0x100
>[12018.874165]  [<c04a11c5>] tick_sched_timer+0x25/0xf0
>[12018.874165]  [<c0471370>] ? __remove_hrtimer+0x40/0xa0
>[12018.874165]  [<c047152d>] __run_hrtimer+0x6d/0x190
>[12018.874165]  [<c04a11a0>] ? tick_nohz_handler+0x100/0x100
>[12018.874165]  [<c04721bf>] hrtimer_interrupt+0xef/0x280
>[12018.874165]  [<c04806c7>] ? sched_clock_cpu+0xe7/0x190
>[12018.874165]  [<c0434689>] smp_apic_timer_interrupt+0x59/0x90
>[12018.874165]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[12018.874165]  [<c0982de0>] apic_timer_interrupt+0x34/0x3c
>[12018.874165]  [<c045007b>] ? devkmsg_read+0x1eb/0x430
>[12018.874165]  [<c045cc20>] ? del_timer+0x70/0x70
>[12018.874165]  [<c045596f>] ? __do_softirq+0x9f/0x1b0
>[12018.874165]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[12018.874165]  <IRQ> 
>[12018.874165]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[12018.874165]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[12018.874165]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[12018.874165]  [<c04a5f9a>] ? generic_exec_single+0x5a/0x80
>[12018.874165]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[12018.874165]  [<c04a606c>] ? smp_call_function_single+0xac/0x150
>[12018.874165]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[12018.874165]  [<c04a63a4>] ? smp_call_function_many+0x1b4/0x210
>[12018.874165]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[12018.874165]  [<c044516b>] ? native_flush_tlb_others+0x2b/0x30
>[12018.874165]  [<c0445228>] ? flush_tlb_mm_range+0x58/0x1f0
>[12018.874165]  [<c05290ec>] ? tlb_flush_mmu+0x8c/0x90
>[12018.874165]  [<c0529107>] ? tlb_finish_mmu+0x17/0x40
>[12018.874165]  [<c052b263>] ? zap_page_range+0xa3/0xd0
>[12018.874165]  [<c05280de>] ? sys_madvise+0x20e/0x640
>[12018.874165]  [<c045f2d5>] ? __set_task_blocked+0x35/0x80
>[12018.874165]  [<c0498f07>] ? getnstimeofday+0x47/0x130
>[12018.874165]  [<c04bf65c>] ? __audit_syscall_entry+0xbc/0x290
>[12018.874165]  [<c04bfb86>] ? __audit_syscall_exit+0x356/0x3b0
>[12018.874165]  [<c09899cd>] ? sysenter_do_call+0x12/0x28
>[12018.874165] Code: 10 00 00 89 14 24 e8 df 21 e3 ff 83 c4 14 5b 5d c3 66 0f 1f 84 00 00 00 00 00 55 89 e5 3e 8d 74 26 00 8b 15 40 3c c3 c0 ed 31 d2 <25> ff ff ff 00 5d c3 90 8d b4 26 00 00 00 00 55 89 e5 57 56 53
>[11958.617342] NMI backtrace for cpu 1
>[11958.617342] Pid: 193, comm: scsi_eh_3 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[11958.617342] EIP: 0060:[<c0693c22>] EFLAGS: 00000092 CPU: 1
>[11958.617342] EIP is at ioread8+0x32/0x40
>[11958.617342] EAX: f7e240ff EBX: f6428000 ECX: c07a6070 EDX: f7e2a140
>[11958.617342] ESI: 00005010 EDI: c07a6740 EBP: f17c7e4c ESP: f17c7e48
>[11958.617342]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[11958.617342] CR0: 8005003b CR2: 46839650 CR3: 31718000 CR4: 000007f0
>[11958.617342] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[11958.617342] DR6: ffff0ff0 DR7: 00000400
>[11958.617342] Process scsi_eh_3 (pid: 193, ti=f17c6000 task=f17c9920 task.ti=f17c6000)
>[11958.617342] Stack:
>[11958.617342]  c07a5870 f17c7e6c c07a60ae f6428000 00000000 00000202 00000202 f6428000
>[11958.617342]  c07a6a80 f17c7e90 c07a6dc7 017c007b c07a00d8 f64280c4 00000202 f6428000
>[11958.617342]  f64280c4 00000202 f17c7ea4 c07a6e93 00000000 00000001 f64296d4 f17c7eec
>[11958.617342] Call Trace:
>[11958.617342]  [<c07a5870>] ? ata_sff_check_status+0x10/0x20
>[11958.617342]  [<c07a60ae>] ata_sff_drain_fifo+0x3e/0x80
>[11958.617342]  [<c07a6a80>] ? ata_pio_sectors+0xa0/0xa0
>[11958.617342]  [<c07a6dc7>] ata_sff_error_handler+0x67/0xe0
>[11958.617342]  [<c07a00d8>] ? ata_eh_release+0x28/0x60
>[11958.617342]  [<c07a6e93>] ata_bmdma_error_handler+0x53/0xf0
>[11958.617342]  [<c07a4333>] ata_scsi_port_error_handler+0x433/0x790
>[11958.617342]  [<c07a8b76>] ? ata_sff_flush_pio_task+0x16/0x60
>[11958.617342]  [<c07a4705>] ata_scsi_error+0x75/0xa0
>[11958.617342]  [<c077c63f>] scsi_error_handler+0xdf/0x610
>[11958.617342]  [<c047e590>] ? default_wake_function+0x10/0x20
>[11958.617342]  [<c04761b5>] ? __wake_up_common+0x45/0x70
>[11958.617342]  [<c047931e>] ? complete+0x4e/0x60
>[11958.617342]  [<c077c560>] ? scsi_eh_get_sense+0x1b0/0x1b0
>[11958.617342]  [<c046d534>] kthread+0x94/0xa0
>[11958.617342]  [<c0989937>] ret_from_kernel_thread+0x1b/0x28
>[11958.617342]  [<c046d4a0>] ? kthread_create_on_node+0xc0/0xc0
>[11958.617342] Code: 3d 00 00 01 00 76 0a 0f b7 d0 ec 0f b6 c0 c3 66 90 55 ba 42 55 b2 c0 89 e5 e8 cb fd ff ff b8 ff 00 00 00 5d c3 8d 74 26 00 8a 00 <0f> b6 c0 c3 8d 76 00 8d bc 27 00 00 00 00 55 3d ff ff 03 00 89
>[12018.587660] NMI backtrace for cpu 3
>[12018.587660] Pid: 506, comm: jbd2/sda6-8 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[12018.587660] EIP: 0060:[<c068d880>] EFLAGS: 00000006 CPU: 3
>[12018.587660] EIP is at __const_udelay+0x0/0x20
>[12018.587660] EAX: 00418958 EBX: 00002710 ECX: fffff000 EDX: fffff000
>[12018.587660] ESI: c0bd8a00 EDI: 00000002 EBP: f7121df0 ESP: f7121de4
>[12018.587660]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[12018.587660] CR0: 8005003b CR2: bfc1c3d8 CR3: 36451000 CR4: 000007f0
>[12018.587660] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[12018.587660] DR6: ffff0ff0 DR7: 00000400
>[12018.587660] Process jbd2/sda6-8 (pid: 506, ti=f7120000 task=f67f25b0 task.ti=f665a000)
>[12018.587660] Stack:
>[12018.587660]  c0435a2a c0ae75ef f75e0ba0 f7121e24 c04d5550 c0afef44 00000003 0000ea81
>[12018.587660]  00000001 00000083 00000002 00000003 c0bd8a00 f67f25b0 00000000 00000003
>[12018.587660]  f7121e38 c045d6ab f75e0980 4efba582 00000aee f7121e58 c04a1208 f7121e58
>[12018.587660] Call Trace:
>[12018.587660]  [<c0435a2a>] ? arch_trigger_all_cpu_backtrace+0x5a/0x80
>[12018.587660]  [<c04d5550>] rcu_check_callbacks+0x4b0/0x540
>[12018.587660]  [<c045d6ab>] update_process_times+0x3b/0x70
>[12018.587660]  [<c04a1208>] tick_sched_timer+0x68/0xf0
>[12018.587660]  [<c0471370>] ? __remove_hrtimer+0x40/0xa0
>[12018.587660]  [<c047152d>] __run_hrtimer+0x6d/0x190
>[12018.587660]  [<c04a11a0>] ? tick_nohz_handler+0x100/0x100
>[12018.587660]  [<c04721bf>] hrtimer_interrupt+0xef/0x280
>[12018.587660]  [<c04806c7>] ? sched_clock_cpu+0xe7/0x190
>[12018.587660]  [<c0434689>] smp_apic_timer_interrupt+0x59/0x90
>[12018.587660]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[12018.587660]  [<c0982de0>] apic_timer_interrupt+0x34/0x3c
>[12018.587660]  [<c098255f>] ? _raw_spin_unlock_irqrestore+0xf/0x20
>[12018.587660]  [<c0483c73>] update_shares+0xa3/0xd0
>[12018.587660]  [<c04863bf>] rebalance_domains+0x3f/0x150
>[12018.587660]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[12018.587660]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[12018.587660]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[12018.587660]  <IRQ> 
>[12018.587660]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[12018.587660]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[12018.587660]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[12018.587660]  [<c060844f>] ? __jbd2_journal_file_buffer+0x7f/0x1c0
>[12018.587660]  [<c0582df1>] ? set_bh_page+0x51/0x80
>[12018.587660]  [<c0611964>] ? jbd2_journal_write_metadata_buffer+0x204/0x2f0
>[12018.587660]  [<c0609ddb>] ? jbd2_journal_commit_transaction+0x61b/0x15e0
>[12018.587660]  [<c060f080>] ? kjournald2+0xa0/0x200
>[12018.587660]  [<c046dad0>] ? wake_up_bit+0x30/0x30
>[12018.587660]  [<c060efe0>] ? jbd2_journal_release_jbd_inode+0xd0/0xd0
>[12018.587660]  [<c046d534>] ? kthread+0x94/0xa0
>[12018.587660]  [<c0989937>] ? ret_from_kernel_thread+0x1b/0x28
>[12018.587660]  [<c046d4a0>] ? kthread_create_on_node+0xc0/0xc0
>[12018.587660] Code: 00 00 00 00 8d bc 27 00 00 00 00 48 75 fd 48 5d c3 8d 76 00 8d bc 27 00 00 00 00 55 89 e5 3e 8d 74 26 00 ff 15 c0 fa bf c0 5d c3 <55> 89 e5 64 8b 15 dc bf cd c0 c1 e0 02 69 d2 fa 00 00 00 f7 e2
>[12018.587660] ------------[ cut here ]------------
>[12018.587660] WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0xac/0xd0()
>[12018.587660] Hardware name: Sun Fire(tm) V60 370-6037               
>[12018.587660] Watchdog detected hard LOCKUP on cpu 3
>[12018.587660] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[12018.587660] Pid: 506, comm: jbd2/sda6-8 Tainted: G        W    3.7.3-101.fc17.i686.PAE #1
>[12018.587660] Call Trace:
>[12018.587660]  [<c044d6a2>] warn_slowpath_common+0x72/0xa0
>[12018.587660]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[12018.587660]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[12018.587660]  [<c04ccb60>] ? touch_nmi_watchdog+0x70/0x70
>[12018.587660]  [<c044d773>] warn_slowpath_fmt+0x33/0x40
>[12018.587660]  [<c04ccc0c>] watchdog_overflow_callback+0xac/0xd0
>[12018.587660]  [<c04ff986>] __perf_event_overflow+0xa6/0x270
>[12018.587660]  [<c042021f>] ? x86_perf_event_set_period+0x11f/0x230
>[12018.587660]  [<c05004f5>] perf_event_overflow+0x15/0x20
>[12018.587660]  [<c0422033>] p4_pmu_handle_irq+0x143/0x1f0
>[12018.587660]  [<c098424b>] perf_event_nmi_handler+0x1b/0x20
>[12018.587660]  [<c09839c9>] nmi_handle.isra.0+0x39/0x60
>[12018.587660]  [<c0983ad8>] do_nmi+0xe8/0x3f0
>[12018.587660]  [<c0983107>] ? nmi_stack_correct+0x2f/0x34
>[12018.587660]  [<c0983107>] nmi_stack_correct+0x2f/0x34
>[12018.587660]  [<c0435a2a>] ? arch_trigger_all_cpu_backtrace+0x5a/0x80
>[12018.587660]  [<c04d5550>] rcu_check_callbacks+0x4b0/0x540
>[12018.587660]  [<c045d6ab>] update_process_times+0x3b/0x70
>[12018.587660]  [<c04a1208>] tick_sched_timer+0x68/0xf0
>[12018.587660]  [<c0471370>] ? __remove_hrtimer+0x40/0xa0
>[12018.587660]  [<c047152d>] __run_hrtimer+0x6d/0x190
>[12018.587660]  [<c04a11a0>] ? tick_nohz_handler+0x100/0x100
>[12018.587660]  [<c04721bf>] hrtimer_interrupt+0xef/0x280
>[12018.587660]  [<c04806c7>] ? sched_clock_cpu+0xe7/0x190
>[12018.587660]  [<c0434689>] smp_apic_timer_interrupt+0x59/0x90
>[12018.587660]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[12018.587660]  [<c0982de0>] apic_timer_interrupt+0x34/0x3c
>[12018.587660]  [<c098255f>] ? _raw_spin_unlock_irqrestore+0xf/0x20
>[12018.587660]  [<c0483c73>] update_shares+0xa3/0xd0
>[12018.587660]  [<c04863bf>] rebalance_domains+0x3f/0x150
>[12018.587660]  [<c048650a>] run_rebalance_domains+0x3a/0x120
>[12018.587660]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[12018.587660]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[12018.587660]  <IRQ>  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[12018.587660]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[12018.587660]  [<c0982de0>] ? apic_timer_interrupt+0x34/0x3c
>[12018.587660]  [<c060844f>] ? __jbd2_journal_file_buffer+0x7f/0x1c0
>[12018.587660]  [<c0582df1>] ? set_bh_page+0x51/0x80
>[12018.587660]  [<c0611964>] ? jbd2_journal_write_metadata_buffer+0x204/0x2f0
>[12018.587660]  [<c0609ddb>] ? jbd2_journal_commit_transaction+0x61b/0x15e0
>[12018.587660]  [<c060f080>] ? kjournald2+0xa0/0x200
>[12018.587660]  [<c046dad0>] ? wake_up_bit+0x30/0x30
>[12018.587660]  [<c060efe0>] ? jbd2_journal_release_jbd_inode+0xd0/0xd0
>[12018.587660]  [<c046d534>] ? kthread+0x94/0xa0
>[12018.587660]  [<c0989937>] ? ret_from_kernel_thread+0x1b/0x28
>[12018.587660]  [<c046d4a0>] ? kthread_create_on_node+0xc0/0xc0
>[12018.587660] ---[ end trace 8435b7bed5fce0f2 ]---
>[12023.304690] ------------[ cut here ]------------
>[12023.304690] WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0xac/0xd0()
>[12023.304690] Hardware name: Sun Fire(tm) V60 370-6037               
>[12023.304690] Watchdog detected hard LOCKUP on cpu 2
>[12023.304690] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[12023.304690] Pid: 32079, comm: java Tainted: G        W    3.7.3-101.fc17.i686.PAE #1
>[12023.304690] Call Trace:
>[12023.304690]  [<c044d6a2>] warn_slowpath_common+0x72/0xa0
>[12023.304690]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[12023.304690]  [<c04ccc0c>] ? watchdog_overflow_callback+0xac/0xd0
>[12023.304690]  [<c04ccb60>] ? touch_nmi_watchdog+0x70/0x70
>[12023.304690]  [<c044d773>] warn_slowpath_fmt+0x33/0x40
>[12023.304690]  [<c04ccc0c>] watchdog_overflow_callback+0xac/0xd0
>[12023.304690]  [<c04ff986>] __perf_event_overflow+0xa6/0x270
>[12023.304690]  [<c042021f>] ? x86_perf_event_set_period+0x11f/0x230
>[12023.304690]  [<c05004f5>] perf_event_overflow+0x15/0x20
>[12023.304690]  [<c0422033>] p4_pmu_handle_irq+0x143/0x1f0
>[12023.304690]  [<c04184d8>] ? sched_clock+0x8/0x10
>[12023.304690]  [<c098424b>] perf_event_nmi_handler+0x1b/0x20
>[12023.304690]  [<c09839c9>] nmi_handle.isra.0+0x39/0x60
>[12023.304690]  [<c0983bcf>] do_nmi+0x1df/0x3f0
>[12023.304690]  [<c0455bb7>] ? irq_exit+0x67/0xb0
>[12023.304690]  [<c043468e>] ? smp_apic_timer_interrupt+0x5e/0x90
>[12023.304690]  [<c0983107>] nmi_stack_correct+0x2f/0x34
>[12023.304690]  [<c04a5f9a>] ? generic_exec_single+0x5a/0x80
>[12023.304690]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[12023.304690]  [<c04a606c>] smp_call_function_single+0xac/0x150
>[12023.304690]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[12023.304690]  [<c04a63a4>] smp_call_function_many+0x1b4/0x210
>[12023.304690]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[12023.304690]  [<c044516b>] native_flush_tlb_others+0x2b/0x30
>[12023.304690]  [<c0445228>] flush_tlb_mm_range+0x58/0x1f0
>[12023.304690]  [<c05290ec>] tlb_flush_mmu+0x8c/0x90
>[12023.304690]  [<c0529107>] tlb_finish_mmu+0x17/0x40
>[12023.304690]  [<c052b263>] zap_page_range+0xa3/0xd0
>[12023.304690]  [<c05280de>] sys_madvise+0x20e/0x640
>[12023.304690]  [<c045f2d5>] ? __set_task_blocked+0x35/0x80
>[12023.304690]  [<c0498f07>] ? getnstimeofday+0x47/0x130
>[12023.304690]  [<c04bf65c>] ? __audit_syscall_entry+0xbc/0x290
>[12023.304690]  [<c04bfb86>] ? __audit_syscall_exit+0x356/0x3b0
>[12023.304690]  [<c09899cd>] sysenter_do_call+0x12/0x28
>[12023.304690] ---[ end trace 8435b7bed5fce0f3 ]---
>[12048.292038] BUG: soft lockup - CPU#2 stuck for 22s! [java:32079]
>[12048.292038] Modules linked in: nfsv3 nfs_acl nfsv4 auth_rpcgss nfs dns_resolver fscache lockd sunrpc ip6t_REJECT ip6table_filter ip6_tables iptable_raw e1000 i2c_i801 iTCO_wdt iTCO_vendor_support lpc_ich serio_raw i2c_core mfd_core e7xxx_edac edac_core microcode binfmt_misc uinput sata_sil
>[12048.292038] Pid: 32079, comm: java Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[12048.292038] EIP: 0060:[<c04a5f9a>] EFLAGS: 00000202 CPU: 2
>[12048.292038] EIP is at generic_exec_single+0x5a/0x80
>[12048.292038] EAX: 00000202 EBX: f75c9700 ECX: 00000001 EDX: fffff000
>[12048.292038] ESI: f07c1e2c EDI: f75c9708 EBP: f07c1e20 ESP: f07c1e04
>[12048.292038]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[12048.292038] CR0: 80050033 CR2: 65262fd8 CR3: 31718000 CR4: 000007f0
>[12048.292038] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[12048.292038] DR6: ffff0ff0 DR7: 00000400
>[12048.292038] Process java (pid: 32079, ti=f07c0000 task=f70ef110 task.ti=f07c0000)
>[12048.292038] Stack:
>[12048.292038]  00000001 f75c9700 00000286 00000001 00000001 f07c1e2c c0445090 f07c1e4c
>[12048.292038]  c04a606c f07c1e80 f75c9700 f75c9700 c0445090 f07c1e80 00000001 00000001
>[12048.292038]  00000002 f65bab64 f07c1e74 c04a63a4 00000001 f67d9268 015baa00 f07c1e80
>[12048.292038] Call Trace:
>[12048.292038]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[12048.292038]  [<c04a606c>] smp_call_function_single+0xac/0x150
>[12048.292038]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[12048.292038]  [<c04a63a4>] smp_call_function_many+0x1b4/0x210
>[12048.292038]  [<c0445090>] ? do_flush_tlb_all+0x50/0x50
>[12048.292038]  [<c044516b>] native_flush_tlb_others+0x2b/0x30
>[12048.292038]  [<c0445228>] flush_tlb_mm_range+0x58/0x1f0
>[12048.292038]  [<c05290ec>] tlb_flush_mmu+0x8c/0x90
>[12048.292038]  [<c0529107>] tlb_finish_mmu+0x17/0x40
>[12048.292038]  [<c052b263>] zap_page_range+0xa3/0xd0
>[12048.292038]  [<c05280de>] sys_madvise+0x20e/0x640
>[12048.292038]  [<c045f2d5>] ? __set_task_blocked+0x35/0x80
>[12048.292038]  [<c0498f07>] ? getnstimeofday+0x47/0x130
>[12048.292038]  [<c04bf65c>] ? __audit_syscall_entry+0xbc/0x290
>[12048.292038]  [<c04bfb86>] ? __audit_syscall_exit+0x356/0x3b0
>[12048.292038]  [<c09899cd>] sysenter_do_call+0x12/0x28
>[12048.292038] Code: 89 d9 89 45 ec 8b 03 89 45 e8 89 f0 e8 a0 01 1f 00 8b 55 ec 89 f8 e8 c6 c5 4d 00 3b 5d e8 74 19 8b 4d e4 85 c9 75 04 eb 08 f3 90 <f6> 46 10 01 75 f8 83 c4 10 5b 5e 5f 5d c3 8b 45 f0 ff 15 08 7d
>[11958.617342] ata4: drained 65536 bytes to clear DRQ
>[12058.934575] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
>[12058.934581] ata4.00: failed command: READ DMA EXT
>[12058.934590] ata4.00: cmd 25/00:00:00:db:8b/00:01:31:00:00/e0 tag 0 dma 131072 in
>[12058.934590]          res ff/ff:ff:ff:ff:ff/ff:ff:ff:ff:ff/ff Emask 0x2 (HSM violation)
>[12058.934592] ata4.00: status: { Busy }
>[12058.934594] ata4.00: error: { ICRC UNC IDNF ABRT }
>[12058.934606] ata4: hard resetting link
>[12059.242061] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
>[12058.936753] Dazed and confused, but trying to continue
>[12058.936753] NMI backtrace for cpu 0
>[12058.936753] Pid: 597, comm: rsyslogd Tainted: G        W    3.7.3-101.fc17.i686.PAE #1 Sun Microsystems     Sun Fire(tm) V60 370-6037               /SE7501WV2S
>[12058.936753] EIP: 0060:[<f7ffc747>] EFLAGS: 00000206 CPU: 0
>[12058.936753] EIP is at e1000_clean_rx_irq+0x117/0x450 [e1000]
>[12058.936753] EAX: eec7a340 EBX: f668a460 ECX: f700bf7c EDX: 8010000b
>[12058.936753] ESI: f14f5a00 EDI: f7f9a9d8 EBP: f700bee8 ESP: f700be74
>[12058.936753]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
>[12058.936753] CR0: 80050033 CR2: 452a57c0 CR3: 364d5000 CR4: 000007f0
>[12058.936753] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>[12058.936753] DR6: ffff0ff0 DR7: 00000400
>[12058.936753] Process rsyslogd (pid: 597, ti=f700a000 task=f72ff110 task.ti=edbe4000)
>[12058.936753] Stack:
>[12058.936753]  f0d8e100 c08835ad ee6b2800 f14f5a00 f6709900 c08835ad f334c120 c0882005
>[12058.936753]  f71f9000 f712a808 f712a71c f71f9064 f712a000 0000000b 0000410e f7f9a9d8
>[12058.936753]  f700bf7c 000005f2 f668a460 0000000b f712a500 f660b680 2300bed8 00000046
>[12058.936753] Call Trace:
>[12058.936753]  [<c08835ad>] ? __kfree_skb+0x3d/0x90
>[12058.936753]  [<c08835ad>] ? __kfree_skb+0x3d/0x90
>[12058.936753]  [<c0882005>] ? skb_free_head+0x45/0x50
>[12058.936753]  [<f7ffd218>] e1000_clean+0x1d8/0x850 [e1000]
>[12058.936753]  [<c04184d8>] ? sched_clock+0x8/0x10
>[12058.936753]  [<c0480482>] ? sched_clock_local+0xb2/0x1a0
>[12058.936753]  [<c088fedd>] net_rx_action+0x11d/0x1f0
>[12058.936753]  [<c045596f>] __do_softirq+0x9f/0x1b0
>[12058.936753]  [<c04558d0>] ? local_bh_enable_ip+0x90/0x90
>[12058.936753]  <IRQ> 
>[12058.936753]  [<c0455bed>] ? irq_exit+0x9d/0xb0
>[12058.936753]  [<c0413e5b>] ? do_IRQ+0x4b/0xc0
>[12058.936753]  [<c047e5af>] ? wake_up_state+0xf/0x20
>[12058.936753]  [<c04a2cc5>] ? wake_futex+0x65/0x80
>[12058.936753]  [<c0989f33>] ? common_interrupt+0x33/0x38
>[12058.936753]  [<c041007b>] ? xen_cpu_die+0x2b/0xf0
>[12058.936753]  [<c047a38f>] ? finish_task_switch+0x3f/0xc0
>[12058.936753]  [<c0981300>] ? __schedule+0x370/0x770
>[12058.936753]  [<c04bfb86>] ? __audit_syscall_exit+0x356/0x3b0
>[12058.936753]  [<c04bfb86>] ? __audit_syscall_exit+0x356/0x3b0
>[12058.936753]  [<c0981993>] ? schedule+0x23/0x60
>[12058.936753]  [<c0982b86>] ? work_resched+0x5/0x20
>[12058.936753]  [<c0980000>] ? check_tnode.part.14+0x3/0x14
>[12058.936753] Code: 01 00 00 8b 4c 24 40 83 c0 01 89 01 0f ae e8 89 f6 0f b6 43 0c 8b 37 c7 07 00 00 00 00 88 44 24 5b 89 74 24 0c 8b 86 b0 00 00 00 <0f> 18 00 90 8b 54 24 54 83 44 24 5c 01 8b 74 24 5c 3b 72 10 0f
>[12059.730994] ata4.00: failed to get Identify Device Data, Emask 0x1
>[12059.735716] ata4.00: failed to get Identify Device Data, Emask 0x1
>[12059.735765] ata4.00: configured for UDMA/33
>[12059.735861] sd 3:0:0:0: [sdb]  
>[12059.735873] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
>[12059.735883] sd 3:0:0:0: [sdb]  
>[12059.735892] Sense Key : Aborted Command [current] [descriptor]
>[12059.735909] Descriptor sense data with sense descriptors (in hex):
>[12059.735916]         72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 ff ff 
>[12059.735956]         ff ff ff ff 
>[12059.735972] sd 3:0:0:0: [sdb]  
>[12059.735993] Add. Sense: Scsi parity error
>[12059.736051] sd 3:0:0:0: [sdb] CDB: 
>[12059.736058] Read(10): 28 00 31 8b db 00 00 01 00 00
>[12059.736103] end_request: I/O error, dev sdb, sector 831249152
>[12059.753972] ata4: EH complete

Actions: View

Attachments on bug 918889: 706385 | 706698 | 706711