Bug 785033

Summary: Kernel panic (rcu_bh_state detected stall)
Product: [Fedora] Fedora Reporter: Jussi Eloranta <eloranta>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 16CC: gansalmon, itamar, jonathan, kernel-maint, madhu.chinakonda, sgruszka
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-09-04 15:50:20 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Jussi Eloranta 2012-01-27 02:44:43 UTC 
Description of problem:

This is on a 4 X 4 core Opteron system with 64GB of ram (with NFS mounted home directories). The kernel version is 3.1.7-1.fc16.x86_64. After couple of days uptime & heavy load, I see the following dmesg (the systems appears to run OK though):

[1389863.137022] INFO: rcu_bh_state detected stall on CPU 9 (t=0 jiffies)
[1389863.137022] sending NMI to all CPUs:
[1389863.137135] NMI backtrace for cpu 0
[1389863.137144] CPU 0 
[1389863.137147] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.137192] 
[1389863.137196] Pid: 6579, comm: imag_time Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.137202] RIP: 0033:[<000000000047d57a>]  [<000000000047d57a>] 0x47d579
[1389863.137213] RSP: 002b:00007f7f23f36bb8  EFLAGS: 00000287
[1389863.137216] RAX: 00007f80099f8090 RBX: 00007f80099f8010 RCX: 00007f7f1c008d60
[1389863.137218] RDX: 0000000000000080 RSI: 0000000000000008 RDI: 3a647adc1d780861
[1389863.137221] RBP: 0000000000000010 R08: bb28e8f0e4a82a0e R09: 0000000000001060
[1389863.137223] R10: 000000000000001a R11: 0000000000100000 R12: 00007f7f1c000a60
[1389863.137225] R13: 0000000000000010 R14: 0000000000000100 R15: 000000000047d820
[1389863.137228] FS:  00007f7f23f37700(0000) GS:ffff880417c00000(0000) knlGS:000000005574c6c0
[1389863.137231] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.137233] CR2: 00000000010d2000 CR3: 000000040b2b4000 CR4: 00000000000006f0
[1389863.137236] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.137238] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.137242] Process imag_time (pid: 6579, threadinfo ffff88040c7f8000, task ffff88040bd4dcc0)
[1389863.137244] 
[1389863.137245] Call Trace:
[1389863.137260] NMI backtrace for cpu 3
[1389863.137267] CPU 3 
[1389863.137269] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.137315] 
[1389863.137320] Pid: 24480, comm: molprop_2010_1_ Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.137327] RIP: 0033:[<0000000001114821>]  [<0000000001114821>] 0x1114820
[1389863.137339] RSP: 002b:00007fffe855d8e0  EFLAGS: 00000287
[1389863.137341] RAX: 00000000000023d2 RBX: 0000000000000060 RCX: 0000000000000052
[1389863.137344] RDX: 0000000000000520 RSI: 00000000155d6740 RDI: 0000000000262e00
[1389863.137346] RBP: 0000000023ec2c70 R08: 0000000023ec2c88 R09: 0000000023ec2c40
[1389863.137349] R10: 0000000023ec2c10 R11: 0000000023ec2bf8 R12: 0000000000000cba
[1389863.137351] R13: 0000000023ec2c28 R14: 0000000023ec2c58 R15: 0000000024125488
[1389863.137354] FS:  0000000012f38870(0063) GS:ffff880417d80000(0000) knlGS:000000005574c6c0
[1389863.137357] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.137360] CR2: 0000000001e8ff0a CR3: 00000008731f5000 CR4: 00000000000006e0
[1389863.137363] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.137365] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.137369] Process molprop_2010_1_ (pid: 24480, threadinfo ffff880c37892000, task ffff880c68324590)
[1389863.137371] 
[1389863.137373] Call Trace:
[1389863.137390] NMI backtrace for cpu 2
[1389863.137397] CPU 2 
[1389863.137399] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.137448] 
[1389863.137453] Pid: 19427, comm: molprop_2010_1_ Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.137460] RIP: 0033:[<0000000001843458>]  [<0000000001843458>] 0x1843457
[1389863.137474] RSP: 002b:00007fff70e48040  EFLAGS: 00000283
[1389863.137476] RAX: 00000000e4777948 RBX: 00000000e4408948 RCX: 00000000000002e8
[1389863.137479] RDX: 00000000e4776948 RSI: 00000000000006d8 RDI: 00000000e4774948
[1389863.137482] RBP: 00000000e4775948 R08: 0000000000004000 R09: 0000000000000800
[1389863.137485] R10: 00000000229b6dc0 R11: 00000000229a92c0 R12: 00000000229ac980
[1389863.137487] R13: 0000000000174000 R14: 000000000111c000 R15: 00000000e48eb948
[1389863.137491] FS:  000000001488e870(0063) GS:ffff880417d00000(0000) knlGS:000000005574c6c0
[1389863.137494] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.137496] CR2: 0000000036e3f058 CR3: 0000000870a0f000 CR4: 00000000000006e0
[1389863.137500] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.137502] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.137506] Process molprop_2010_1_ (pid: 19427, threadinfo ffff88040ca0e000, task ffff88040bcdc590)
[1389863.137508] 
[1389863.137510] Call Trace:
[1389863.137529] NMI backtrace for cpu 5
[1389863.137537] CPU 5 
[1389863.137539] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.137590] 
[1389863.137595] Pid: 6581, comm: imag_time Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.137602] RIP: 0033:[<000000000047d567>]  [<000000000047d567>] 0x47d566
[1389863.137616] RSP: 002b:00007f7f22f34bd8  EFLAGS: 00000287
[1389863.137618] RAX: 00007f7f04027050 RBX: 00007f7f04001270 RCX: 00007f8011ac2260
[1389863.137621] RDX: 0000000000000080 RSI: 0000000000000025 RDI: baf1b7380f670811
[1389863.137624] RBP: 0000000000100000 R08: 3d603aaeee0d1b7c R09: 0000000000000010
[1389863.137626] R10: 000000000000009b R11: 0000000000000010 R12: 00007f8011ac2010
[1389863.137629] R13: 0000000000001060 R14: 0000000000000100 R15: 000000000047d8a0
[1389863.137633] FS:  00007f7f22f35700(0000) GS:ffff880817c80000(0000) knlGS:000000005574c6c0
[1389863.137636] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.137638] CR2: 00007f7f0c03f000 CR3: 000000040b2b4000 CR4: 00000000000006e0
[1389863.137642] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.137645] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.137649] Process imag_time (pid: 6581, threadinfo ffff88040c844000, task ffff88040c462e60)
[1389863.137651] 
[1389863.137653] Call Trace:
[1389863.137676] NMI backtrace for cpu 4
[1389863.137687] CPU 4 
[1389863.137689] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.137754] 
[1389863.137760] Pid: 7177, comm: classical Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.137768] RIP: 0033:[<0000000000419459>]  [<0000000000419459>] 0x419458
[1389863.137787] RSP: 002b:00007fb00109cdd0  EFLAGS: 00000202
[1389863.137789] RAX: 0000000000000263 RBX: 0000000000000200 RCX: 00007fb4b9d7a060
[1389863.137794] RDX: 000000000000014a RSI: 000000000001c269 RDI: 000000000001c26a
[1389863.137796] RBP: 0000000000000200 R08: fffffffa7b7ec000 R09: 00007fb4b9d7b1f0
[1389863.137798] R10: 00007faf355671f0 R11: 0000000000000119 R12: 00007fb4818a8010
[1389863.137801] R13: 00007faefd094010 R14: 0000000000002000 R15: 0000000000000003
[1389863.137806] FS:  00007fb00109d700(0000) GS:ffff880817c00000(0000) knlGS:000000005574c6c0
[1389863.137809] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.137811] CR2: 00007f10102b3760 CR3: 000000023ac52000 CR4: 00000000000006e0
[1389863.137815] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.137818] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.137822] Process classical (pid: 7177, threadinfo ffff880c680b6000, task ffff880c683f2e60)
[1389863.137824] 
[1389863.137826] Call Trace:
[1389863.137022] NMI backtrace for cpu 9
[1389863.137022] CPU 9 
[1389863.137022] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.137022] 
[1389863.137022] Pid: 19428, comm: molprop_2010_1_ Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.137022] RIP: 0010:[<ffffffff812b97f2>]  [<ffffffff812b97f2>] find_next_bit+0x12/0xd0
[1389863.137022] RSP: 0000:ffff880c17c83d68  EFLAGS: 00000083
[1389863.137022] RAX: 0000000000000100 RBX: 0000000000000002 RCX: 000000000000000b
[1389863.137022] RDX: 000000000000000b RSI: 0000000000000100 RDI: ffffffff81b60d60
[1389863.137022] RBP: ffff880c17c83d68 R08: ffffffff81b60d60 R09: 000000000000000b
[1389863.137022] R10: 0000000000000000 R11: 0000000000000001 R12: 000000000000d402
[1389863.137022] R13: ffffffff81b60d60 R14: 0000000000000400 R15: 0000000000000096
[1389863.137022] FS:  0000000014197870(0063) GS:ffff880c17c80000(0000) knlGS:000000005574c6c0
[1389863.137022] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.137022] CR2: 0000000084dabfd8 CR3: 0000000ad048b000 CR4: 00000000000006e0
[1389863.137022] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.137022] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.137022] Process molprop_2010_1_ (pid: 19428, threadinfo ffff8801ab4ec000, task ffff88040dcd5cc0)
[1389863.137022] Stack:
[1389863.137022]  ffff880c17c83db8 ffffffff81031693 000000000000000e 000000000000000a
[1389863.137022]  ffff880c17c83dd8 0000000000002710 ffffffff81a30180 ffffffff81a30280
[1389863.137022]  0000000000000000 0004f012ba916259 ffff880c17c83dc8 ffffffff81035987
[1389863.137022] Call Trace:
[1389863.137022]  <IRQ> 
[1389863.137022]  [<ffffffff81031693>] default_send_IPI_mask_sequence_phys+0x53/0xe0
[1389863.137022]  [<ffffffff81035987>] physflat_send_IPI_all+0x17/0x20
[1389863.137022]  [<ffffffff810318a1>] arch_trigger_all_cpu_backtrace+0x61/0xa0
[1389863.137022]  [<ffffffff810e07ef>] __rcu_pending+0x1bf/0x380
[1389863.137022]  [<ffffffff810e0dbb>] rcu_check_callbacks+0x10b/0x120
[1389863.137022]  [<ffffffff8107cb08>] update_process_times+0x48/0x90
[1389863.137022]  [<ffffffff8109f6b4>] tick_sched_timer+0x64/0xc0
[1389863.137022]  [<ffffffff81092100>] __run_hrtimer+0x70/0x1e0
[1389863.137022]  [<ffffffff8109f650>] ? tick_nohz_handler+0x100/0x100
[1389863.137022]  [<ffffffff81019fd9>] ? read_tsc+0x9/0x20
[1389863.137022]  [<ffffffff81092a7b>] hrtimer_interrupt+0xeb/0x210
[1389863.137022]  [<ffffffff815df989>] smp_apic_timer_interrupt+0x69/0x99
[1389863.137022]  [<ffffffff815dd85e>] apic_timer_interrupt+0x6e/0x80
[1389863.137022]  <EOI> 
[1389863.137022] Code: ea 01 48 8b 34 d7 48 85 f6 74 ee 48 c1 e2 06 48 0f bd f6 48 8d 04 32 5d c3 55 48 39 f2 48 89 f0 48 89 e5 73 79 48 89 d1 49 89 d1 <48> c1 e9 06 49 83 e1 c0 4c 8d 04 cf 48 89 f7 4c 29 cf 83 e2 3f 
[1389863.137022] Call Trace:
[1389863.137022]  <IRQ>  [<ffffffff81031693>] default_send_IPI_mask_sequence_phys+0x53/0xe0
[1389863.137022]  [<ffffffff81035987>] physflat_send_IPI_all+0x17/0x20
[1389863.137022]  [<ffffffff810318a1>] arch_trigger_all_cpu_backtrace+0x61/0xa0
[1389863.137022]  [<ffffffff810e07ef>] __rcu_pending+0x1bf/0x380
[1389863.137022]  [<ffffffff810e0dbb>] rcu_check_callbacks+0x10b/0x120
[1389863.137022]  [<ffffffff8107cb08>] update_process_times+0x48/0x90
[1389863.137022]  [<ffffffff8109f6b4>] tick_sched_timer+0x64/0xc0
[1389863.137022]  [<ffffffff81092100>] __run_hrtimer+0x70/0x1e0
[1389863.137022]  [<ffffffff8109f650>] ? tick_nohz_handler+0x100/0x100
[1389863.137022]  [<ffffffff81019fd9>] ? read_tsc+0x9/0x20
[1389863.137022]  [<ffffffff81092a7b>] hrtimer_interrupt+0xeb/0x210
[1389863.137022]  [<ffffffff815df989>] smp_apic_timer_interrupt+0x69/0x99
[1389863.137022]  [<ffffffff815dd85e>] apic_timer_interrupt+0x6e/0x80
[1389863.137022]  <EOI> 
[1389863.138002] NMI backtrace for cpu 8
[1389863.138002] CPU 8 
[1389863.138002] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.138002] 
[1389863.138002] Pid: 24479, comm: molprop_2010_1_ Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.138002] RIP: 0033:[<00000000010fbe7b>]  [<00000000010fbe7b>] 0x10fbe7a
[1389863.138002] RSP: 002b:00007fff95c34710  EFLAGS: 00000283
[1389863.138002] RAX: 0000000000000000 RBX: 000000000000002c RCX: 0000000000000138
[1389863.138002] RDX: 0000000000000001 RSI: 0000000000001373 RDI: 00000000143e0078
[1389863.138002] RBP: 00000000264d61a0 R08: 0000000000009c10 R09: 0000000000000080
[1389863.138002] R10: 0000000000004000 R11: 00000000264da1a0 R12: 000000000000001d
[1389863.138002] R13: 00000000143cb308 R14: 000000000011add0 R15: 000000000000001d
[1389863.138002] FS:  000000001339c870(0063) GS:ffff880c17c00000(0000) knlGS:000000005574c6c0
[1389863.138002] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.138002] CR2: 0000000124576000 CR3: 0000000e1e7f9000 CR4: 00000000000006e0
[1389863.138002] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.138002] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.138002] Process molprop_2010_1_ (pid: 24479, threadinfo ffff880e1e7e6000, task ffff880c68385cc0)
[1389863.138002] 
[1389863.138002] Call Trace:
[1389863.138002] NMI backtrace for cpu 6
[1389863.138002] CPU 6 
[1389863.138002] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.138002] 
[1389863.138002] Pid: 24481, comm: molprop_2010_1_ Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.138002] RIP: 0033:[<0000000001114899>]  [<0000000001114899>] 0x1114898
[1389863.138002] RSP: 002b:00007fff11c377e0  EFLAGS: 00000202
[1389863.138002] RAX: 00000000000022e0 RBX: 0000000000000088 RCX: 0000000000000030
[1389863.138002] RDX: 0000000000000300 RSI: 0000000022ca2ef8 RDI: 000000000065a280
[1389863.138002] RBP: 0000000024a2cf38 R08: 0000000024a2cf70 R09: 0000000024a2cec8
[1389863.138002] R10: 0000000024a2ce58 R11: 0000000024a2ce20 R12: 00000000000017ec
[1389863.138002] R13: 0000000024a2ce90 R14: 0000000024a2cf00 R15: 00000000250871f0
[1389863.138002] FS:  0000000013aa2870(0063) GS:ffff880817d00000(0000) knlGS:000000005574c6c0
[1389863.138002] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.138002] CR2: 00007f10103cc1ab CR3: 0000000c3788b000 CR4: 00000000000006e0
[1389863.138002] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.138002] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.138002] Process molprop_2010_1_ (pid: 24481, threadinfo ffff8808e447a000, task ffff8808e47f0000)
[1389863.138002] 
[1389863.138002] Call Trace:
[1389863.138009] NMI backtrace for cpu 10
[1389863.138009] CPU 10 
[1389863.138009] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.138009] 
[1389863.138009] Pid: 7175, comm: classical Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.138009] RIP: 0033:[<0000000000419435>]  [<0000000000419435>] 0x419434
[1389863.138009] RSP: 002b:00007fffe79105c0  EFLAGS: 00000246
[1389863.138009] RAX: 0000000000000268 RBX: 0000000000000200 RCX: 00007fb4b9d7c060
[1389863.138009] RDX: 000000000000014b RSI: 000000000001c26a RDI: 000000000001c26b
[1389863.138009] RBP: 0000000000000200 R08: fffffffa7b7ec000 R09: 00007fb4b9d7d230
[1389863.138009] R10: 00007faf35569230 R11: 000000000000011d R12: 00007fb4818a8010
[1389863.138009] R13: 00007faefd094010 R14: 0000000000002000 R15: 00000000006d8f20
[1389863.138009] FS:  00007fb704958780(0000) GS:ffff880c17d00000(0000) knlGS:000000005574c6c0
[1389863.138009] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.138009] CR2: 000000019d806000 CR3: 000000023ac52000 CR4: 00000000000006e0
[1389863.138009] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.138009] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.138009] Process classical (pid: 7175, threadinfo ffff88012c15a000, task ffff880104c48000)
[1389863.138009] 
[1389863.138009] Call Trace:
[1389863.138023] NMI backtrace for cpu 1
[1389863.138023] CPU 1 
[1389863.138023] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.138023] 
[1389863.138023] Pid: 19430, comm: molprop_2010_1_ Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.138023] RIP: 0033:[<0000000001843462>]  [<0000000001843462>] 0x1843461
[1389863.138023] RSP: 002b:00007fff52a0f2c0  EFLAGS: 00000206
[1389863.138023] RAX: 00000000e2d191d8 RBX: 00000000e29aa1d8 RCX: 0000000000000000
[1389863.138023] RDX: 00000000e2d181d8 RSI: 00000000000006d8 RDI: 00000000e2d161d8
[1389863.138023] RBP: 00000000e2d171d8 R08: 0000000000004000 R09: 0000000000000800
[1389863.138023] R10: 0000000020d56ff0 R11: 0000000020d494f0 R12: 0000000020d50270
[1389863.138023] R13: 0000000000004000 R14: 0000000000f1ab80 R15: 00000000e2d191d8
[1389863.138023] FS:  0000000012e3b870(0063) GS:ffff880417c80000(0000) knlGS:000000005574c6c0
[1389863.138023] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.138023] CR2: 00007f7f1c03f000 CR3: 00000008a803a000 CR4: 00000000000006e0
[1389863.138023] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.138023] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.138023] Process molprop_2010_1_ (pid: 19430, threadinfo ffff88031879a000, task ffff88040cf0c590)
[1389863.138023] 
[1389863.138023] Call Trace:
[1389863.138026] NMI backtrace for cpu 7
[1389863.138026] CPU 7 
[1389863.138026] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.138026] 
[1389863.138026] Pid: 19429, comm: molprop_2010_1_ Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.138026] RIP: 0033:[<0000000001843903>]  [<0000000001843903>] 0x1843902
[1389863.138026] RSP: 002b:00007fff42b353c0  EFLAGS: 00000216
[1389863.138026] RAX: 00000000e29cdbe0 RBX: 00000000001fc000 RCX: 0000000000004000
[1389863.138026] RDX: 0000000000093918 RSI: 0000000000000800 RDI: 000000000020a438
[1389863.138026] RBP: 000000001faf4470 R08: 00000000e29cebe0 R09: 00000000e29cfbe0
[1389863.138026] R10: 0000000000000400 R11: 00000000000006d8 R12: 0000000000093d10
[1389863.138026] R13: 000000000020a830 R14: 00000000e29d0be0 R15: 00000000e2bcbbe0
[1389863.138026] FS:  0000000012af3870(0063) GS:ffff880817d80000(0000) knlGS:000000005574c6c0
[1389863.138026] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.138026] CR2: 0000000035155078 CR3: 00000008a816e000 CR4: 00000000000006e0
[1389863.138026] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.138026] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.138026] Process molprop_2010_1_ (pid: 19429, threadinfo ffff880318756000, task ffff88040cef9730)
[1389863.138026] 
[1389863.138026] Call Trace:
[1389863.139007] NMI backtrace for cpu 11
[1389863.139007] CPU 11 
[1389863.139007] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.139007] 
[1389863.139007] Pid: 7184, comm: classical Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.139007] RIP: 0033:[<00000000004194a6>]  [<00000000004194a6>] 0x4194a5
[1389863.139007] RSP: 002b:00007faffd895dd0  EFLAGS: 00000287
[1389863.139007] RAX: 0000000000000234 RBX: 0000000000000200 RCX: 00007fb4b9f2c060
[1389863.139007] RDX: 0000000000000223 RSI: 000000000001c342 RDI: 000000000001c343
[1389863.139007] RBP: 0000000000000200 R08: fffffffa7b7ec000 R09: 00007fb4b9f2c180
[1389863.139007] R10: 00007faf35718180 R11: 0000000000000012 R12: 00007fb4818a8010
[1389863.139007] R13: 00007faefd094010 R14: 0000000000002000 R15: 0000000000000003
[1389863.139007] FS:  00007faffd896700(0000) GS:ffff880c17d80000(0000) knlGS:000000005574c6c0
[1389863.139007] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.139007] CR2: 000000014e670000 CR3: 000000023ac52000 CR4: 00000000000006e0
[1389863.139007] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.139007] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.139007] Process classical (pid: 7184, threadinfo ffff880e9f164000, task ffff88100c34c590)
[1389863.139007] 
[1389863.139007] Call Trace:
[1389863.139006] NMI backtrace for cpu 14
[1389863.139006] CPU 14 
[1389863.139006] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.139006] 
[1389863.139006] Pid: 7183, comm: classical Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.139006] RIP: 0033:[<0000000000419417>]  [<0000000000419417>] 0x419416
[1389863.139006] RSP: 002b:00007faffe096dd0  EFLAGS: 00000206
[1389863.139006] RAX: 0000000000000410 RBX: 0000000000000200 RCX: 00007fb4b9f28060
[1389863.139006] RDX: 0000000000000221 RSI: 000000000001c340 RDI: 000000000001c341
[1389863.139006] RBP: 0000000000000200 R08: fffffffa7b7ec000 R09: 00007fb4b9f29f40
[1389863.139006] R10: 00007faf35715f50 R11: 00000000000001ef R12: 00007fb4818a8010
[1389863.139006] R13: 00007faefd094010 R14: 0000000000002000 R15: 0000000000000003
[1389863.139006] FS:  00007faffe097700(0000) GS:ffff881027d00000(0000) knlGS:000000005574c6c0
[1389863.139006] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.139006] CR2: 00007f7f0403f000 CR3: 000000023ac52000 CR4: 00000000000006e0
[1389863.139006] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.139006] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.139006] Process classical (pid: 7183, threadinfo ffff880c5ac9a000, task ffff88100c34ae60)
[1389863.139006] 
[1389863.139006] Call Trace:
[1389863.139009] NMI backtrace for cpu 13
[1389863.139009] CPU 13 
[1389863.139009] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.139009] 
[1389863.139009] Pid: 6578, comm: imag_time Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.139009] RIP: 0033:[<000000000047d57f>]  [<000000000047d57f>] 0x47d57e
[1389863.139009] RSP: 002b:00007f7f24737bd8  EFLAGS: 00000202
[1389863.139009] RAX: 00007f7f300838d0 RBX: 00007f7f300008d0 RCX: 00007f800811d810
[1389863.139009] RDX: 0000000000000080 RSI: 0000000000000080 RDI: 3ad2597e8e01a000
[1389863.139009] RBP: 0000000000100000 R08: 3d3b677b81b74100 R09: 0000000000000010
[1389863.139009] R10: 0000000000000002 R11: 0000000000000010 R12: 00007f800811d010
[1389863.139009] R13: 0000000000001060 R14: 0000000000000100 R15: 000000000047d8a0
[1389863.139009] FS:  00007f7f24738700(0000) GS:ffff881027c80000(0000) knlGS:000000005574c6c0
[1389863.139009] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.139009] CR2: 00007f7f2c03f000 CR3: 000000040b2b4000 CR4: 00000000000006f0
[1389863.139009] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.139009] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.139009] Process imag_time (pid: 6578, threadinfo ffff88040b1ee000, task ffff88007d4f4590)
[1389863.139009] 
[1389863.139009] Call Trace:
[1389863.139006] NMI backtrace for cpu 15
[1389863.139006] CPU 15 
[1389863.139006] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.139006] 
[1389863.139006] Pid: 6582, comm: imag_time Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.139006] RIP: 0033:[<000000000047d570>]  [<000000000047d570>] 0x47d56f
[1389863.139006] RSP: 002b:00007f7f22733bb8  EFLAGS: 00000202
[1389863.139006] RAX: 00007f800b998b10 RBX: 00007f800b998810 RCX: 00007f7f2c030e00
[1389863.139006] RDX: 0000000000000080 RSI: 0000000000000030 RDI: bb0926d97713a8f9
[1389863.139006] RBP: 0000000000000010 R08: bb571233afc9af43 R09: 0000000000001060
[1389863.139006] R10: 000000000000003a R11: 0000000000100000 R12: 00007f7f2c000c60
[1389863.139006] R13: 0000000000000010 R14: 0000000000000100 R15: 000000000047d820
[1389863.139006] FS:  00007f7f22734700(0000) GS:ffff881027d80000(0000) knlGS:000000005574c6c0
[1389863.139006] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.139006] CR2: 00007f7f2803f000 CR3: 000000040b2b4000 CR4: 00000000000006e0
[1389863.139006] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.139006] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.139006] Process imag_time (pid: 6582, threadinfo ffff88040c87c000, task ffff88040dc8dcc0)
[1389863.139006] 
[1389863.139006] Call Trace:
[1389863.139009] NMI backtrace for cpu 12
[1389863.139009] CPU 12 
[1389863.139009] Modules linked in: virtio_net kvm_amd kvm nfs fscache auth_rpcgss nfs_acl lockd microcode nv_tco amd64_edac_mod edac_core edac_mce_amd forcedeth serio_raw k10temp i2c_nforce2 i2c_core sunrpc uinput ata_generic pata_acpi sata_nv pata_amd [last unloaded: scsi_wait_scan]
[1389863.139009] 
[1389863.139009] Pid: 7182, comm: classical Not tainted 3.1.7-1.fc16.x86_64 #1 empty empty/S4980
[1389863.139009] RIP: 0033:[<0000000000419491>]  [<0000000000419491>] 0x419490
[1389863.139009] RSP: 002b:00007faffe897dd0  EFLAGS: 00000206
[1389863.139009] RAX: 00000000000002eb RBX: 0000000000000200 RCX: 00007fb4b9f2a060
[1389863.139009] RDX: 0000000000000222 RSI: 000000000001c341 RDI: 000000000001c342
[1389863.139009] RBP: 0000000000000200 R08: fffffffa7b7ec000 R09: 00007fb4b9f2acf0
[1389863.139009] R10: 00007faf35716d00 R11: 00000000000000ca R12: 00007fb4818a8010
[1389863.139009] R13: 00007faefd094010 R14: 0000000000002000 R15: 0000000000000003
[1389863.139009] FS:  00007faffe898700(0000) GS:ffff881027c00000(0000) knlGS:000000005574c6c0
[1389863.139009] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1389863.139009] CR2: 00007f7f3003f000 CR3: 000000023ac52000 CR4: 00000000000006e0
[1389863.139009] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[1389863.139009] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[1389863.139009] Process classical (pid: 7182, threadinfo ffff880c5accc000, task ffff881008fddcc0)
[1389863.139009] 
[1389863.139009] Call Trace:
Comment 1 Stanislaw Gruszka 2012-02-03 20:00:29 UTC 
Please install kernel-debug, it should print some more useful information where the problem is.
Comment 2 Dave Jones 2012-03-22 16:54:16 UTC 
[mass update]
kernel-3.3.0-4.fc16 has been pushed to the Fedora 16 stable repository.
Please retest with this update.
Comment 3 Dave Jones 2012-03-22 16:57:52 UTC 
[mass update]
kernel-3.3.0-4.fc16 has been pushed to the Fedora 16 stable repository.
Please retest with this update.
Comment 4 Dave Jones 2012-03-22 17:09:08 UTC 
[mass update]
kernel-3.3.0-4.fc16 has been pushed to the Fedora 16 stable repository.
Please retest with this update.