Bug 452692
| Summary: | crash with 2.6.24.7-65.el5rt | ||
|---|---|---|---|
| Product: | Red Hat Enterprise MRG | Reporter: | Roland Westrelin <roland.westrelin> |
| Component: | realtime-kernel | Assignee: | Steven Rostedt <srostedt> |
| Status: | CLOSED ERRATA | QA Contact: | |
| Severity: | urgent | Docs Contact: | |
| Priority: | low | ||
| Version: | 1.0 | CC: | bhu, David.Holmes, davids, williams |
| Target Milestone: | 1.0.1 | Keywords: | Reopened |
| Target Release: | --- | ||
| Hardware: | i386 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | 2.6.24.7-72.el5rt | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2008-08-26 19:57:42 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Have you seen this with the -69 kernel? It had some major rwlock changes that probably addressed this. I agree with Clark. The bug looks very similar to what the rwlock code in the latest kernel fixes. This bug can be closed. We haven't seen it with either -69 or -72. closing Need to reopen and set state to Modified for 1.0.1 release. An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2008-0585.html |
Description of problem: crash with the following message. Similar crashes happen often while running our nightly tests on our RT JVM. BUG: unable to handle kernel NULL pointer dereference at virtual address 00000003 printing eip: c0516514 *pdpt = 000000003293c001 *pde = 0000000000000000 Oops: 0002 [#1] PREEMPT SMP Modules linked in: ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge nfsd auth_rpcgss exportfs nfs lockd nfs_acl autofs4 hidp rfcomm l2cap bluetooth sunrpc iscsi_tcp ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp scsi_tgt ib_sdp ib_ipoib ipv6 rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ib_sa ib_mad ib_core nls_utf8 loop dm_mirror dm_multipath dm_mod<1>BUG: unable to handle kernel NULL pointer dereference at virtual address 00000000 printing eip: c064a1e6 *pdpt = 000000002ba73001 *pde = 0000000000000000 video output sbs sbshc battery ac parport_pc lp parport pata_amd pata_acpi sr_mod cdrom joydev sg serio_raw e1000 button k8temp i2c_nforce2 hwmon ata_generic forcedeth i2c_core pcspkr usb_storage sata_nv libata mptsas mptscsih scsi_transport_sas mptbase sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd Pid: 29567, comm: java Not tainted (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0516514>] EFLAGS: 00010082 CPU: 0 EIP is at plist_del+0x2f/0x72 EAX: 00000003 EBX: c064a1e2 ECX: e6813f10 EDX: e6813f14 ESI: e6a42d50 EDI: e6a42d4c EBP: ef17ce90 ESP: ef17ce80 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00000003 Process java (pid: 29567, ti=ef17c000 task=efcab000 task.ti=ef17c000) Stack: efcab67c f15ed3c8 f15ed3c8 e6a42d34 ef17cec4 c0451d94 f15ed3dc f15ed3c8 00000296 ef17cec4 00000000 f15ed3ec 00000002 c0649979 f15ed3c8 00000296 00000000 ef17ced4 c06499b0 f15ed3c0 e9a4c658 ef17cedc c06498cd ef17cf70 Call Trace: [<c0451d94>] ? wakeup_next_waiter+0x86/0x201 [<c0649979>] ? rt_mutex_slowunlock+0xc/0x5b [<c06499b0>] ? rt_mutex_slowunlock+0x43/0x5b [<c06498cd>] ? rt_mutex_unlock+0x8/0xa [<c0451352>] ? do_futex+0x867/0x98b [<c04688a5>] ? audit_syscall_entry+0x113/0x13d [<c045155d>] ? sys_futex+0xe7/0xfa [<c0407068>] ? do_syscall_trace+0x14c/0x198 [<c04041fe>] ? syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00000003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b62f>] .... __spin_lock_irqsave+0x1d/0x6e .....[<c0649979>] .. ( <= rt_mutex_slowunlock+0xc/0x5b) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0451d2c>] .. ( <= wakeup_next_waiter+0x1e/0x201) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0405528>] .. ( <= die+0x49/0x1f9) INFO: lockdep is turned off. Code: 89 c7 56 89 d0 53 83 ec 04 8d 77 04 89 55 f0 e8 d0 ff ff ff 39 77 04 74 33 8b 4f 0c 83 e9 0c 8d 51 04 8b 59 04 8b 42 04 89 43 04 <89> 18 8b 46 04 89 71 04 89 56 04 89 10 89 42 04 8b 57 04 8b 46 EIP: [<c0516514>] plist_del+0x2f/0x72 SS:ESP 0068:ef17ce80 ---[ end trace 10cace17a517b871 ]--- Oops: 0000 [#2] PREEMPT SMP Modules linked in: ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge nfsd auth_rpcgss exportfs nfs lockd nfs_acl autofs4 hidp rfcomm l2cap bluetooth sunrpc iscsi_tcp ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp scsi_tgt ib_sdp ib_ipoib ipv6 rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ib_sa ib_mad ib_core nls_utf8 loop dm_mirror dm_multipath dm_mod video output sbs sbshc battery ac parport_pc lp parport pata_amd pata_acpi sr_mod cdrom joydev sg serio_raw e1000 button k8temp i2c_nforce2 hwmon ata_generic forcedeth i2c_core pcspkr usb_storage sata_nv libata mptsas mptscsih scsi_transport_sas mptbase sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd Pid: 47, comm: sirq-net-rx/3 Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c064a1e6>] EFLAGS: 00010286 CPU: 3 EIP is at rt_write_slowunlock+0x28b/0x28f EAX: 00000000 EBX: f8ccff80 ECX: f78fa000 EDX: 00000000 ESI: 00000000 EDI: f4418980 EBP: f78fad18 ESP: f78fad10 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 preempt:00000001 Process sirq-net-rx/3 (pid: 47, ti=f78fa000 task=f78f9000 task.ti=f78fa000) Stack: f14a483c f4418980 f78fad34 c0451b91 f8cc4c66 00000246 f8ccff80 f14a483c f8ccff80 f78fad40 c064af30 00009510 f78fad64 f8cc4c66 f4418980 00000002 00000008 c0451c3e f14a483c f4418980 f4418980 f78fad84 f8cc883b 00007530 Call Trace: [<c0451b91>] ? rt_rwlock_write_unlock+0x4b/0x51 [<f8cc4c66>] ? __nf_ct_refresh_acct+0xf1/0x150 [nf_conntrack] [<c064af30>] ? rt_write_unlock+0x1d/0x20 [<f8cc4c66>] ? __nf_ct_refresh_acct+0xf1/0x150 [nf_conntrack] [<c0451c3e>] ? rt_rwlock_read_unlock+0xa7/0xad [<f8cc883b>] ? udp_packet+0xa5/0xb2 [nf_conntrack] [<f8cc8796>] ? udp_packet+0x0/0xb2 [nf_conntrack] [<f8cc587b>] ? nf_conntrack_in+0x385/0x42e [nf_conntrack] [<c04279d5>] ? add_preempt_count+0x1b/0xfc [<c061dba6>] ? udp_queue_rcv_skb+0x1bb/0x25d [<f8c2c1d8>] ? ipv4_conntrack_in+0x0/0x13 [nf_conntrack_ipv4] [<f8c2c1e9>] ? ipv4_conntrack_in+0x11/0x13 [nf_conntrack_ipv4] [<c05fa964>] ? nf_iterate+0x38/0x6b [<c05ffb98>] ? ip_rcv_finish+0x0/0x310 [<c05faa3b>] ? nf_hook_slow+0x52/0xbe [<c05ffb98>] ? ip_rcv_finish+0x0/0x310 [<c0600347>] ? ip_rcv+0x205/0x23b [<c05ffb98>] ? ip_rcv_finish+0x0/0x310 [<c0600142>] ? ip_rcv+0x0/0x23b [<c05e169b>] ? netif_receive_skb+0x37e/0x418 [<f897692e>] ? e1000_clean_rx_irq+0x385/0x43f [e1000] [<f89765a9>] ? e1000_clean_rx_irq+0x0/0x43f [e1000] [<f8974008>] ? e1000_clean+0x66/0x219 [e1000] [<c05e38bc>] ? net_rx_action+0x9f/0x1c9 [<c04279d5>] ? add_preempt_count+0x1b/0xfc [<c04337b3>] ? ksoftirqd+0x16d/0x266 [<c0433646>] ? ksoftirqd+0x0/0x266 [<c044086b>] ? kthread+0x3b/0x64 [<c0440830>] ? kthread+0x0/0x64 [<c0404e4b>] ? kernel_thread_helper+0x7/0x10 ======================= --------------------------- | preempt count: 00000001 ] | 1-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0405528>] .. ( <= die+0x49/0x1f9) INFO: lockdep is turned off. Code: 45 d8 c7 40 64 8c 00 00 00 8b 45 dc e8 cd 17 00 00 8b 45 e0 8b 55 ec e8 79 18 00 00 64 a1 00 40 7d c0 e8 6d 82 e0 ff 83 c4 2c 5b <03> 00 00 00 55 89 e5 57 89 c7 56 53 83 ec 20 8d 40 04 89 45 e8 EIP: [<c064a1e6>] rt_write_slowunlock+0x28b/0x28f SS:ESP 0068:f78fad10 ---[ end trace 10cace17a517b871 ]--- BUG: unable to handle kernel NULL pointer dereference at virtual address 00000000 printing eip: c064a1e6 *pdpt = 0000000000825001 *pde = 0000000000000000 Oops: 0000 [#3] PREEMPT SMP Modules linked in: ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge nfsd auth_rpcgss exportfs nfs lockd nfs_acl autofs4 hidp rfcomm l2cap bluetooth sunrpc iscsi_tcp ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp scsi_tgt ib_sdp ib_ipoib ipv6 rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ib_sa ib_mad ib_core nls_utf8 loop dm_mirror dm_multipath dm_mod video output sbs sbshc battery ac parport_pc lp parport pata_amd pata_acpi sr_mod cdrom joydev sg serio_raw e1000 button k8temp i2c_nforce2 hwmon ata_generic forcedeth i2c_core pcspkr usb_storage sata_nv libata mptsas mptscsih scsi_transport_sas mptbase sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd Pid: 47, comm: sirq-net-rx/3 Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c064a1e6>] EFLAGS: 00010296 CPU: 3 EIP is at rt_write_slowunlock+0x28b/0x28f EAX: 00000000 EBX: c077b380 ECX: f78fa000 EDX: 00000000 ESI: 00000000 EDI: f78f8ff8 EBP: f78fabc4 ESP: f78fabbc DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 preempt:00000001 Process sirq-net-rx/3 (pid: 47, ti=f78fa000 task=f78f9000 task.ti=f78fa000) Stack: f78f9000 f78f8ff8 f78fabe0 c0451b91 c0431735 00000246 c077b380 f78f9000 c077b380 f78fabec c064af30 f78f8ff8 f78fac24 c0431735 f78fad10 c06e0068 f78fac10 c042ed83 f7855000 f78f919c f78fac1c f78fac10 f78fac10 f78facd8 Call Trace: [<c0451b91>] ? rt_rwlock_write_unlock+0x4b/0x51 [<c0431735>] ? do_exit+0x4f7/0x76f [<c064af30>] ? rt_write_unlock+0x1d/0x20 [<c0431735>] ? do_exit+0x4f7/0x76f [<c042ed83>] ? printk+0x1b/0x1d [<c04056d0>] ? die+0x1f1/0x1f9 [<c064ddd9>] ? do_page_fault+0x724/0x80a [<c0649f74>] ? rt_write_slowunlock+0x19/0x28f [<c064d6b5>] ? do_page_fault+0x0/0x80a [<c064c11a>] ? error_code+0x72/0x78 [<c064a1e6>] ? rt_write_slowunlock+0x28b/0x28f [<c0451b91>] ? rt_rwlock_write_unlock+0x4b/0x51 [<f8cc4c66>] ? __nf_ct_refresh_acct+0xf1/0x150 [nf_conntrack] [<c064af30>] ? rt_write_unlock+0x1d/0x20 [<f8cc4c66>] ? __nf_ct_refresh_acct+0xf1/0x150 [nf_conntrack] [<c0451c3e>] ? rt_rwlock_read_unlock+0xa7/0xad [<f8cc883b>] ? udp_packet+0xa5/0xb2 [nf_conntrack] [<f8cc8796>] ? udp_packet+0x0/0xb2 [nf_conntrack] [<f8cc587b>] ? nf_conntrack_in+0x385/0x42e [nf_conntrack] [<c04279d5>] ? add_preempt_count+0x1b/0xfc [<c061dba6>] ? udp_queue_rcv_skb+0x1bb/0x25d [<f8c2c1d8>] ? ipv4_conntrack_in+0x0/0x13 [nf_conntrack_ipv4] [<f8c2c1e9>] ? ipv4_conntrack_in+0x11/0x13 [nf_conntrack_ipv4] [<c05fa964>] ? nf_iterate+0x38/0x6b [<c05ffb98>] ? ip_rcv_finish+0x0/0x310 [<c05faa3b>] ? nf_hook_slow+0x52/0xbe [<c05ffb98>] ? ip_rcv_finish+0x0/0x310 [<c0600347>] ? ip_rcv+0x205/0x23b [<c05ffb98>] ? ip_rcv_finish+0x0/0x310 [<c0600142>] ? ip_rcv+0x0/0x23b [<c05e169b>] ? netif_receive_skb+0x37e/0x418 [<f897692e>] ? e1000_clean_rx_irq+0x385/0x43f [e1000] [<f89765a9>] ? e1000_clean_rx_irq+0x0/0x43f [e1000] [<f8974008>] ? e1000_clean+0x66/0x219 [e1000] [<c05e38bc>] ? net_rx_action+0x9f/0x1c9 [<c04279d5>] ? add_preempt_count+0x1b/0xfc [<c04337b3>] ? ksoftirqd+0x16d/0x266 [<c0433646>] ? ksoftirqd+0x0/0x266 [<c044086b>] ? kthread+0x3b/0x64 [<c0440830>] ? kthread+0x0/0x64 [<c0404e4b>] ? kernel_thread_helper+0x7/0x10 ======================= --------------------------- | preempt count: 00000001 ] | 1-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0405528>] .. ( <= die+0x49/0x1f9) INFO: lockdep is turned off. Code: 45 d8 c7 40 64 8c 00 00 00 8b 45 dc e8 cd 17 00 00 8b 45 e0 8b 55 ec e8 79 18 00 00 64 a1 00 40 7d c0 e8 6d 82 e0 ff 83 c4 2c 5b <03> 00 00 00 55 89 e5 57 89 c7 56 53 83 ec 20 8d 40 04 89 45 e8 EIP: [<c064a1e6>] rt_write_slowunlock+0x28b/0x28f SS:ESP 0068:f78fabbc ---[ end trace 10cace17a517b871 ]--- Fixing recursive fault but reboot is needed! BUG: unable to handle kernel NULL pointer dereference at virtual address 00000040 printing eip: c064a1e6 *pdpt = 00000000344fe001 *pde = 0000000000000000 Oops: 0000 [#4] PREEMPT SMP Modules linked in: ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge nfsd auth_rpcgss exportfs nfs lockd nfs_acl autofs4 hidp rfcomm l2cap bluetooth sunrpc iscsi_tcp ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp scsi_tgt ib_sdp ib_ipoib ipv6 rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ib_sa ib_mad ib_core nls_utf8 loop dm_mirror dm_multipath dm_mod video output sbs sbshc battery ac parport_pc lp parport pata_amd pata_acpi sr_mod cdrom joydev sg serio_raw e1000 button k8temp i2c_nforce2 hwmon ata_generic forcedeth i2c_core pcspkr usb_storage sata_nv libata mptsas mptscsih scsi_transport_sas mptbase sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd Pid: 5225, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c064a1e6>] EFLAGS: 00010296 CPU: 3 EIP is at rt_write_slowunlock+0x28b/0x28f EAX: 00000040 EBX: c077b380 ECX: f309d000 EDX: 00000000 ESI: 00000000 EDI: e6883f68 EBP: f309de84 ESP: f309de7c DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00000001 Process automount (pid: 5225, ti=f309d000 task=f41bd000 task.ti=f309d000) Stack: fffffff4 e6883f68 f309dea0 c0451b91 c042d2aa 00000246 c077b380 fffffff4 c077b380 f309deac c064af30 e6883000 f309df00 c042d2aa 00000000 e6883000 f309dfb8 c0425c3c f309dfb8 b31e04b4 003d0f00 e6883000 e6883000 00010000 Call Trace: [<c0451b91>] ? rt_rwlock_write_unlock+0x4b/0x51 [<c042d2aa>] ? copy_process+0x12bb/0x1459 [<c064af30>] ? rt_write_unlock+0x1d/0x20 [<c042d2aa>] ? copy_process+0x12bb/0x1459 [<c0425c3c>] ? default_wake_function+0x0/0x1a [<c0450b3f>] ? do_futex+0x54/0x98b [<c042d650>] ? do_fork+0x7a/0x1e1 [<c041b528>] ? smp_apic_timer_interrupt+0xb7/0xc5 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c04688a5>] ? audit_syscall_entry+0x113/0x13d [<c045155d>] ? sys_futex+0xe7/0xfa [<c0407068>] ? do_syscall_trace+0x14c/0x198 [<c0402252>] ? sys_clone+0x33/0x39 [<c04041fe>] ? syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00000001 ] | 1-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0405528>] .. ( <= die+0x49/0x1f9) INFO: lockdep is turned off. Code: 45 d8 c7 40 64 8c 00 00 00 8b 45 dc e8 cd 17 00 00 8b 45 e0 8b 55 ec e8 79 18 00 00 64 a1 00 40 7d c0 e8 6d 82 e0 ff 83 c4 2c 5b <03> 00 00 00 55 89 e5 57 89 c7 56 53 83 ec 20 8d 40 04 89 45 e8 EIP: [<c064a1e6>] rt_write_slowunlock+0x28b/0x28f SS:ESP 0068:f309de7c ---[ end trace 10cace17a517b871 ]--- BUG: unable to handle kernel NULL pointer dereference at virtual address 00000040 printing eip: c064a1e6 *pdpt = 00000000344fe001 *pde = 0000000000000000 Oops: 0000 [#5] PREEMPT SMP Modules linked in: ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge nfsd auth_rpcgss exportfs nfs lockd nfs_acl autofs4 hidp rfcomm l2cap bluetooth sunrpc iscsi_tcp ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp scsi_tgt ib_sdp ib_ipoib ipv6 rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ib_sa ib_mad ib_core nls_utf8 loop dm_mirror dm_multipath dm_mod video output sbs sbshc battery ac parport_pc lp parport pata_amd pata_acpi sr_mod cdrom joydev sg serio_raw e1000 button k8temp i2c_nforce2 hwmon ata_generic forcedeth i2c_core pcspkr usb_storage sata_nv libata mptsas mptscsih scsi_transport_sas mptbase sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd Pid: 5225, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c064a1e6>] EFLAGS: 00010296 CPU: 3 EIP is at rt_write_slowunlock+0x28b/0x28f EAX: 00000040 EBX: c077b380 ECX: f309d000 EDX: 00000000 ESI: 00000000 EDI: f41bcff8 EBP: f309dd30 ESP: f309dd28 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 preempt:00000001 Process automount (pid: 5225, ti=f309d000 task=f41bd000 task.ti=f309d000) Stack: f41bd000 f41bcff8 f309dd4c c0451b91 c0431735 00000246 c077b380 f41bd000 c077b380 f309dd58 c064af30 f41bcff8 f309dd90 c0431735 f309de7c c06e0068 f309dd7c c042ed83 f45f8000 f41bd19c f309dd88 f309dd7c f309dd7c f309de44 Call Trace: [<c0451b91>] ? rt_rwlock_write_unlock+0x4b/0x51 [<c0431735>] ? do_exit+0x4f7/0x76f [<c064af30>] ? rt_write_unlock+0x1d/0x20 [<c0431735>] ? do_exit+0x4f7/0x76f [<c042ed83>] ? printk+0x1b/0x1d [<c04056d0>] ? die+0x1f1/0x1f9 [<c064ddd9>] ? do_page_fault+0x724/0x80a [<c0649f74>] ? rt_write_slowunlock+0x19/0x28f [<c064d6b5>] ? do_page_fault+0x0/0x80a [<c064c11a>] ? error_code+0x72/0x78 [<c064a1e6>] ? rt_write_slowunlock+0x28b/0x28f [<c0451b91>] ? rt_rwlock_write_unlock+0x4b/0x51 [<c042d2aa>] ? copy_process+0x12bb/0x1459 [<c064af30>] ? rt_write_unlock+0x1d/0x20 [<c042d2aa>] ? copy_process+0x12bb/0x1459 [<c0425c3c>] ? default_wake_function+0x0/0x1a [<c0450b3f>] ? do_futex+0x54/0x98b [<c042d650>] ? do_fork+0x7a/0x1e1 [<c041b528>] ? smp_apic_timer_interrupt+0xb7/0xc5 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c04688a5>] ? audit_syscall_entry+0x113/0x13d [<c045155d>] ? sys_futex+0xe7/0xfa [<c0407068>] ? do_syscall_trace+0x14c/0x198 [<c0402252>] ? sys_clone+0x33/0x39 [<c04041fe>] ? syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00000001 ] | 1-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0405528>] .. ( <= die+0x49/0x1f9) INFO: lockdep is turned off. Code: 45 d8 c7 40 64 8c 00 00 00 8b 45 dc e8 cd 17 00 00 8b 45 e0 8b 55 ec e8 79 18 00 00 64 a1 00 40 7d c0 e8 6d 82 e0 ff 83 c4 2c 5b <03> 00 00 00 55 89 e5 57 89 c7 56 53 83 ec 20 8d 40 04 89 45 e8 EIP: [<c064a1e6>] rt_write_slowunlock+0x28b/0x28f SS:ESP 0068:f309dd28 ---[ end trace 10cace17a517b871 ]--- Fixing recursive fault but reboot is needed! BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fbd>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xfa/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fbd>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xfa/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fbd>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xfa/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fbd>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xfa/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) BUG: spinlock lockup on CPU#2, java/29592, efcab658 Pid: 29592, comm: java Tainted: G D 2.6.24.7-65.el5rtdebug #1 [<c064bedb>] _raw_spin_lock+0xe8/0x109 [<c064b26f>] __spin_lock+0x54/0x62 [<c0452fda>] ? task_blocks_on_rt_mutex+0x1a4/0x22b [<c0452fda>] task_blocks_on_rt_mutex+0x1a4/0x22b [<c0649aa7>] rt_spin_lock_slowlock+0xae/0x1cf [<c064aacc>] __rt_spin_lock+0x44/0x47 [<c064ac35>] rt_spin_lock+0x4a/0x58 [<c045033f>] ? futex_lock_pi+0x136/0x8e2 [<c045033f>] futex_lock_pi+0x136/0x8e2 [<c044a901>] ? trace_hardirqs_off+0xb/0xd [<c0418c76>] ? send_IPI_mask_bitmask+0x96/0xa7 [<c0425777>] ? task_rq_lock+0x36/0x5d [<c064ba89>] ? __spin_unlock_irqrestore+0x39/0x59 [<c0452464>] ? rt_mutex_adjust_prio+0x15/0x2c [<c0451462>] do_futex+0x977/0x98b [<c064a1e2>] ? rt_write_slowunlock+0x287/0x28f [<c04688a5>] ? audit_syscall_entry+0x113/0x13d [<c045155d>] sys_futex+0xe7/0xfa [<c0407068>] ? do_syscall_trace+0x14c/0x198 [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00000002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [<c064b62f>] .... __spin_lock_irqsave+0x1d/0x6e .....[<c0649a21>] .. ( <= rt_spin_lock_slowlock+0x28/0x1cf) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0452fda>] .. ( <= task_blocks_on_rt_mutex+0x1a4/0x22b) BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fbd>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xfa/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fc0>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xfd/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fbb>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xf8/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fc0>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xfd/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fc0>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xfd/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) BUG: soft lockup - CPU#3 stuck for 11s! [automount:5224] Pid: 5224, comm: automount Tainted: G D (2.6.24.7-65.el5rtdebug #1) EIP: 0060:[<c0418fc0>] EFLAGS: 00000293 CPU: 3 EIP is at native_smp_call_function_mask+0xfd/0x120 EAX: c0418c7d EBX: 00000007 ECX: 00000001 EDX: ffffb300 ESI: 00000003 EDI: 00000001 EBP: f417fd80 ESP: f417fd40 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 preempt:00010003 CR0: 8005003b CR2: aae58000 CR3: 343b1000 CR4: 000006f0 DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 DR6: ffff0ff0 DR7: 00000400 [<c044ca78>] ? trace_hardirqs_on+0xb/0xd [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c0419126>] ? do_flush_tlb_all+0x0/0x81 [<c041a521>] smp_call_function+0x1e/0x22 [<c0432ac0>] on_each_cpu+0x2a/0x61 [<c0418bde>] flush_tlb_all+0x1e/0x20 [<c0485777>] kmap_high+0x29e/0x463 [<c0420dbd>] kmap+0x4e/0x5a [<c04a096e>] pipe_write+0x23a/0x3d9 [<c045495a>] ? rt_up_read+0x1d/0x20 [<c049a4c5>] do_sync_write+0xc5/0x102 [<c044092f>] ? autoremove_wake_function+0x0/0x38 [<c0445ff0>] ? getnstimeofday+0x34/0xc4 [<c049a400>] ? do_sync_write+0x0/0x102 [<c049ace5>] vfs_write+0xab/0x162 [<c049b3a7>] sys_write+0x43/0x6d [<c04041fe>] syscall_call+0x7/0xb ======================= --------------------------- | preempt count: 00010003 ] | 3-level deep critical section nesting: ---------------------------------------- .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c0425777>] .. ( <= task_rq_lock+0x36/0x5d) .. [<c05132d3>] .... delay_tsc+0x15/0x86 .....[<c0513229>] .. ( <= __delay+0x9/0xb) .. [<c064b22f>] .... __spin_lock+0x14/0x62 .....[<c046b626>] .. ( <= softlockup_tick+0xc4/0x11b) Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: