Description of problem: Box oopsed and rebooted today, I found following Oops report in logs: Unable to handle kernel paging request at virtual address a8c0527a printing eip: c0147ec2 *pde = 00000000 Oops: 0000 [#1] SMP Modules linked in: loop ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc tun md5 ipv6 ipt_state iptable_filter ipt_MASQUERADE iptable_nat ip_conntrack ipt_MARK iptable_mangle ip_tables microcode video button battery ac uhci_hcd tulip e1000 e100 mii floppy sg reiserfs raid1 dm_mod aic79xx sd_mod scsi_mod CPU: 1 EIP: 0060:[put_page+2/117] Not tainted VLI EIP: 0060:[<c0147ec2>] Not tainted VLI EFLAGS: 00010206 (2.6.11-1.27_FC3smp) EIP is at put_page+0x2/0x75 eax: a8c0527a ebx: 0000006a ecx: d71affff edx: a8c0527a esi: f4ef4b80 edi: f4ef4b80 ebp: f6f28800 esp: c03f4dbc ds: 007b es: 007b ss: 0068 Process squid (pid: 9598, threadinfo=c03f4000 task=f4bce020) Stack: c02998d4 1a97d800 f6f28e80 c029ebca 0000007e d71affff f7476000 f4ef4b80 fffffff4 c029edb5 f7476000 00000292 f91e505c f1ca3314 f1ca332c dc5ab01e f4ef4b80 c02bab8e 00000004 f1ca3300 00000000 00000004 00000002 c02baac2 Call Trace: [skb_release_data+57/111] skb_release_data+0x39/0x6f [<c02998d4>] skb_release_data+0x39/0x6f [__skb_linearize+198/276] __skb_linearize+0xc6/0x114 [<c029ebca>] __skb_linearize+0xc6/0x114 [dev_queue_xmit+413/686] dev_queue_xmit+0x19d/0x2ae [<c029edb5>] dev_queue_xmit+0x19d/0x2ae [ip_finish_output2+204/528] ip_finish_output2+0xcc/0x210 [<c02bab8e>] ip_finish_output2+0xcc/0x210 [ip_finish_output2+0/528] ip_finish_output2+0x0/0x210 [<c02baac2>] ip_finish_output2+0x0/0x210 [nf_hook_slow+221/253] nf_hook_slow+0xdd/0xfd [<c02a7fd4>] nf_hook_slow+0xdd/0xfd [ip_finish_output2+0/528] ip_finish_output2+0x0/0x210 [<c02baac2>] ip_finish_output2+0x0/0x210 [ip_forward_finish+0/75] ip_forward_finish+0x0/0x4b [<c02b9445>] ip_forward_finish+0x0/0x4b [ip_finish_output+290/605] ip_finish_output+0x122/0x25d [<c02ba987>] ip_finish_output+0x122/0x25d [ip_finish_output2+0/528] ip_finish_output2+0x0/0x210 [<c02baac2>] ip_finish_output2+0x0/0x210 [ip_forward_finish+0/75] ip_forward_finish+0x0/0x4b [<c02b9445>] ip_forward_finish+0x0/0x4b [ip_forward_finish+0/75] ip_forward_finish+0x0/0x4b [<c02b9445>] ip_forward_finish+0x0/0x4b [ip_forward_finish+42/75] ip_forward_finish+0x2a/0x4b [<c02b946f>] ip_forward_finish+0x2a/0x4b [nf_hook_slow+221/253] nf_hook_slow+0xdd/0xfd [<c02a7fd4>] nf_hook_slow+0xdd/0xfd [ip_forward_finish+0/75] ip_forward_finish+0x0/0x4b The box itself is dual-P4 Xeon 3GHz with 2G RAM. it does some forwarding between 5 network interfaces, there around 400 iptables rules (including various forms of NAT, stateful drop/accept stuff etc.) How reproducible: So far I have only this one oops, but with this same kernel there was a hang some time ago, that did not left any traces, not even on serial console.
this should be fixed in the 2.6.11-1.35_FC3 kernel currently in updates-testing. (Should be going to updates-proper tomorrow hopefully).
It oopsed yesterday with very similar backtrace with 2.6.11-1.35. I booted into latest errata kernel for FC3 now (2.6.12-1.1376_FC3) to see if that one shows same problem. The box is the same as with previous oops. Unable to handle kernel paging request at virtual address 00e2ffff printing eip: c0147e32 *pde = 00000000 Oops: 0000 [#1] SMP Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic sl hc tun md5 ipv6 ipt_state iptable_filter ipt_MASQUERADE iptable_nat ip_conntrack ipt_MARK iptable_mangle ip_tables microcode video button battery ac uhci_hcd tu lip e1000 e100 mii floppy sg reiserfs raid1 dm_mod aic79xx sd_mod scsi_mod CPU: 2 EIP: 0060:[<c0147e32>] Not tainted VLI EFLAGS: 00010202 (2.6.11-1.35_FC3smp) EIP is at put_page+0x2/0x75 eax: 00e2ffff ebx: 00000001 ecx: 00000000 edx: 00e2ffff esi: cf6b3280 edi: cf6b3280 ebp: d3b7e000 esp: c03f6dbc ds: 007b es: 007b ss: 0068 Process swapper (pid: 0, threadinfo=c03f6000 task=f7c12020) Stack: c0299ae4 f553a800 d3b7e680 c029edda 00000085 00e2ffff f785b000 cf6b3280 fffffff4 c029efc5 f785b000 00000292 f91e505c d9733414 d973342c de64381e cf6b3280 c02bad9e 00000004 d9733400 00000000 00000004 00000002 c02bacd2^MCall Trace: [<c0299ae4>] skb_release_data+0x39/0x6f [<c029edda>] __skb_linearize+0xc6/0x114 [<c029efc5>] dev_queue_xmit+0x19d/0x2ae [<c02bad9e>] ip_finish_output2+0xcc/0x210 [<c02bacd2>] ip_finish_output2+0x0/0x210 [<c02a81e4>] nf_hook_slow+0xdd/0xfd [<c02bacd2>] ip_finish_output2+0x0/0x210 [<c02b9655>] ip_forward_finish+0x0/0x4b [<c02bab97>] ip_finish_output+0x122/0x25d [<c02bacd2>] ip_finish_output2+0x0/0x210 [<c02b9655>] ip_forward_finish+0x0/0x4b [<c02b9655>] ip_forward_finish+0x0/0x4b [<c02b967f>] ip_forward_finish+0x2a/0x4b [<c02a81e4>] nf_hook_slow+0xdd/0xfd [<c02b9655>] ip_forward_finish+0x0/0x4b [<c02b95f3>] ip_forward+0x253/0x2b5 [<c02b9655>] ip_forward_finish+0x0/0x4b [<c02b8368>] ip_rcv_finish+0x179/0x2b1 [<c02a7e0b>] nf_iterate+0x52/0x9b [<c02b81ef>] ip_rcv_finish+0x0/0x2b1 [<c02b81ef>] ip_rcv_finish+0x0/0x2b1 [<c02a81e4>] nf_hook_slow+0xdd/0xfd [<c02b81ef>] ip_rcv_finish+0x0/0x2b1 [<c02b8049>] ip_rcv+0x352/0x4f8 [<c02b81ef>] ip_rcv_finish+0x0/0x2b1 [<c01470b7>] cache_flusharray+0x52/0xae [<c029f6ba>] netif_receive_skb+0x23e/0x2b1 [<c029f7ab>] process_backlog+0x7e/0x110 [<c029f8ad>] net_rx_action+0x70/0xf2 [<c0126132>] __do_softirq+0x62/0xd0 [<c0106295>] do_softirq+0x42/0x49 ======================= [<c0106187>] do_IRQ+0x57/0x89 [<c01048f6>] common_interrupt+0x1a/0x20 [<c010203d>] default_idle+0x23/0x29 [<c01020c4>] cpu_idle+0x4e/0x5c Code: 00 1b 00 e9 57 fb ff ff 8d 05 ec f8 45 c0 e8 42 00 1b 00 e9 55 fe ff ff 8d 05 ec f8 45 c0 e8 56 00 1b 00 e9 8f fe ff ff 90 89 c2 <8b> 00 f6 c4 80 75 2f 8b 02 f6 c4 08 75 27 8b 02 89 d1 f6 c4 80 <0>Kernel panic - not syncing: Fatal exception in interrupt
Happened again with 2.6.12-1.1376_FC3smp: Unable to handle kernel paging request at virtual address 29b1a040 printing eip: c014ce02 *pde = 00004001 Oops: 0000 [#1] SMP Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc tun md5 ipv6 ipt_state iptable_filter ipt_MASQUERADE iptable_nat ip_conntrack ipt_MARK iptable_mangle ip_tables microcode video button battery ac uhci_hcd tulip e1000 e100 mii floppy sg reiserfs raid1 dm_mod aic79xx sd_mod scsi_mod CPU: 3 EIP: 0060:[<c014ce02>] Not tainted VLI EFLAGS: 00010202 (2.6.12-1.1376_FC3smp) EIP is at put_page+0x2/0x75 eax: 29b1a040 ebx: 00000001 ecx: e9b1a680 edx: 29b1a040 esi: f7aa1080 edi: f7aa1080 ebp: f5647800 esp: c0406ea8 ds: 007b es: 007b ss: 0068 Process swapper (pid: 0, threadinfo=c0406000 task=c221ca80) Stack: c02a0bcb 0bb2d800 f5647e80 c02a5e90 00000079 29b1a040 f7e2a000 f7aa1080 fffffff4 c02a6075 80000000 00000286 c048cf20 f2db4114 f2db412c e9b1a01e f7aa1080 c02c21a9 00000000 f7e2a000 c02c2322 80000000 f2db4100 f7aa1080 Call Trace: [<c02a0bcb>] skb_release_data+0x50/0x83 [<c02a5e90>] __skb_linearize+0xc6/0x114 [<c02a6075>] dev_queue_xmit+0x197/0x2a8 [<c02c21a9>] ip_finish_output+0xd5/0x24e [<c02c2322>] ip_finish_output2+0x0/0x1e0 [<c02c0bfb>] ip_forward+0x16b/0x2e5 [<c02c0d75>] ip_forward_finish+0x0/0x4b [<c02bf6e7>] ip_rcv+0x350/0x551 [<c02bf8e8>] ip_rcv_finish+0x0/0x2a8 [<c02a6775>] netif_receive_skb+0x228/0x279 [<c014c099>] cache_flusharray+0x52/0xac [<c02a6851>] process_backlog+0x8b/0x11d [<c02a6965>] net_rx_action+0x82/0x175 [<c0126469>] __do_softirq+0x69/0xd5 [<c0106688>] do_softirq+0x45/0x4c ======================= [<c0106577>] do_IRQ+0x57/0x89 [<c0116953>] smp_apic_timer_interrupt+0xbd/0xc6 [<c0104a2e>] common_interrupt+0x1a/0x20 [<c010201a>] default_idle+0x0/0x29 [<c010203d>] default_idle+0x23/0x29 [<c01020d3>] cpu_idle+0x5d/0x6c Code: 55 fb ff ff 8d 05 6c e9 46 c0 e8 ea 2f 1b 00 e9 53 fe ff ff 8d 05 6c e9 46 c0 e8 fe 2f 1b 00 e9 8f fe ff ff 90 90 90 90 90 89 c2 <8b> 00 f6 c4 80 75 2f 8b 02 f6 c4 08 75 27 8b 02 89 d1 f6 c4 80 <0>Kernel panic - not syncing: Fatal exception in interrupt
Are you doing anything interesting with the "tun" device on this machine? Or is it just loaded and unused?
tun device is used by single openvpn connection with very little traffic most of the time RX bytes:6795098 (6.4 MiB) TX bytes:2257997 (2.1 MiB) for last almost 6 days.
This is a mass-update to all currently open Fedora Core 3 kernel bugs. Fedora Core 3 support has transitioned to the Fedora Legacy project. Due to the limited resources of this project, typically only updates for new security issues are released. As this bug isn't security related, it has been migrated to a Fedora Core 4 bug. Please upgrade to this newer release, and test if this bug is still present there. This bug has been placed in NEEDINFO_REPORTER state. Due to the large volume of inactive bugs in bugzilla, if this bug is still in this state in two weeks time, it will be closed. Should this bug still be relevant after this period, the reporter can reopen the bug at any time. Any other users on the Cc: list of this bug can request that the bug be reopened by adding a comment to the bug. Thank you.
This is a mass-update to all currently open kernel bugs. A new kernel update has been released (Version: 2.6.15-1.1830_FC4) based upon a new upstream kernel release. Please retest against this new kernel, as a large number of patches go into each upstream release, possibly including changes that may address this problem. This bug has been placed in NEEDINFO_REPORTER state. Due to the large volume of inactive bugs in bugzilla, if this bug is still in this state in two weeks time, it will be closed. Should this bug still be relevant after this period, the reporter can reopen the bug at any time. Any other users on the Cc: list of this bug can request that the bug be reopened by adding a comment to the bug. If this bug is a problem preventing you from installing the release this version is filed against, please see bug 169613. Thank you.
Closing per previous comment.