Bug 1018906

Summary: BUG: unable to handle kernel paging request
Product: [Fedora] Fedora Reporter: Ldap Tester <ldap.tester>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 19CC: embedded_linux520, gansalmon, itamar, jonathan, kernel-maint, madhu.chinakonda, marcelo.barbosa, michele, wmealing
Target Milestone: ---Flags: wmealing: needinfo? (embedded_linux520)
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-23 14:41:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Ldap Tester 2013-10-14 16:31:37 UTC
I have been getting kernel panics, infrequently and randomly, since the end of August with messages like these:

Oct 10 12:11:22 foobar kernel: [798927.004470] BUG: unable to handle kernel paging request at 0000000100000000
Oct 10 12:11:22 foobar kernel: [798927.004716] IP: [<ffffffff8118ef53>] __kmalloc+0x93/0x250
Oct 10 12:11:22 foobar kernel: [798927.004895] PGD 1246d3067 PUD 0 
Oct 10 12:11:22 foobar kernel: [798927.004999] Oops: 0000 [#1] SMP 
Oct 10 12:11:22 foobar kernel: [798927.005079] Modules linked in: xt_limit xt_TCPMSS nf_nat_irc nf_nat_ftp xt_conntrack iptable_nat nf_nat_ipv4 iptable_mangle xt_LOG ipt_MASQUERADE xt_DSCP xt_REDIRECT nf_nat nf_conntrack_irc nf_conntrack_ftp nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd sunrpc fscache snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev snd_page_alloc i2c_piix4 snd_timer snd i2c_core e1000 microcode serio_raw parport_pc parport soundcore mperf binfmt_misc uinput ata_generic pata_acpi
Oct 10 12:11:22 foobar kernel: [798927.005079] CPU: 3 PID: 27933 Comm: nc Not tainted 3.11.2-201.fc19.x86_64 #1
Oct 10 12:11:22 foobar kernel: [798927.005079] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Oct 10 12:11:22 foobar kernel: [798927.005079] task: ffff8801ea6f44a0 ti: ffff880126d44000 task.ti: ffff880126d44000
Oct 10 12:11:22 foobar kernel: [798927.005079] RIP: 0010:[<ffffffff8118ef53>]  [<ffffffff8118ef53>] __kmalloc+0x93/0x250
Oct 10 12:11:22 foobar kernel: [798927.005079] RSP: 0018:ffff880126d45a48  EFLAGS: 00010246
Oct 10 12:11:22 foobar kernel: [798927.005079] RAX: 0000000000000000 RBX: ffffffff81cbdfc0 RCX: 0000000000008020
Oct 10 12:11:22 foobar kernel: [798927.005079] RDX: 0000000004d9cbbe RSI: 0000000000000000 RDI: 0000000000000015
Oct 10 12:11:22 foobar kernel: [798927.005079] RBP: ffff880126d45a88 R08: 0000000000016f40 R09: ffffffffa02240b3
Oct 10 12:11:22 foobar kernel: [798927.005079] R10: ffff880217001800 R11: 00000000ea791d3f R12: 0000000000008020
Oct 10 12:11:22 foobar kernel: [798927.005079] R13: 0000000100000000 R14: 00000000000000b0 R15: ffff880217001800
Oct 10 12:11:22 foobar kernel: [798927.005079] FS:  00007fb71edb7840(0000) GS:ffff88021fd80000(0000) knlGS:0000000000000000
Oct 10 12:11:22 foobar kernel: [798927.005079] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 10 12:11:22 foobar kernel: [798927.005079] CR2: 0000000100000000 CR3: 00000001f0b3e000 CR4: 00000000000006e0
Oct 10 12:11:22 foobar kernel: [798927.005079] Stack:
Oct 10 12:11:22 foobar kernel: [798927.005079]  00000000f17463f0 ffff880103878914 ffff88012b3daaf0 ffffffff81cbdfc0
Oct 10 12:11:22 foobar kernel: [798927.005079]  0000000000000080 0000000000000018 0000000000000000 ffff8801ee390fd8
Oct 10 12:11:22 foobar kernel: [798927.005079]  ffff880126d45ad0 ffffffffa02240b3 ffff8801ee390fd8 0000000000000003
Oct 10 12:11:22 foobar kernel: [798927.005079] Call Trace:
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffffa02240b3>] __nf_ct_ext_add_length+0x163/0x1a0 [nf_conntrack]
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffffa021d9e0>] nf_conntrack_in+0x6f0/0xa00 [nf_conntrack]
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffffa018378d>] ipv4_conntrack_local+0x4d/0x50 [nf_conntrack_ipv4]
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81579ebb>] nf_iterate+0x8b/0xa0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81586050>] ? ip_forward_options+0x1c0/0x1c0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81579f44>] nf_hook_slow+0x74/0x130
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81586050>] ? ip_forward_options+0x1c0/0x1c0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81587f12>] __ip_local_out+0xb2/0xc0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81535e32>] ? __sk_dst_check+0x42/0x90
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81587f32>] ip_local_out+0x12/0x30
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff8158829c>] ip_queue_xmit+0x14c/0x3f0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff8159f299>] tcp_transmit_skb+0x479/0x8e0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff815a1e46>] tcp_connect+0x4d6/0x5b0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff810b8a4e>] ? getnstimeofday+0xe/0x30
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff810b8ad6>] ? ktime_get_real+0x16/0x50
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff815a5411>] tcp_v4_connect+0x321/0x480
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff815b9695>] __inet_stream_connect+0xa5/0x320
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff815a29f0>] ? tcp_delack_timer_handler+0x200/0x200
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff8158e04e>] ? inet_csk_init_xmit_timers+0x6e/0xa0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff815b9948>] inet_stream_connect+0x38/0x50
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81533f67>] SYSC_connect+0xe7/0x120
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff811a9e9e>] ? alloc_file+0x1e/0xc0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff815311dc>] ? sock_alloc_file+0x9c/0x130
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff811c3e60>] ? __fd_install+0x20/0x50
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff812981c6>] ? security_file_fcntl+0x16/0x20
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff811b8f8a>] ? SyS_fcntl+0x41a/0x5a0
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81534f1e>] SyS_connect+0xe/0x10
Oct 10 12:11:22 foobar kernel: [798927.005079]  [<ffffffff81656919>] system_call_fastpath+0x16/0x1b
Oct 10 12:11:22 foobar kernel: [798927.005079] Code: 25 48 dc 00 00 49 8b 50 08 49 83 78 10 00 4d 8b 28 0f 84 9d 01 00 00 4d 85 ed 0f 84 94 01 00 00 49 63 42 20 4d 8b 02 41 f6 c0 0f <49> 8b 5c 05 00 0f 85 31 01 00 00 48 8d 4a 01 4c 89 e8 49 8d 30 
Oct 10 12:11:22 foobar kernel: [798927.005079] RIP  [<ffffffff8118ef53>] __kmalloc+0x93/0x250
Oct 10 12:11:22 foobar kernel: [798927.005079]  RSP <ffff880126d45a48>
Oct 10 12:11:22 foobar kernel: [798927.005079] CR2: 0000000100000000
Oct 10 12:11:22 foobar kernel: [798927.034835] ---[ end trace 6cd98e23cc3799c1 ]---
Oct 10 12:11:22 foobar kernel: [798927.362952] BUG: unable to handle kernel paging request at 0000000100000000
Oct 10 12:11:22 foobar kernel: [798927.363272] IP: [<ffffffff8118dfc3>] kmem_cache_alloc+0x73/0x210
Oct 10 12:11:22 foobar kernel: [798927.363272] PGD 1e9318067 PUD 0 
Oct 10 12:11:22 foobar kernel: [798927.363272] Oops: 0000 [#2] SMP 
Oct 10 12:11:22 foobar kernel: [798927.363272] Modules linked in: xt_limit xt_TCPMSS nf_nat_irc nf_nat_ftp xt_conntrack iptable_nat nf_nat_ipv4 iptable_mangle xt_LOG ipt_MASQUERADE xt_DSCP xt_REDIRECT nf_nat nf_conntrack_irc nf_conntrack_ftp nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd sunrpc fscache snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm ppdev snd_page_alloc i2c_piix4 snd_timer snd i2c_core e1000 microcode serio_raw parport_pc parport soundcore mperf binfmt_misc uinput ata_generic pata_acpi
Oct 10 12:11:22 foobar kernel: [798927.363272] CPU: 3 PID: 27941 Comm: sh Tainted: G      D      3.11.2-201.fc19.x86_64 #1
Oct 10 12:11:22 foobar kernel: [798927.363272] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Oct 10 12:11:22 foobar kernel: [798927.363272] task: ffff880112375b80 ti: ffff88011c2b0000 task.ti: ffff88011c2b0000
Oct 10 12:11:22 foobar kernel: [798927.363272] RIP: 0010:[<ffffffff8118dfc3>]  [<ffffffff8118dfc3>] kmem_cache_alloc+0x73/0x210
Oct 10 12:11:22 foobar kernel: [798927.363272] RSP: 0018:ffff88011c2b1ea8  EFLAGS: 00010246
Oct 10 12:11:22 foobar kernel: [798927.363272] RAX: 0000000000000000 RBX: ffffffffffffffea RCX: ffff88011c2b1fd8
Oct 10 12:11:22 foobar kernel: [798927.363272] RDX: 0000000004d9cbc4 RSI: 00000000000000d0 RDI: ffff880217001800
Oct 10 12:11:22 foobar kernel: [798927.363272] RBP: ffff88011c2b1ee0 R08: 0000000000016f40 R09: ffffffff8108e896
Oct 10 12:11:22 foobar kernel: [798927.363272] R10: 000000000000000f R11: 0000000000000246 R12: 0000000100000000
Oct 10 12:11:22 foobar kernel: [798927.363272] R13: 00000000000000d0 R14: ffff880217001800 R15: ffff880217001800
Oct 10 12:11:22 foobar kernel: [798927.363272] FS:  00007f04f180e740(0000) GS:ffff88021fd80000(0000) knlGS:0000000000000000
Oct 10 12:11:22 foobar kernel: [798927.363272] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 10 12:11:22 foobar kernel: [798927.363272] CR2: 0000000100000000 CR3: 00000002131aa000 CR4: 00000000000006e0
Oct 10 12:11:22 foobar kernel: [798927.363272] Stack:
Oct 10 12:11:22 foobar kernel: [798927.363272]  0000000016394f6f ffffffff8108e896 ffffffffffffffea ffff880112375b80
Oct 10 12:11:22 foobar kernel: [798927.363272]  00000000012bf550 000000000000025f 000000000000025f ffff88011c2b1f00
Oct 10 12:11:22 foobar kernel: [798927.363272]  ffffffff8108e896 ffffffffffffffea 0000000000000001 ffff88011c2b1f68
Oct 10 12:11:22 foobar kernel: [798927.363272] Call Trace:
Oct 10 12:11:22 foobar kernel: [798927.363272]  [<ffffffff8108e896>] ? prepare_creds+0x26/0x1a0
Oct 10 12:11:22 foobar kernel: [798927.363272]  [<ffffffff8108e896>] prepare_creds+0x26/0x1a0
Oct 10 12:11:22 foobar kernel: [798927.363272]  [<ffffffff811a6a4d>] SyS_faccessat+0x3d/0x200
Oct 10 12:11:22 foobar kernel: [798927.363272]  [<ffffffff811a6c28>] SyS_access+0x18/0x20
Oct 10 12:11:22 foobar kernel: [798927.363272]  [<ffffffff81656919>] system_call_fastpath+0x16/0x1b
Oct 10 12:11:22 foobar kernel: [798927.363272] Code: 25 48 dc 00 00 49 8b 50 08 49 83 78 10 00 4d 8b 20 0f 84 2e 01 00 00 4d 85 e4 0f 84 25 01 00 00 49 63 46 20 4d 8b 06 41 f6 c0 0f <49> 8b 1c 04 0f 85 4b 01 00 00 48 8d 4a 01 4c 89 e0 49 8d 30 e8 
Oct 10 12:11:22 foobar kernel: [798927.363272] RIP  [<ffffffff8118dfc3>] kmem_cache_alloc+0x73/0x210
Oct 10 12:11:22 foobar kernel: [798927.363272]  RSP <ffff88011c2b1ea8>
Oct 10 12:11:22 foobar kernel: [798927.363272] CR2: 0000000100000000
Oct 10 12:11:22 foobar kernel: [798927.407917] ---[ end trace 6cd98e23cc3799c2 ]---

This has happened with several 3.10.x kernels and now with 3.11.2-201.fc19.x86_64.  The failing system is Fedora 19 with all updates applied daily.  The system is a guest machine of VirtualBox-4.2 (from virtualbox.org).  This problem has happened under the last two versions of VirtualBox, currently at 4.2.18_88780.  The host machine is also Fedora 19 with all updates applied daily.  The problem has happened with 3.10.x and 3.11.x kernels running on the host as well.  It is not a hardware problem, as I have seen it on several identically configured boxes. I have been running this configuration for two years with no problem until recently.

Comment 1 Josh Boyer 2013-10-14 17:16:46 UTC
Does it only happen in VirtualBox?  Can you see if you can recreate under KVM instead?

Comment 2 Ldap Tester 2013-10-14 18:58:56 UTC
(In reply to Josh Boyer from comment #1)
> Does it only happen in VirtualBox?  Can you see if you can recreate under
> KVM instead?

Sorry, I cannot perform that test.

Comment 3 Justin M. Forbes 2014-01-03 22:09:22 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 19 kernel bugs.

Fedora 19 has now been rebased to 3.12.6-200.fc19.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 20, and are still experiencing this issue, please change the version to Fedora 20.

If you experience different issues, please open a new bug report for those.

Comment 4 Michele Baldessari 2014-01-04 15:02:28 UTC
Likely fixed via 
commit 443d20fd188208aa4df2118ad49f9168e411016d
Author: Helmut Schaa <helmut.schaa>
Date:   Fri Dec 20 14:41:54 2013 +0100

    netfilter: nf_ct_timestamp: Fix BUG_ON after netns deletion
    
    When having nf_conntrack_timestamp enabled deleting a netns
    can lead to the following BUG being triggered:
    
    [63836.660000] Kernel bug detected[#1]:
    [63836.660000] CPU: 0 PID: 0 Comm: swapper Not tainted 3.10.18 #14
    [63836.660000] task: 802d9420 ti: 802d2000 task.ti: 802d2000
    [63836.660000] $ 0   : 00000000 00000000 00000000 00000000
    [63836.660000] $ 4   : 00000001 00000004 00000020 00000020
    [63836.660000] $ 8   : 00000000 80064910 00000000 00000000
    [63836.660000] $12   : 0bff0002 00000001 00000000 0a0a0abe
    [63836.660000] $16   : 802e70a0 85f29d80 00000000 00000004
    [63836.660000] $20   : 85fb62a0 00000002 802d3bc0 85fb62a0
    [63836.660000] $24   : 00000000 87138110
    [63836.660000] $28   : 802d2000 802d3b40 00000014 871327cc
    [63836.660000] Hi    : 000005ff
    [63836.660000] Lo    : f2edd000
    [63836.660000] epc   : 87138794 __nf_ct_ext_add_length+0xe8/0x1ec [nf_conntrack]
    [63836.660000]     Not tainted
    [63836.660000] ra    : 871327cc nf_conntrack_in+0x31c/0x7b8 [nf_conntrack]
    [63836.660000] Status: 1100d403 KERNEL EXL IE
    [63836.660000] Cause : 00800034
    [63836.660000] PrId  : 0001974c (MIPS 74Kc)
    [63836.660000] Modules linked in: ath9k ath9k_common pppoe ppp_async iptable_nat ath9k_hw ath pppox ppp_generic nf_nat_ipv4 nf_co
    [63836.660000] Process swapper (pid: 0, threadinfo=802d2000, task=802d9420, tls=00000000)
    [63836.660000] Stack : 802e70a0 871323d4 00000005 87080234 802e70a0 86d2a840 00000000 00000000
    [63836.660000] Call Trace:
    [63836.660000] [<87138794>] __nf_ct_ext_add_length+0xe8/0x1ec [nf_conntrack]
    [63836.660000] [<871327cc>] nf_conntrack_in+0x31c/0x7b8 [nf_conntrack]
    [63836.660000] [<801ff63c>] nf_iterate+0x90/0xec
 

has not hit stable yet

Comment 5 Josh Boyer 2014-01-12 14:19:59 UTC
3.12.7 is in updates-testing now.

Comment 6 Justin M. Forbes 2014-03-10 14:48:08 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 19 kernel bugs.

Fedora 19 has now been rebased to 3.13.5-100.fc19.  Please test this kernel update and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you experience different issues, please open a new bug report for those.

Comment 7 Justin M. Forbes 2014-06-23 14:41:16 UTC
*********** MASS BUG UPDATE **************
This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 4 weeks. If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously.

Comment 8 zhaoquanbin 2016-04-26 11:56:21 UTC
Where can i get the patch?

Comment 9 Wade Mealing 2017-01-26 03:30:36 UTC
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=443d20fd188208aa4df2118ad49f9168e411016d

Is where you can get the patch.. but they would have updated the kernel many times now.
 
[linux-stable]$ git tag --contains 443d20fd188208aa4df2118ad49f9168e411016d | head -n 5
v3.13
v3.13-rc7
v3.13-rc8
v3.13.1
v3.13.10

I'd choose a kernel that was more recent than that, try fedora 25 ?