Description of problem: We have two virtual machines with Fedora 20 on two different hosts with Fedora 20. From kernels 3.13 and subsequent, the virtual machine stop responding exactly after 3 days from boot. Version-Release number of selected component (if applicable): kernel 3.13 and subsequent. Now we are using 3.14.9-200 How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: here are the messages on the console [260027.360246] BUG: unable to handle kernel NULL pointer dereference at 0000000000000018 [260027.361115] IP: [<ffffffff8136cea7>] assoc_array_gc+0x2f7/0x540 [260027.361115] PGD dae15067 PUD cfc24067 PMD 0 [260027.361115] Oops: 0000 [#1] SMP [260027.361115] Modules linked in: xt_nat xt_mark nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_ni [260027.361115] CPU: 0 PID: 26011 Comm: kworker/0:1 Not tainted 3.14.9-200.fc20.x86_64 #1 [260027.361115] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [260027.361115] Workqueue: events key_garbage_collector [260027.361115] task: ffff8800918bd580 ti: ffff8800aac14000 task.ti: ffff8800aac14000 [260027.361115] RIP: 0010:[<ffffffff8136cea7>] [<ffffffff8136cea7>] assoc_array_gc+0x2f7/0x540 [260027.361115] RSP: 0018:ffff8800aac15d40 EFLAGS: 00010206 [260027.361115] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff8800aaecacc0 [260027.361115] RDX: ffff8800daecf440 RSI: 0000000000000001 RDI: ffff8800aadc2bc0 [260027.361115] RBP: ffff8800aac15da8 R08: 0000000000000001 R09: 0000000000000003 [260027.361115] R10: ffffffff8136ccc7 R11: 0000000000000000 R12: 0000000000000000 [260027.361115] R13: 0000000000000000 R14: 0000000000000070 R15: 0000000000000001 [260027.361115] FS: 0000000000000000(0000) GS:ffff88011fc00000(0000) knlGS:0000000000000000 [260027.361115] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [260027.361115] CR2: 0000000000000018 CR3: 00000000db10d000 CR4: 00000000000006f0 [260027.361115] Stack: [260027.361115] ffff8800aac15d50 0000000000000011 ffff8800aac15db8 ffffffff812e2a70 [260027.361115] ffff880091a00600 0000000000000000 ffff8800aadc2bc3 00000000cd42c987 [260027.361115] ffff88003702df20 ffff88003702dfa0 0000000053b65c09 ffff8800aac15fd8 [260027.361115] Call Trace: [260027.361115] [<ffffffff812e2a70>] ? keyring_detect_cycle_iterator+0x30/0x30 [260027.361115] [<ffffffff812e3e75>] keyring_gc+0x75/0x80 [260027.361115] [<ffffffff812e1424>] key_garbage_collector+0x154/0x3c0 [260027.361115] [<ffffffff810a67b6>] process_one_work+0x176/0x430 [260027.361115] [<ffffffff810a744b>] worker_thread+0x11b/0x3a0 [260027.361115] [<ffffffff810a7330>] ? rescuer_thread+0x3b0/0x3b0 [260027.361115] [<ffffffff810ae1a8>] kthread+0xd8/0xf0 [260027.361115] [<ffffffff810ae0d0>] ? insert_kthread_work+0x40/0x40 [260027.361115] [<ffffffff816ffb7c>] ret_from_fork+0x7c/0xb0 [260027.361115] [<ffffffff810ae0d0>] ? insert_kthread_work+0x40/0x40 [260027.361115] Code: 08 4c 8b 22 0f 84 bf 00 00 00 41 83 c7 01 49 83 e4 fc 41 83 ff 0f 4c 89 65 c0 0f 8f 5a fe ff ff 48 8b 45 c0 4d 63 cf 49 83 c1 02 <4e> 8b 34 c8 4d 85 f6 0f 84 be 00 00 00 41 f6 c6 01 0f 84 92 [260027.361115] RIP [<ffffffff8136cea7>] assoc_array_gc+0x2f7/0x540 [260027.361115] RSP <ffff8800aac15d40> [260027.361115] CR2: 0000000000000018 [260027.361115] ---[ end trace 1129028a088c0cbd ]--- [260027.407625] BUG: unable to handle kernel paging request at ffffffffffffffd8 [260027.408434] IP: [<ffffffff810ae790>] kthread_data+0x10/0x20 [260027.408434] PGD 1c0f067 PUD 1c11067 PMD 0 [260027.408434] Oops: 0000 [#2] SMP
Hi the problem is still there. Now we are using the latest Fedora 20 kernel, 3.15.10-200 The system hang after 3 days and a few hours. Thank you for your support! Lorenzo
IP: [<ffffffff8136cea7>] assoc_array_gc+0x2f7/0x540 I think this is line 1550: ptr = node->slots[slot]; RAX is node and R15 is slot.
Created attachment 935884 [details] Fix for keyrings garbage collector I managed to reproduce the problem in a userspace test harness. The problem is that gc doesn't correctly handle the root node being a shortcut. This patch should fix it.
Thanks David. I'll get this rolled into the Fedora kernels today. I'm assuming you're going to send this upstream with stable CC'd?
Fixed in Fedora git.
Proposed as a Freeze Exception for 21-alpha by Fedora user jwboyer using the blocker tracking app because: This is a security issue where a local unprivileged user could crash the system.
kernel-3.16.2-301.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/kernel-3.16.2-301.fc21
Package kernel-3.16.2-301.fc21: * should fix your issue, * was pushed to the Fedora 21 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing kernel-3.16.2-301.fc21' as soon as you are able to, then reboot. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-10693/kernel-3.16.2-301.fc21 then log in and leave karma (feedback).
+1 FE
+1 FE -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers
Hi are you going to build a F20 kernel? Lorenzo
The patch has been applied upstream: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=95389b08d93d5c06ec63ab49bd732b0069b7c35e
(In reply to Lorenzo Sartoratti from comment #11) > Hi > are you going to build a F20 kernel? > > Lorenzo Yes.
+1 FE, that gives it enough votes for accepted status. Sorry we missed reviewing this prior to RC1 :(
kernel-3.16.2-201.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/kernel-3.16.2-201.fc20
kernel-3.14.19-100.fc19 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/kernel-3.14.19-100.fc19
kernel-3.16.2-201.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report.
kernel-3.16.2-301.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report.
Hi, after 6 days with the patched kernel both f20 virtual machines are still up and running. Thank you for your work! Great job! Lorenzo
kernel-3.14.19-100.fc19 has been pushed to the Fedora 19 stable repository. If problems still persist, please make note of it in this bug report.