Bug 1282706

Summary: Kernel panic - not syncing: Fatal exception in interrupt
Product: [Fedora] Fedora Reporter: poma <pomidorabelisima>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: rawhideCC: gansalmon, itamar, jonathan, kernel-maint, labbott, madhu.chinakonda, mchehab, mcsontos, michal.jnn, orion, pbrobinson, praiskup, robatino, vlee
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-30 17:35:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description poma 2015-11-17 08:17:26 UTC
Description of problem:
Kernel panic - not syncing: Fatal exception in interrupt

Version-Release number of selected component (if applicable):
4.4.0-0.rc1.git0.1.fc24.x86_64

How reproducible:
101%

Steps to Reproduce:
1. Boot 4.4.0-0.rc1.git0.1.fc24.x86_64

Actual results:
Kernel panic

Expected results:
Kernel -NOT- panic

Additional info:
...
[    3.259387] ------------[ cut here ]------------
[    3.279241] WARNING: CPU: 1 PID: 1 at arch/x86/mm/dump_pagetables.c:225 note_page+0x5e1/0x780()
[    3.300641] x86/mm: Found insecure W+X mapping at address ffff880000806000/0xffff880000806000
[    3.323793] Modules linked in:[    3.324838] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input3

[    3.369743] 
[    3.410378] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.4.0-0.rc1.git0.1.fc24.x86_64 #1
[    3.434592] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015
[    3.454885]  0000000000000000 00000000d00ca0e8 ffff88006d89bd48 ffffffff813aebdf
[    3.479814]  ffff88006d89bd90 ffff88006d89bd80 ffffffff810a1a32 ffff88006d89be90
[    3.502826]  8000000000000163 0000000000000004 0000000000000000 0000000000000000
[    3.526346] Call Trace:
[    3.545647]  [<ffffffff813aebdf>] dump_stack+0x44/0x55
[    3.562857]  [<ffffffff810a1a32>] warn_slowpath_common+0x82/0xc0
[    3.585882]  [<ffffffff810a1acc>] warn_slowpath_fmt+0x5c/0x80
[    3.604974]  [<ffffffff8106f6b1>] note_page+0x5e1/0x780
[    3.622990]  [<ffffffff8106fb57>] ptdump_walk_pgd_level_core+0x307/0x450
[    3.646398]  [<ffffffff8106fd47>] ptdump_walk_pgd_level_checkwx+0x17/0x20
[    3.666146]  [<ffffffff81065a64>] mark_rodata_ro+0xf4/0x100
[    3.683298]  [<ffffffff817842d0>] ? rest_init+0x80/0x80
[    3.705663]  [<ffffffff817842ed>] kernel_init+0x1d/0xe0
[    3.721344]  [<ffffffff817902cf>] ret_from_fork+0x3f/0x70
[    3.734937]  [<ffffffff817842d0>] ? rest_init+0x80/0x80
[    3.753230] ---[ end trace 47744449998b48c5 ]---
[    3.775405] x86/mm: Checked W+X mappings: FAILED, 21811 W+X pages found.
...
[    5.781043] ------------[ cut here ]------------
[    5.802183] WARNING: CPU: 0 PID: 18 at kernel/cgroup_pids.c:97 pids_cancel.constprop.5+0x31/0x40()
[    5.840373] Modules linked in: drm(+)[    5.842997] [drm] Initialized drm 1.1.0 20060810

[    5.862519]  serio_raw 8139cp(+) virtio_pci(+) virtio_ring virtio mii ata_generic pata_acpi scsi_transport_iscsi loop
[    5.924600] CPU: 0 PID: 18 Comm: rcuos/1 Tainted: G        W       4.4.0-0.rc1.git0.1.fc24.x86_64 #1
[    5.959619] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
[    5.983758] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015
[    5.983762]  0000000000000000 0000000066f080e1 ffff88006d91bd50 ffffffff813aebdf
[    5.983763]  0000000000000000 ffff88006d91bd88 ffffffff810a1a32 ffff88003ea2f800
[    5.983764]  ffff88003eb20000 ffff88003ec6c600 ffff88003ea13d00 ffff88006d90b980
[    5.983765] Call Trace:
[    5.983772]  [<ffffffff813aebdf>] dump_stack+0x44/0x55
[    5.983775]  [<ffffffff810a1a32>] warn_slowpath_common+0x82/0xc0
[    5.983780]  [<ffffffff810a1b7a>] warn_slowpath_null+0x1a/0x20
[    5.983782]  [<ffffffff81137041>] pids_cancel.constprop.5+0x31/0x40
[    5.983783]  [<ffffffff81137175>] pids_free+0x25/0x40
[    5.983784]  [<ffffffff811365f1>] cgroup_free+0x41/0x70
[    5.983786]  [<ffffffff8109ecd2>] __put_task_struct+0x42/0x140
[    5.983788]  [<ffffffff810a2d94>] delayed_put_task_struct+0x64/0x70
[    5.983790]  [<ffffffff811044b3>] rcu_nocb_kthread+0x173/0x500
[    5.983792]  [<ffffffff810e3570>] ? wake_atomic_t_function+0x70/0x70
[    5.983794]  [<ffffffff81104340>] ? get_state_synchronize_rcu+0x20/0x20
[    5.983796]  [<ffffffff810bfbd8>] kthread+0xd8/0xf0
[    5.983798]  [<ffffffff810bfb00>] ? kthread_worker_fn+0x160/0x160
[    5.983800]  [<ffffffff817902cf>] ret_from_fork+0x3f/0x70
[    5.983801]  [<ffffffff810bfb00>] ? kthread_worker_fn+0x160/0x160
[    5.983803] ---[ end trace 47744449998b48c6 ]---
[    5.983812] BUG: unable to handle kernel NULL pointer dereference at 00000000000000c1
[    5.983814] IP: [<ffffffff81137015>] pids_cancel.constprop.5+0x5/0x40
[    5.983815] PGD 0 
[    5.983816] Oops: 0002 [#1] SMP 
[    5.983822] Modules linked in: drm_kms_helper ttm drm serio_raw 8139cp(+) virtio_pci(+) virtio_ring virtio mii ata_generic pata_acpi scsi_transport_iscsi loop
[    5.983825] CPU: 0 PID: 18 Comm: rcuos/1 Tainted: G        W       4.4.0-0.rc1.git0.1.fc24.x86_64 #1
[    5.983825] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015
[    5.983826] task: ffff88006d90b980 ti: ffff88006d918000 task.ti: ffff88006d918000
[    5.983828] RIP: 0010:[<ffffffff81137015>]  [<ffffffff81137015>] pids_cancel.constprop.5+0x5/0x40
[    5.983829] RSP: 0018:ffff88006d91bdb0  EFLAGS: 00010202
[    5.983830] RAX: 0000000000000024 RBX: 0000000000000001 RCX: 0000000164a9918f
[    5.983830] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001
[    5.983831] RBP: ffff88006d91bdc0 R08: 0000000000000008 R09: 0000000000000024
[    5.983831] R10: ffff88003e8b8c80 R11: 0000000000033d9c R12: ffff88003eb20000
[    5.983832] R13: ffff88003ec6c600 R14: ffff88003ea13d00 R15: ffff88006d90b980
[    5.983837] FS:  00007f8f0443b8c0(0000) GS:ffff88006e400000(0000) knlGS:0000000000000000
[    5.983838] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[    5.983839] CR2: 00000000000000c1 CR3: 0000000002c09000 CR4: 00000000000006f0
[    5.983841] Stack:
[    5.983842]  ffffffff81137175 000000000000000b ffff88006d91bde8 ffffffff811365f1
[    5.983843]  ffff88003eb20000 ffff88003eb20000 ffff88003eb209f0 ffff88006d91be08
[    5.983844]  ffffffff8109ecd2 ffff88003eb209f0 ffff88003eb20000 ffff88006d91be30
[    5.983845] Call Trace:
[    5.983846]  [<ffffffff81137175>] ? pids_free+0x25/0x40
[    5.983847]  [<ffffffff811365f1>] cgroup_free+0x41/0x70
[    5.983848]  [<ffffffff8109ecd2>] __put_task_struct+0x42/0x140
[    5.983849]  [<ffffffff810a2d94>] delayed_put_task_struct+0x64/0x70
[    5.983850]  [<ffffffff811044b3>] rcu_nocb_kthread+0x173/0x500
[    5.983852]  [<ffffffff810e3570>] ? wake_atomic_t_function+0x70/0x70
[    5.983858]  [<ffffffff81104340>] ? get_state_synchronize_rcu+0x20/0x20
[    5.983860]  [<ffffffff810bfbd8>] kthread+0xd8/0xf0
[    5.983861]  [<ffffffff810bfb00>] ? kthread_worker_fn+0x160/0x160
[    5.983862]  [<ffffffff817902cf>] ret_from_fork+0x3f/0x70
[    5.983864]  [<ffffffff810bfb00>] ? kthread_worker_fn+0x160/0x160
[    5.983874] Code: 5b 5d c3 48 89 df 48 c7 c2 92 08 a6 81 48 c7 c6 04 1c a5 81 e8 dd 61 11 00 31 c0 5b 5d c3 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 <f0> 48 83 87 c0 00 00 00 ff 78 01 c3 80 3d 9f a8 bd 00 00 74 02 
[    5.983876] RIP  [<ffffffff81137015>] pids_cancel.constprop.5+0x5/0x40
[    5.983876]  RSP <ffff88006d91bdb0>
[    5.983876] CR2: 00000000000000c1
[    5.983879] ---[ end trace 47744449998b48c7 ]---
[    5.983880] Kernel panic - not syncing: Fatal exception in interrupt
[    5.984719] Kernel Offset: disabled
[    5.984719] ---[ end Kernel panic - not syncing: Fatal exception in interrupt

Comment 2 poma 2015-11-19 10:31:44 UTC
Revert "cgroup: add cgroup_subsys->free() method and use it to fix pids controller" 
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=afcf6c8
&
Revert "cgroup: keep zombies associated with their original cgroups"
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2e91fa7

fix Kernel panic.

Tested on baremetal and QEMU with:
qemu-2.5.0-0.1.rc0.fc24.x86_64
kernel-4.4.0-0.rc1.git1.2.fc24.x86_64

Comment 3 Laura Abbott 2015-11-20 01:00:30 UTC
*** Bug 1281409 has been marked as a duplicate of this bug. ***

Comment 4 Andre Robatino 2015-11-23 12:00:08 UTC
Am still seeing this with kernel-4.4.0-0.rc1.git3.1.fc24.x86_64, though with this kernel I can at least occasionally get it to boot up fully, while the earlier kernels fail before I can even log into GNOME. ((Until this kernel, I had to update using rescue mode.) This is in a VirtualBox 5.0.10 guest.

Comment 5 Laura Abbott 2015-11-24 16:06:33 UTC
Can you test https://koji.fedoraproject.org/koji/taskinfo?taskID=11914111 which has the reverts suggested?

Comment 6 Andre Robatino 2015-11-24 18:56:34 UTC
(In reply to Laura Abbott from comment #5)
> Can you test https://koji.fedoraproject.org/koji/taskinfo?taskID=11914111
> which has the reverts suggested?

Appears to work - I booted and shut down several times without seeing the problem. Thanks.

Comment 7 Laura Abbott 2015-11-24 23:29:57 UTC
A potential fix is being discussed http://marc.info/?l=linux-kernel&m=144830855403820 . It doesn't seem to be complete though (still have the warning) so I'll just monitor the thread.

Comment 8 poma 2015-11-25 16:26:38 UTC
(In reply to Laura Abbott from comment #7)
> A potential fix is being discussed
> http://marc.info/?l=linux-kernel&m=144830855403820 . It doesn't seem to be
> complete though (still have the warning) so I'll just monitor the thread.


https://patchwork.kernel.org/patch/7685771/mbox
cgroup-for-4.4-fixes-cgroup-make-css_set-pin-its-css-s-to-avoid-use-afer-free.patch

PASSED = Kernel -NOT- panicking / none related WARNINGs

Tested with:
4.4.0-0.rc2.git1.2.fc24.x86_64
(4.4.0-0.rc2.git1.1.fc24.x86_64 + patch)

Comment 9 poma 2015-11-25 16:28:47 UTC
(In reply to poma from comment #8)
> (In reply to Laura Abbott from comment #7)
> > A potential fix is being discussed
> > http://marc.info/?l=linux-kernel&m=144830855403820 . It doesn't seem to be
> > complete though (still have the warning) so I'll just monitor the thread.
> 
> 
> https://patchwork.kernel.org/patch/7685771/mbox
> cgroup-for-4.4-fixes-cgroup-make-css_set-pin-its-css-s-to-avoid-use-afer-
> free.patch
> 
> PASSED = Kernel -NOT- panicking / none related WARNINGs
> 
> Tested with:
> 4.4.0-0.rc2.git1.2.fc24.x86_64
> (4.4.0-0.rc2.git1.1.fc24.x86_64 + patch)


Valid for both, baremetal and libvirt(QEMU), of course.

Comment 10 poma 2015-11-30 17:31:51 UTC
(In reply to poma from comment #9)
> (In reply to poma from comment #8)
> > (In reply to Laura Abbott from comment #7)
> > > A potential fix is being discussed
> > > http://marc.info/?l=linux-kernel&m=144830855403820 . It doesn't seem to be
> > > complete though (still have the warning) so I'll just monitor the thread.
> > 
> > 
> > https://patchwork.kernel.org/patch/7685771/mbox
> > cgroup-for-4.4-fixes-cgroup-make-css_set-pin-its-css-s-to-avoid-use-afer-
> > free.patch
> > 
> > PASSED = Kernel -NOT- panicking / none related WARNINGs
> > 
> > Tested with:
> > 4.4.0-0.rc2.git1.2.fc24.x86_64
> > (4.4.0-0.rc2.git1.1.fc24.x86_64 + patch)
> 
> 
> Valid for both, baremetal and libvirt(QEMU), of course.


Merge branch 'for-4.4-fixes' into for-next

"cgroup: make css_set pin its css's to avoid use-afer-free"
https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git/commit/kernel/cgroup.c?h=for-next&id=53254f9

A patch of the day, for kernel-4.4.0-0.rc3.git0.1.fc24

Comment 11 poma 2015-11-30 17:35:23 UTC
(In reply to poma from comment #10)
> (In reply to poma from comment #9)
> > (In reply to poma from comment #8)
> > > (In reply to Laura Abbott from comment #7)
> > > > A potential fix is being discussed
> > > > http://marc.info/?l=linux-kernel&m=144830855403820 . It doesn't seem to be
> > > > complete though (still have the warning) so I'll just monitor the thread.
> > > 
> > > 
> > > https://patchwork.kernel.org/patch/7685771/mbox
> > > cgroup-for-4.4-fixes-cgroup-make-css_set-pin-its-css-s-to-avoid-use-afer-
> > > free.patch
> > > 
> > > PASSED = Kernel -NOT- panicking / none related WARNINGs
> > > 
> > > Tested with:
> > > 4.4.0-0.rc2.git1.2.fc24.x86_64
> > > (4.4.0-0.rc2.git1.1.fc24.x86_64 + patch)
> > 
> > 
> > Valid for both, baremetal and libvirt(QEMU), of course.
> 
> 
> Merge branch 'for-4.4-fixes' into for-next
> 
> "cgroup: make css_set pin its css's to avoid use-afer-free"
> https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git/commit/kernel/
> cgroup.c?h=for-next&id=53254f9
> 
> A patch of the day, for kernel-4.4.0-0.rc3.git0.1.fc24

Of course,
Tested-by: poma <pomidorabelisima>

Comment 12 Josh Boyer 2015-12-01 18:21:36 UTC
*** Bug 1287198 has been marked as a duplicate of this bug. ***