Bug 772730

Summary: Kernel panic after ACPI power event with x86_64 kernel
Product: [Fedora] Fedora Reporter: Didier <d.bz-redhat>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 17CC: 1800yolk, dwmw2, gansalmon, itamar, jfeeney, jonathan, kernel-maint, len.brown, madhu.chinakonda, mkhusid, tianyu.lan
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: first=3.1.6 tested=3.4.2 acpi
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-07-21 22:54:21 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
screenshot 1 of kernel panic
none
screenshot 2 of kernel panic
none
screenshot 3 of kernel panic
none
i1090 dmesg (single user mode)
none
acpidump --output i1090_acpidump.out
none
screenshot 4 of kernel panic (kernel-3.3.0-4.fc16)
none
debug.patch
none
debug.patch none

Description Didier 2012-01-09 19:58:26 UTC
Description of problem:

Dell Inspiron Duo i1090 (Atom N570). When connecting or disconnecting the power connector while the system is awake (= not suspended), the x86_64 kernel panics. This does not happen with an i686 kernel.


Version-Release number of selected component (if applicable):

kernel-3.1.6-1.fc16.x86_64


How reproducible:

Always.


Steps to Reproduce:
1. Boot x86_64 kernel (tested with standard boot, single user and "rdbreak=pre-udev")
2. Remove (or attach) power cord
3. Kernel panic


* Does not happen with i686 kernels ;
* "acpi=none" makes no difference, "pci=noapic" does not boot ;
* Reported on Ubuntu as early as July 2011 : 
   - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/414541/comments/6
   - http://ubuntuforums.org/showthread.php?t=1839057
* Power can be connected and disconnected when the system is in sleep mode, as long as, before resuming, power is reverted to its original state before the suspend.



I installed and ran kdump as specified in wiki (http://fedoraproject.org/wiki/How_to_use_kdump_to_debug_kernel_crashes), unfortunately to no avail.
The i1090 has no serial port.

Hence, please find some panic screenshots in attach.
1. in 'img1', some info has already scrolled past the visible screen ;
2. after a pause of approx. 20 seconds, 'img2' appears' ;
3. after approx. 2 seconds, 'img3' is the final feedback.

Comment 1 Didier 2012-01-09 19:59:38 UTC
Created attachment 551654 [details]
screenshot 1 of kernel panic

Comment 2 Didier 2012-01-09 20:00:18 UTC
Created attachment 551655 [details]
screenshot 2 of kernel panic

Comment 3 Didier 2012-01-09 20:01:12 UTC
Created attachment 551656 [details]
screenshot 3 of kernel panic

Comment 4 Didier 2012-01-09 20:17:26 UTC
Created attachment 551658 [details]
i1090 dmesg (single user mode)

Comment 5 Josh Boyer 2012-01-09 20:47:35 UTC
It seems the firmware is registering two devices with the same name (or so I'm told).  Can you attach the output of acpidump?

Comment 6 Didier 2012-01-09 21:32:19 UTC
Created attachment 551677 [details]
acpidump --output i1090_acpidump.out

(In reply to comment #5)
> Can you attach the output of acpidump?

Thank you for the fast reply, Josh ; much appreciated.
Attachment #5 [details].

Comment 7 Josh Boyer 2012-01-10 16:42:16 UTC
OK, there are definitely two ACAD entries in the ACPI DSDT table.  One is on the system bus, the other is off of PCI0 but the naming makes them ambiguous.  I'll send this to linux-acpi

Comment 9 Didier 2012-01-10 19:05:55 UTC
Thank you, Josh.
Would it be relevant if I'd attach i686 dmesg/acpi data, for the sake of comparison ?

Comment 10 Mike Khusid 2012-02-07 13:46:11 UTC
I can reproduce this crash at will on my Dell Inspiron Duo i1090 with 64-bit kernel. Stables with 32-bit one.

Comment 11 Didier 2012-02-08 08:58:27 UTC
I circumvented the x64 kernel crash by hibernating (instead of suspending) before attaching/detaching the power connector.

Unfortunately, resume on hibernation is currently failing (starting with 3.2.x kernel series, I guess). Bugger !

Comment 12 Josh Boyer 2012-03-01 17:24:07 UTC
*** Bug 767896 has been marked as a duplicate of this bug. ***

Comment 13 Brian Westwood 2012-03-02 02:51:10 UTC
In case more screenshots are needed:
http://picasaweb.google.com/103156720104886162342/FedoraCrash

Comment 14 Dave Jones 2012-03-22 16:40:17 UTC
[mass update]
kernel-3.3.0-4.fc16 has been pushed to the Fedora 16 stable repository.
Please retest with this update.

Comment 15 Dave Jones 2012-03-22 16:45:21 UTC
[mass update]
kernel-3.3.0-4.fc16 has been pushed to the Fedora 16 stable repository.
Please retest with this update.

Comment 16 Dave Jones 2012-03-22 16:54:12 UTC
[mass update]
kernel-3.3.0-4.fc16 has been pushed to the Fedora 16 stable repository.
Please retest with this update.

Comment 17 Didier 2012-03-22 18:45:29 UTC
I tested this morning with kernel-3.3.0-4.fc16 :
- in single user mode, 4 or 5 subsequent power source changes yielded no oops ;
- booted in initlevel 5, the power source change immediately invoked an oops.

I will run a new batch of tests tomorrow.

Comment 18 Didier 2012-03-23 09:33:31 UTC
Created attachment 572207 [details]
screenshot 4 of kernel panic (kernel-3.3.0-4.fc16)

Tested with kernel-3.3.0-4.fc16 x86_64 ; the kernel still OOPSes, both in single- and multiuser.

I updated the screenshot (the 3.3.0 stack trace may reveal more info).

Comment 19 Didier 2012-04-23 11:33:46 UTC
Tested with kernel-3.3.2-1.fc16.x86_64 : the kernel still panics.

Comment 20 Didier 2012-05-30 15:44:19 UTC
Kernel panic in F17 too
(tested with kernel-3.3.7-1.fc17.x86_64).

Comment 21 Didier 2012-06-04 09:34:32 UTC
Tested with kernel-3.4.0-1.fc17.x86_64 (updates-testing repo) :
the kernel still panics when connecting or removing an AC adapter.

Comment 22 Didier 2012-06-19 07:34:02 UTC
Panics with 3.4.2-4.fc17.x86_64.

Not wanting to spam, so is there any use in me reporting failing kernels ?

Comment 23 Dave Jones 2012-06-19 17:35:11 UTC
I poked linux-acpi upstream again.

Comment 24 Didier 2012-06-19 17:39:07 UTC
/* bows */

Comment 25 Lan Tianyu 2012-06-26 07:17:48 UTC
Created attachment 594375 [details]
debug.patch

hi Didier:
         Can you try this patch and test again?

Comment 26 Didier 2012-06-27 18:21:05 UTC
Dear Lan,

The patch applied cleanly against kernel-3.4.4-3.fc17 (Fedora 17 updates-testing).

I can now change power states at will without oopses ; thank you VERY much.


I would be very much obliged if this patch could be accepted for the next kernel update.
Status : CLOSED -> UPSTREAM ?

Comment 27 Lan Tianyu 2012-06-29 07:24:48 UTC
Created attachment 595219 [details]
debug.patch

hi Didier:
         Can you help me to test this patch which maybe more decent? If ok, I will send this one to acpi maillist. Thx.

Comment 28 Didier 2012-06-29 12:08:42 UTC
Dear Lan,

Due to vacation abroad, I'll be able to test in about two weeks ...

Comment 29 Brian Westwood 2012-06-30 00:32:10 UTC
I'll test it, but I'm not sure where to put that code. If you add a few instructions so I get the gist of it, I'd be happy to help

Comment 30 Lan Tianyu 2012-06-30 12:24:05 UTC
Thanks. That's easy. Just add a check return value of power_supply_register() in the driver/acpi/ac.c.

at line 295.
result = power_supply_register(&ac->device->dev, &ac->charger); 
if (result) 
	goto end

Comment 31 Brian Westwood 2012-06-30 23:38:00 UTC
I used to run Fedora on my Dell, but now I run arch.. I did a search for ac.c but there is no such file on the system, so clearly arch handles drivers differently. I found an ac.ko.gz file, and a few acpi related header files, but nothing that I could pin down as the driver. Sorry I couldn't help!

Comment 32 Lan Tianyu 2012-07-01 04:38:29 UTC
Can you git a kernerl tree?
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
modified code, compile kernel and test.

Comment 33 Didier 2012-07-17 14:25:31 UTC
(In reply to comment #27)
> Created attachment 595219 [details]

This patch, applied to kernel 3.4.5-2.fc17, works perfectly.

Thanks, Lan !

Comment 34 Josh Boyer 2012-07-17 14:37:04 UTC
(In reply to comment #33)
> (In reply to comment #27)
> > Created attachment 595219 [details]
> 
> This patch, applied to kernel 3.4.5-2.fc17, works perfectly.
> 
> Thanks, Lan !

Thank you for testing Didier.  Lan, if you post this upstream could you please put a link to it in this bug?  We can get the patch added into the Fedora kernel shortly after that.

Comment 35 Lan Tianyu 2012-07-18 03:17:12 UTC
hi josh:
         I have sent this patch to acpi maillist but no response.

http://marc.info/?l=linux-acpi&m=134138075901658&w=2

Comment 36 Josh Boyer 2012-07-18 13:06:55 UTC
(In reply to comment #35)
> hi josh:
>          I have sent this patch to acpi maillist but no response.
> 
> http://marc.info/?l=linux-acpi&m=134138075901658&w=2

That's fine.  We'll pull it into Fedora for now and we can adjust the patch as needed.

Many thanks again.

Comment 37 Josh Boyer 2012-07-18 13:44:34 UTC
Patch applied to all Fedora branches.

Comment 38 Fedora Update System 2012-07-20 14:05:22 UTC
kernel-3.4.6-2.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/kernel-3.4.6-2.fc17

Comment 39 Fedora Update System 2012-07-21 02:58:43 UTC
Package kernel-3.4.6-2.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing kernel-3.4.6-2.fc17'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-10952/kernel-3.4.6-2.fc17
then log in and leave karma (feedback).

Comment 40 Fedora Update System 2012-07-21 22:54:21 UTC
kernel-3.4.6-2.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 41 Fedora Update System 2012-07-24 11:12:49 UTC
kernel-3.4.6-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/kernel-3.4.6-1.fc16

Comment 42 Len Brown 2012-07-26 23:16:33 UTC
shipped upstream in Linux-3.6-merge.
was also cc: -stable


commit f197ac13f6eeb351b31250b9ab7d0da17434ea36
Author: Lan Tianyu <tianyu.lan>
Date:   Fri Jul 20 13:29:16 2012 +0800

    ACPI/AC: prevent OOPS on some boxes due to missing check power_supply_register() return value check

Comment 43 Fedora Update System 2012-07-27 09:52:43 UTC
kernel-3.4.6-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.