Bug 1659225 - Boot crash in acpi
Summary: Boot crash in acpi
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel   
(Show other bugs)
Version: 29
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL: https://bugzilla.kernel.org/show_bug....
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-13 20:34 UTC by Michael J Gruber
Modified: 2019-04-06 20:47 UTC (History)
20 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
4.19.8 boot kernelmsgs with crash (85.06 KB, text/plain)
2018-12-13 20:34 UTC, Michael J Gruber
no flags Details
4.19.8 atomacpi boot kernelmsgs without crash (88.32 KB, text/plain)
2018-12-13 20:34 UTC, Michael J Gruber
no flags Details
4.19.7 boot kernelmsgs with crash (89.10 KB, text/plain)
2018-12-13 20:35 UTC, Michael J Gruber
no flags Details
4.19.3 boot kernelmsgs without crash (94.32 KB, text/plain)
2018-12-13 20:36 UTC, Michael J Gruber
no flags Details
lspci (2.22 KB, text/plain)
2018-12-13 20:36 UTC, Michael J Gruber
no flags Details
lscpu (1008 bytes, text/plain)
2018-12-13 20:37 UTC, Michael J Gruber
no flags Details
4.19.2-300 boot kernelmsgs with crash (84.58 KB, text/plain)
2018-12-14 20:15 UTC, Michael J Gruber
no flags Details
4.19.2-301 boot kernelmsgs with crash (84.10 KB, text/plain)
2018-12-14 20:16 UTC, Michael J Gruber
no flags Details
4.19.6 boot kernelmsgs with crash (85.63 KB, text/plain)
2018-12-14 20:17 UTC, Michael J Gruber
no flags Details
4.19.9 boot kernelmsgs with crash (85.89 KB, text/plain)
2018-12-14 20:18 UTC, Michael J Gruber
no flags Details
4.19.4-300 boot kernelmsgs without crash (92.02 KB, text/plain)
2018-12-14 20:20 UTC, Michael J Gruber
no flags Details
4.19.5 boot kernelmsgs without crash (89.56 KB, text/plain)
2018-12-14 20:20 UTC, Michael J Gruber
no flags Details
Kernel logs + hardware info (128.78 KB, text/plain)
2018-12-27 23:30 UTC, Francois Andrieu
no flags Details
4.20.0 boot kernelmsgs with crash (86.33 KB, text/plain)
2018-12-29 20:08 UTC, Michael J Gruber
no flags Details
4.20.4 crash (88.30 KB, text/plain)
2019-01-30 20:47 UTC, Michael J Gruber
no flags Details
4.20.4 no crash with revert patch (92.85 KB, text/plain)
2019-01-30 20:48 UTC, Michael J Gruber
no flags Details
4.20.5 crash (88.55 KB, text/plain)
2019-01-30 20:49 UTC, Michael J Gruber
no flags Details
4.20.5 no crash with revert patch (88.30 KB, text/plain)
2019-01-30 20:49 UTC, Michael J Gruber
no flags Details

Description Michael J Gruber 2018-12-13 20:34:00 UTC
Created attachment 1514197 [details]
4.19.8 boot kernelmsgs with crash

Description of problem:
Specific kernels in the 4.19 series crash during boot in the acpi component on specific hardware (see below).

Version-Release number of selected component (if applicable):
4.19.8-300 crashes as well as 4.19.7, not 4.19.3

How reproducible:
always

Steps to Reproduce:
1. boot

Actual results:
crash during boot

Expected results:
boot without crash

Additional info:
This is is an old HP/Compag 6715b laptop. lscpu and lspci output is attached. 4.18 kernels worked without problems. Problem appeared with 4.19.1, disappeared with 4.19.3 (and 5), reappeared with 4.19.7 and 4.19.8.

Note that the koj build from https://koji.fedoraproject.org/koji/taskinfo?taskID=31411873 ("atomacpi") does work nicely!

Comment 1 Michael J Gruber 2018-12-13 20:34 UTC
Created attachment 1514198 [details]
4.19.8 atomacpi boot kernelmsgs without crash

Comment 2 Michael J Gruber 2018-12-13 20:35 UTC
Created attachment 1514199 [details]
4.19.7 boot kernelmsgs with crash

Comment 3 Michael J Gruber 2018-12-13 20:36 UTC
Created attachment 1514200 [details]
4.19.3 boot kernelmsgs without crash

Comment 4 Michael J Gruber 2018-12-13 20:36 UTC
Created attachment 1514201 [details]
lspci

Comment 5 Michael J Gruber 2018-12-13 20:37 UTC
Created attachment 1514202 [details]
lscpu

Comment 6 Jeremy Cline 2018-12-14 15:20:22 UTC
Thanks for filing this.

What does the v4.19.1/v4.19.2 crash look like? The patch I reverted in https://koji.fedoraproject.org/koji/taskinfo?taskID=31411873 arrived in v4.19.6 so it's probably an unrelated problem that got fixed, but it'd be good to see it.

Also, are you okay with me adding you to the Cc list when I email upstream about this?

Comment 7 Michael J Gruber 2018-12-14 20:15 UTC
Created attachment 1514466 [details]
4.19.2-300 boot kernelmsgs with crash

Comment 8 Michael J Gruber 2018-12-14 20:16 UTC
Created attachment 1514467 [details]
4.19.2-301 boot kernelmsgs with crash

Comment 9 Michael J Gruber 2018-12-14 20:17 UTC
Created attachment 1514468 [details]
4.19.6 boot kernelmsgs with crash

Comment 10 Michael J Gruber 2018-12-14 20:18 UTC
Created attachment 1514469 [details]
4.19.9 boot kernelmsgs with crash

Comment 11 Michael J Gruber 2018-12-14 20:20 UTC
Created attachment 1514470 [details]
4.19.4-300 boot kernelmsgs without crash

Comment 12 Michael J Gruber 2018-12-14 20:20 UTC
Created attachment 1514472 [details]
4.19.5 boot kernelmsgs without crash

Comment 13 Michael J Gruber 2018-12-14 20:25:50 UTC
BZ needs attachment multi-upload (or I need to find out how)...

Here are a few more data points for that same machine. (cc upstream is okay as confirmed by pm already)

Comment 14 Francois Andrieu 2018-12-27 23:23:40 UTC
Can confirm I have the exact same issue with a custom blade server from Online.net, with what seems to be a completely different hardware than OP.
This server is a production one so tests are limited for now.
I only have 3 kernels available: 
4.19.8-200 : Crash not long after the call trace (acpi_ns_get_normalized_pathname)  
4.19.10-200 : Same thing, but I got it to boot (kind of) successfully one time only. Ethernet device was not detected and server crash during the shutdown sequence few minutes after.
4.18.18-200 : Currently running kernel. Work just fine

Comment 15 Francois Andrieu 2018-12-27 23:30 UTC
Created attachment 1517144 [details]
Kernel logs + hardware info

This log include the 4.19.10-200 log (the one that completed the boot sequence for some reason, without the eth device and crash at shutdown)
and the 4.19.8-200 log, which didn't succeed

Comment 16 Michael J Gruber 2018-12-29 20:08 UTC
Created attachment 1517367 [details]
4.20.0 boot kernelmsgs with crash

This is with 4.20.0 from knurd's kernel-vanilla-stable. Things won't get better wih 4.20, it seems.

Comment 17 Michael J Gruber 2018-12-31 12:20:00 UTC
4.19.10 does work if I revert the same commit as jcline in his atomacpi build, see https://copr.fedorainfracloud.org/coprs/mjg/kernel-hp6715b/

I guess I should not pester all current Fedora kernel updates with negative karma because of this issue, should I?

Comment 18 Jeremy Cline 2019-01-10 18:22:57 UTC
Yes, this is an upstream issue so it needs to get addressed there. The patch in question, commit 4abb951b73ff ("ACPICA: AML interpreter: add region addresses in global list during initialization") was first introduced in upstream v4.19.2 (commit 22083c028d0b), reverted in v4.19.3 (commit 8ef305fbc50d), and finally included again in v4.19.6 (commit 87403e35bc56). I sent an email in response to its inclusion in v4.19.6, but it must have slipped through the cracks for the ACPI maintainers. I think the best thing to do would be to report it upstream: https://bugzilla.kernel.org/enter_bug.cgi?product=ACPI.

Comment 19 Justin M. Forbes 2019-01-29 16:13:11 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 29 kernel bugs.

Fedora 29 has now been rebased to 4.20.5-200.fc29.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you experience different issues, please open a new bug report for those.

Comment 20 Michael J Gruber 2019-01-30 20:47 UTC
Created attachment 1525141 [details]
4.20.4 crash

Comment 21 Michael J Gruber 2019-01-30 20:48 UTC
Created attachment 1525142 [details]
4.20.4 no crash with revert patch

Comment 22 Michael J Gruber 2019-01-30 20:49 UTC
Created attachment 1525143 [details]
4.20.5 crash

Comment 23 Michael J Gruber 2019-01-30 20:49 UTC
Created attachment 1525145 [details]
4.20.5 no crash with revert patch

Comment 24 Michael J Gruber 2019-01-30 20:52:09 UTC
(In reply to Justin M. Forbes from comment #19)
> *********** MASS BUG UPDATE **************
> 
> We apologize for the inconvenience.  There are a large number of bugs to go
> through and several of them have gone stale.  Due to this, we are doing a
> mass bug update across all of the Fedora 29 kernel bugs.
> 
> Fedora 29 has now been rebased to 4.20.5-200.fc29.  Please test this kernel
> update (or newer) and let us know if you issue has been resolved or if it is
> still present with the newer kernel.
> 
> If you experience different issues, please open a new bug report for those.

There is no change for the better with 4.20.4 nor 4.20.5. Same crash, and the crash goes away with the same revert patch.

ACPI people did not even bother to answer GKH's question why they went back and forth with this patch, it seems. I doubt filing a bug an the kernel's bz would help :(

Comment 25 Michael J Gruber 2019-02-27 09:43:45 UTC
There is a new patch by Eric Schmauss who has been very patient with my rebuild/reboot/debug cycles: see the kernel bz at https://bugzilla.kernel.org/attachment.cgi?id=281267

Eric would be happy to see more testing by others who have experienced the same issue. His patch avoids the revert and fixes the problem on my machine. A Fedora 29 build (Fedora 4.20.5-200 kernel + Eric's patch) can be found here:

http://koji.fedoraproject.org/koji/taskinfo?taskID=33042688

Let me know if you want to test on other archs and need a kernel build.

Comment 26 recteurlp 2019-04-06 02:11:05 UTC
Hello, still got this bug with kernel 5.0.5-200.x86_64 on f29. Is a newer build available with Eric's patch ?
Thanks

Comment 27 Michael J Gruber 2019-04-06 20:47:05 UTC
The patch is not yet in Fedora's kernel. It suppose it will be in 5.1.

I regularly build Fedora kernels with Eric's patch at: https://copr.fedorainfracloud.org/coprs/mjg/kernel-hp6715b/

Note that I build them usually as soon as they hit updates-testing, so you might pick up a kernel there before the regular Fedora kernel hits the updates repo.


Note You need to log in before you can comment on or make changes to this bug.