Bug 1285235

Summary: 3.10.0-327.el7 crashes on boot on HP N54L
Product: Red Hat Enterprise Linux 7 Reporter: Pablo Iranzo Gómez <pablo.iranzo>
Component: kernelAssignee: Prarit Bhargava <prarit>
kernel sub component: Other QA Contact: Red Hat Kernel QE team <kernel-qe>
Status: CLOSED DUPLICATE Docs Contact:
Severity: medium    
Priority: medium CC: fabian.arrotin, gasmith, javier.ramirez, jentrena, ossman, pablo.iranzo, pasteur, rainer.traut, riehecky, slawomir, toracat
Version: 7.2   
Target Milestone: rc   
Target Release: 7.3   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-02 14:00:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1203710    
Attachments:
Description Flags
Image of iLO capture
none
sosreport
none
Capture during 7.2 install from USB none

Description Pablo Iranzo Gómez 2015-11-25 09:30:53 UTC
Description of problem:

After updating system to 7.2 with kernel 3.10.0-327.el7 system crashes on boot.

Version-Release number of selected component (if applicable):

Kernel 3.10.0-327.el7

How reproducible:

Upgrade kernel to 3.10.0-327.el7, reboot system, crash appears

Actual results:
System crashes on boot


Expected results:
System boots properly


Additional info:

Booting with previous kernel makes the system boot again

Comment 1 Pablo Iranzo Gómez 2015-11-25 09:31:42 UTC
Created attachment 1098640 [details]
Image of iLO capture

Comment 2 Pablo Iranzo Gómez 2015-11-25 10:19:49 UTC
I'm unable to get sysrq working either via console or via ipmitool sol access

A colleague has same problem.

From the OS I can trigger crash and it works.

Any hint?

Comment 4 Javier Ramirez 2015-11-25 11:22:10 UTC
Same happening in my AMD Turion(tm) II Neo N54L Dual-Core Processor, I have no ILO so no screenshots of the panic, but I want to mention that is happening so early in the boot process that kdump nor sysrq works. I'm attaching a sosreport just in case.

Comment 5 Javier Ramirez 2015-11-25 11:23:57 UTC
Created attachment 1098745 [details]
sosreport

Comment 6 Gary Smith 2015-11-25 14:02:43 UTC
Created attachment 1098825 [details]
Capture during 7.2 install from USB

I tried to install 7.2 via USB stick on my N54L and got the attached backtrace as soon as it booted, which is different from comment#1

Comment 8 Prarit Bhargava 2015-12-02 13:04:34 UTC
Can you try booting with the kernel parameter 

initcall_blacklist=clocksource_done_booting

?

P.

Comment 10 Julio Entrena Perez 2015-12-02 13:57:09 UTC
(In reply to Prarit Bhargava from comment #8)
> Can you try booting with the kernel parameter 
> 
> initcall_blacklist=clocksource_done_booting

Yep, that option allows it to boot up successfully.

Comment 11 Prarit Bhargava 2015-12-02 14:00:00 UTC
(In reply to Julio Entrena Perez from comment #10)
> (In reply to Prarit Bhargava from comment #8)
> > Can you try booting with the kernel parameter 
> > 
> > initcall_blacklist=clocksource_done_booting
> 
> Yep, that option allows it to boot up successfully.

\o/ :)

Okay, a fix for this was put into the z-stream release for RHEL7.2.  Unfortunately you'll have to install with this option and then immediately upgrade the kernel to the RHEL7.2.z release.

I'm closing this bug as a DUPLICATE of BZ 1265283.

P.

*** This bug has been marked as a duplicate of bug 1265283 ***

Comment 12 Prarit Bhargava 2015-12-02 14:02:17 UTC
(In reply to Javier Ramirez from comment #4)
> Same happening in my AMD Turion(tm) II Neo N54L Dual-Core Processor, I have
> no ILO so no screenshots of the panic, but I want to mention that is
> happening so early in the boot process that kdump nor sysrq works. I'm
> attaching a sosreport just in case.

FWIW ... for some odd reason the AMD Turion processors seem to hit this more often than any other processor.  I'm not sure why that is, but it has been resolved in BZ 1265283.

P.

Comment 13 Julio Entrena Perez 2015-12-02 14:02:59 UTC
Thanks prarit++

Comment 14 Fabian Arrotin 2015-12-15 09:07:18 UTC
Just to add to the bug report that initcall_blacklist=clocksource_done_booting kernel parameter also solves the same issue on the following lenovo thinkpad : 

 Manufacturer: LENOVO
 Product Name: 25453PG
 Version: ThinkPad Edge
 Processor model name: AMD Athlon(tm) II Neo K345 Dual-Core Processor

Comment 15 Prarit Bhargava 2016-01-03 16:51:32 UTC
*** Bug 1295056 has been marked as a duplicate of this bug. ***

Comment 16 Tru Huynh 2016-01-27 12:34:42 UTC
from the CentOS bugs entry: https://bugs.centos.org/view.php?id=9860#c25506

<quote>On my HP N40L, that workaround does let it boot, but I don't really trust this. For instance, I found that reading sysfs current_clocksource crashed the system, even as an unprivileged user!

  $ cat /sys/devices/system/clocksource/clocksource0/current_clocksource

The crash is easy to understand -- clocksource_done_booting() is supposed to set pointer curr_clocksource, and sysfs_show_current_clocksources() dereferences it. 
</quote>

Comment 17 Prarit Bhargava 2016-01-27 12:44:35 UTC
(In reply to Tru Huynh from comment #16)
> from the CentOS bugs entry: https://bugs.centos.org/view.php?id=9860#c25506
> 
> <quote>On my HP N40L, that workaround does let it boot, but I don't really
> trust this. For instance, I found that reading sysfs current_clocksource
> crashed the system, even as an unprivileged user!
> 
>   $ cat /sys/devices/system/clocksource/clocksource0/current_clocksource
> 
> The crash is easy to understand -- clocksource_done_booting() is supposed to
> set pointer curr_clocksource, and sysfs_show_current_clocksources()
> dereferences it. 
> </quote>

The workaround is only meant to get you through the initial install.  After the installation, upgrade to the latest functioning kernel, and then reboot.  A permanent fix for this issue will be available at some point in z-stream.  Please contact your support person for further details.

P.