Bug 1903332 - Can't boot with kernel-5.9.10-200.fc33.x86_64 on Asus UX305CA/UX305CA
Summary: Can't boot with kernel-5.9.10-200.fc33.x86_64 on Asus UX305CA/UX305CA
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 33
Hardware: x86_64
OS: Unspecified
unspecified
urgent
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-12-01 20:17 UTC by Andreas Tunek
Modified: 2021-11-30 16:12 UTC (History)
24 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-30 16:12:42 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
journalctl ``-b`` output - 5.9.8 (630.34 KB, text/plain)
2020-12-01 20:17 UTC, Andreas Tunek
no flags Details
dmesg - 5.9.8 (74.62 KB, text/plain)
2020-12-03 19:53 UTC, Andreas Tunek
no flags Details

Description Andreas Tunek 2020-12-01 20:17:36 UTC
Created attachment 1735383 [details]
journalctl ``-b`` output - 5.9.8

1. Please describe the problem:
I can't boot with that version of Linux. Other versions selected in boot menu works fine.

2. What is the Version-Release number of the kernel:
kernel-5.9.10-200.fc33.x86_64

3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :
kernel-5.9.10-200.fc33.x86_64

kernel-5.9.8-200.fc33 works.


4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:
Do a system update. Computer will not boot.


5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:
Will not test. Sorry.


6. Are you running any modules that not shipped with directly Fedora's kernel?:
No.


7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.

Comment 1 Andreas Tunek 2020-12-01 20:20:37 UTC
Comment on attachment 1735383 [details]
journalctl ``-b`` output - 5.9.8

This is wrong kernel log.

Comment 2 Andreas Tunek 2020-12-01 20:21:36 UTC
I can't find any boot record from 5.9.10 boot so I guess it was not created....

Comment 3 Andreas Tunek 2020-12-03 19:30:53 UTC
I can't boot from 5.9.11-200.fc33 either.

Comment 4 Andreas Tunek 2020-12-03 19:49:33 UTC
kernel-5.9.12-200.fc33 is also affected.

Comment 5 Andreas Tunek 2020-12-03 19:53:46 UTC
Created attachment 1736166 [details]
dmesg - 5.9.8

Comment 6 Chris Murphy 2020-12-03 20:08:24 UTC
What happens if you edit the GRUB menu entry for 5.9.10/11 and remove 'quiet rhgb'? Does it get stuck somewhere? Cell video or photo of that boot scroll might reveal something, but right now there's nothing to go on. Doesn't boot is totally open ended - from this report I can't even tell if you get to a GRUB menu, so focus on what *does* happen.

Comment 7 Chris Murphy 2020-12-03 20:09:54 UTC
OK and dmesg logs for the kernel that works also doesn't tell us anything about why it's not booting with 5.9.10/11.

Comment 8 Andreas Tunek 2020-12-03 21:05:52 UTC
https://photos.app.goo.gl/J6EgG24T5PVmT5cXA

Video of bootable and non-bootable kernels/Linux. Sorry for the quality :)

Comment 9 Chris Murphy 2020-12-03 21:39:26 UTC
OK try again, in the video you still have 'rhgb' parameter and it needs to be removed to see the boot scroll.

Comment 10 Chris Murphy 2020-12-03 21:41:11 UTC
Also, it might be useful to know if the problem happens with 5.9.9.

Comment 11 Andreas Tunek 2020-12-04 15:46:36 UTC
Videos with correct (?) boot parameters: https://photos.app.goo.gl/kPdc4gmMQffyEP6D8

Comment 12 Andreas Tunek 2020-12-04 15:50:12 UTC
How do I install Linux 5.9.9?

Comment 13 Chris Murphy 2020-12-05 00:25:01 UTC
https://koji.fedoraproject.org upper right hand corner search for kernel, in the list find the version and click on it, and then make sure you find the correct arch and download the correct files. The minimum three are kernel, kernel-core, kernel-modules to boot, which is adequate for testing this bug. Download those rpms. And then you can just cd to the download location and 'dnf install *rpm' to install them.

https://koji.fedoraproject.org/koji/buildinfo?buildID=1643692

Comment 14 Chris Murphy 2020-12-05 00:39:06 UTC
(In reply to Andreas Tunek from comment #11)
> Videos with correct (?) boot parameters:
> https://photos.app.goo.gl/kPdc4gmMQffyEP6D8

OK so in the first video with kernel 5.9.12, it gets to EFI stub and that's it. We see nothing else. That's pretty clearly a kernel regression/bug but I have no idea what and there's still not much to go on. Best to narrow down the *first* stable kernel this fails in.

Is this the latest firmware available from the manufacturer? It might matter, even though this is clearly some kind of kernel regression. 

[    0.000000] DMI: ASUSTeK COMPUTER INC. UX305CA/UX305CA, BIOS UX305CA.205 02/22/2016

Next it would maybe be useful to know if this is already fixed in 5.10.

https://koji.fedoraproject.org/koji/buildinfo?buildID=1655402

This time, again you want x86_64, and the same three rpms I mentioned above. This may not boot by default after installation so make sure you F8 or however you get to the GRUB menu, to make sure you explicitly boot this kernel. If it works, chances are the problem has been discovered and fixed in the current mainline kernel, and will soon be backported to stable. If it's not fixed here, there's a good chance it's not yet discovered and it'll be worth the trouble to bisect. That might sound daunting but it's a clear and usually unambiguous way of finding the exact commit that causes this problem. And then upstream can be informed that they broke something, and they will get it fixed. But yeah, if no one tells them, it's not likely to just magically get fixed.

Comment 15 Didier G 2020-12-06 12:37:06 UTC
I have an Asus UX305UA too.

It boots fine with 5.8.x but it does not boot,black screen after grub choice, with:

kernel-5.9.11-100.fc32.x86_64
kernel-5.9.11-200.fc33.x86_64
kernel-5.9.12-200.fc33.x86_64

Comment 16 Didier G 2020-12-06 13:54:54 UTC
I tried kernel-5.9.1-300.fc33.x86_64 and I got same problem

I also tried to install kernel-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64 on fedora 33 but installation scriplet did not terminate.

Comment 17 Andreas Tunek 2020-12-06 17:58:49 UTC
It tried to install the rawhide kernel but got the following error:



sudo dnf update --enablerepo=rawhide kernel
Last metadata expiration check: 0:02:07 ago on Sun 06 Dec 2020 06:53:59 PM CET.
Dependencies resolved.
==================================================================================================
 Package            Arch       Version                                          Repository   Size
==================================================================================================
Installing dependencies:
 kernel             x86_64     5.10.0-0.rc6.20201204git34816d20f173.92.fc34     rawhide     140 k
 kernel-core        x86_64     5.10.0-0.rc6.20201204git34816d20f173.92.fc34     rawhide      35 M
 kernel-modules     x86_64     5.10.0-0.rc6.20201204git34816d20f173.92.fc34     rawhide      31 M

Transaction Summary
==================================================================================================
Install  3 Packages

Total size: 67 M
Installed size: 107 M
Is this ok [y/N]: y
Downloading Packages:
[SKIPPED] kernel-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64.rpm: Already downloaded     
[SKIPPED] kernel-core-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64.rpm: Already downloaded
[SKIPPED] kernel-modules-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64.rpm: Already downloaded
warning: /var/cache/dnf/rawhide-2d95c80a1fa0a67d/packages/kernel-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 45719a39: NOKEY
Fedora - Rawhide - Developmental packages for the next Fedora rel 1.6 MB/s | 1.6 kB     00:00    
GPG key at file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-33-x86_64 (0x9570FF31) is already installed
The GPG keys listed for the "Fedora - Rawhide - Developmental packages for the next Fedora release" repository are already installed but they are not correct for this package.
Check that the correct key URLs are configured for this repository.. Failing package is: kernel-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64
 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-33-x86_64
Public key for kernel-core-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64.rpm is not installed. Failing package is: kernel-core-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64
 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-33-x86_64
Public key for kernel-modules-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64.rpm is not installed. Failing package is: kernel-modules-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64
 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-33-x86_64
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: GPG check FAILED


Can you solve this somehow?

Comment 18 Chris Murphy 2020-12-06 21:33:57 UTC
I just download the three RPMs from koji using Firefox, and then:

cd ~/Downloads
sudo dnf install *rpm

That installs all of these without error.
-rw-r--r--. 1 chris chris   142977 Dec  4 18:28  kernel-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64.rpm
-rw-r--r--. 1 chris chris 36850413 Dec  4 18:29  kernel-core-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64.rpm
-rw-r--r--. 1 chris chris 32860133 Dec  4 18:29  kernel-modules-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64.rpm

Comment 19 Andreas Tunek 2020-12-07 06:25:00 UTC
I tried the rawhide kernel from a liveUSB and it couldn't boot either.

Comment 20 Didier G 2020-12-07 19:53:50 UTC
I also did a test with kernel-5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64

dnf upgrade time to install this kernel is very long but it finish !

With this kernel 5.10 I get the same black screen so if I resume situation on my Asus UX305UA:

- 5.8.16-200.fc32.x86_64 works fine
- 5.9.1-300.fc33.x86_64 introduces the problem
- 5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64 does not fix it

Comment 21 Chris Murphy 2020-12-07 20:32:13 UTC
Well at this point it's a bug hunt. The way I would go about this is cloning the stable git repo of the kernel, and do a bisect between the two dot releases (last good, first bad). It'll let you use version tags instead of commit hash. Once the bad commit is found we'll know what list to go complain about breaking boot.

I can't tell if Didier and Andreas are even having the same problem on the same hardware because you're each reporting different versions for the first bad kernel.

Comment 22 Didier G 2020-12-07 21:49:59 UTC
I double checked and I confirm on my Asus UX305UA with both 5.9.8 and 5.9.1 I get a black screen.

Comment 23 Chris Murphy 2020-12-07 22:32:19 UTC
Andreas reports a UX305CA. Given different models and different kernel version breakage, I think they are separate bugs even if they manifest the same. But in either case, I think the way forward is doing a kernel bisect. The more rc's come out for 5.10 that don't already have the problem fixed, tells me it's not on upstream's radar yet.

Comment 24 Didier G 2020-12-08 00:04:09 UTC
I fixed my problem on Asus UX305UA with an upgrade of BIOS
from:
DMI: ASUSTeK COMPUTER INC. UX305UA/UX305UA, BIOS UX305UA.201 10/12/2015 (BAD)
to:
DMI: ASUSTeK COMPUTER INC. UX305UA/UX305UA, BIOS UX305UA.302 04/18/2019 (GOOD)

I can boot now on 5.9.x and 5.10.x with no problem.

Thanks for your efforts.

Comment 25 Chris Murphy 2020-12-08 02:33:13 UTC
Good to know, thanks for the report. I suggested doing that in c14, but I'm not sure if Andreas has had a chance to see if a newer firmware is available and if it helps.

Comment 26 Didier G 2020-12-08 07:40:11 UTC
(In reply to Chris Murphy from comment #14)
> Is this the latest firmware available from the manufacturer? It might
> matter, even though this is clearly some kind of kernel regression. 
> 
> [    0.000000] DMI: ASUSTeK COMPUTER INC. UX305CA/UX305CA, BIOS UX305CA.205
> 02/22/2016

According Asustek site at https://www.asus.com/us/Laptops/ASUS-ZenBook-UX305CA/HelpDesk_BIOS/ it seem latest BIOS for UX305CA is Version 304 2019/06/03

Hope this helps to fix Andreas problem.

Comment 27 Pavel 2020-12-10 07:03:55 UTC
Same here,last working kernel 5.9.8-200.fc33.x86_64, after that all kernels get stuck right away so not sure it creates any logs.
Laptop computer Asus UX303U.

Comment 28 Andreas Tunek 2020-12-11 08:03:32 UTC
How do you install the new BIOS?

Comment 29 Didier G 2020-12-11 20:11:17 UTC
(In reply to Andreas Tunek from comment #28)
> How do you install the new BIOS?

Il your machine has a dual boot Fedora and Windows the easiest way to upgrade BIOS is to install Asus BIOS-Utilities for your version of Windows from this page: https://www.asus.com/us/Laptops/ASUS-ZenBook-UX305CA/HelpDesk_Download/

Comment 30 Andreas Tunek 2020-12-12 22:35:33 UTC
I do not have Windows installed on this device, is it possible to udate the BIOS any other way?

Comment 31 Didier G 2020-12-13 11:44:49 UTC
(In reply to Andreas Tunek from comment #30)
> I do not have Windows installed on this device, is it possible to udate the
> BIOS any other way?

Download User's manual for UX305CA from https://www.asus.com/us/Laptops/ASUS-ZenBook-UX305CA/HelpDesk_Manual/ and go to chapter "To update the BIOS".

In short, you have to:

- download new BIOS form https://www.asus.com/us/Laptops/ASUS-ZenBook-UX305CA/HelpDesk_BIOS/
- write it on an USB stick
- boot and access to BIOS
- choose the right option to update

Comment 32 Andreas Tunek 2020-12-13 21:35:09 UTC
Installed new bios and everything seems to work. Thanks for the help and instructions!

Comment 33 Hans de Goede 2020-12-14 08:44:48 UTC
Although it is good that the BIOS update fixes this, we cannot just go and assume that everyone will update their BIOS and since 5.9.8 did work, this clearly is a kernel regression and we really should fix it.

Can someone who is seeing this (Pavel?) git bisect this ?  It should not be much work since things broke between 5.9.8 and 5.9.9 so there should only be a few bisect iterations to find the culprit. I can provide detailed instructions if necessary.

Note maybe first trt 5.9.14 and 5.10 before doing the bisect:

https://koji.fedoraproject.org/koji/buildinfo?buildID=1657548
https://koji.fedoraproject.org/koji/buildinfo?buildID=1655402

See here for instructions for installing a kernel directly from koji:
https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt

Note since these are official builds there is on need to disable secure-boot.

Comment 34 Pavel 2020-12-14 12:10:47 UTC
Sorry, I am not sure what git bisect means. Only reported my bug.

Tried to follow your instructions for trying 5.9.14 and 5.10

5.9.14-200.fc33.x86_64
still no boot

5.10.0-0.rc6.90.fc34.x86_64
5.10.0-0.rc6.20201204git34816d20f173.92.fc34.x86_64
not sure I tried the right packages but the two I tried just hung when trying to install (rpm -ivh...)

Comment 35 Chris Murphy 2020-12-14 17:38:05 UTC
I'm not seeing any comment mention whether the regression starts with 5.9.9. Best to start there. And then bisect.

Comment 36 Andreas Tunek 2020-12-17 07:06:31 UTC
I would love to help but since I updated the bios I guess I can't. So therefore I closed it.

Comment 37 Pavel 2020-12-18 02:33:22 UTC
kernel 5.9.15-200.fc33.x86_64 seems to boot again (without updating BIOS). I will wait for proper update to confirm, this is testing kernel from koji.

Comment 38 Pavel 2020-12-18 07:53:20 UTC
To my previous comment, the kernel 5.9.15-200 seemed to boot on occasions but failed to do so consistently.
Updated my BIOS and all good now.

Comment 39 Sorin Sbarnea 2021-01-14 12:02:27 UTC
I got into the same problem on a Lenovo T460p which was upgraded to 33 some time ago. Still, the system never had any configuration changes that could explain failure to boot.

It worth noting that this was installed using legacy-BIOS disk partitioning, not UEFI. I took a screenshot and did a full reinstall.

Comment 40 Ben Cotton 2021-11-04 16:19:24 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 41 Ben Cotton 2021-11-30 16:12:42 UTC
Fedora 33 changed to end-of-life (EOL) status on 2021-11-30. Fedora 33 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.