Bug 2212012 - Fedora 38 workstation hangs on boot after upgrading to kernel 6.3.4
Summary: Fedora 38 workstation hangs on boot after upgrading to kernel 6.3.4
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 38
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL: https://discussion.fedoraproject.org/...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-06-03 01:27 UTC by Sean
Modified: 2023-06-19 12:16 UTC (History)
21 users (show)

Fixed In Version:
Doc Type: ---
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: ---
Embargoed:


Attachments (Terms of Use)
fpaste and cat /proc/sys/kernel/tainted output from my system (15.76 KB, text/plain)
2023-06-03 01:36 UTC, Sean
no flags Details
dmesg from kernel 6.2.15 (71.86 KB, text/plain)
2023-06-09 23:06 UTC, Sean
no flags Details

Description Sean 2023-06-03 01:27:52 UTC
I just updated my Fedora 38 workstation system to the latest kernel available (6.3.4) and installed all the other needed update packages via the GNOME software program. When the system finished installing the kernel and other packages and rebooted, it sent me to the GRUB screen. When I selected kernel 6.3.4 to boot, the system hung on a black screen with only an underscore (_) visible. I attempted to reboot the system with the new kernel several times but it hung every time. However, when I selected the previous kernel (6.2.15) in GRUB, the system booted normally (I am typing this post on the system in question while running kernel 6.2.15 now).

I am unsure if something became corrupted during the update process or if there is another issue. Also, if this information helps, my system is a Dell Inspiron 3891 desktop. It does not have a dedicated GPU and only uses the integrated Intel graphics (Intel UHD Graphics 730).
 
Reproducible: Always

Steps to Reproduce:
1.Turn on desktop
2.Select kernel 6.3.4 from GRUB screen

Actual Results:  
System hangs indefinitely

Expected Results:  
System boots into Fedora 38 workstation

This issue affects multiple users - please see https://bodhi.fedoraproject.org/updates/FEDORA-2023-514965dd8a

Also please see https://discussion.fedoraproject.org/t/fedora-hangs-on-boot-after-upgrading-to-kernel-6-3-4/83605

Comment 1 Sean 2023-06-03 01:36:13 UTC
Created attachment 1968675 [details]
fpaste and cat /proc/sys/kernel/tainted output from my system

Comment 3 Sean 2023-06-05 01:04:37 UTC
I just updated to kernel 6.3.5 via GNOME software and the same issue is still present. When attempting to boot kernel 6.3.5, the system hangs on a black screen with only an underscore (_) visible. I have attempted to reboot the system several times but kernel 6.3.5 does not boot. Kernel 6.2.15 still boots normally.

Comment 4 Christopher Klooz 2023-06-06 10:55:02 UTC
The user has `cat /proc/sys/kernel/tainted` = 0; seems to be Intel graphics. But just to be sure, he tested both 6.3.5-200 but also the 6.3.5-201 (the one for nvidia from BZ#2211784 )

The error seems to occur before root is mounted, so journalctl does not get logs of the issue in 6.3.X. However, the user provided some logs of the working 6.2.15 [1] to offer some indication.

Some more details in the ask.fedora topic [2]

[1] https://controlc.com/1e07d11b -> password: Pv&#4FoQ20%1
[2] https://discussion.fedoraproject.org/t/fedora-hangs-on-boot-after-upgrading-to-kernel-6-3-4/83605/34

Comment 5 Sean 2023-06-09 02:33:15 UTC
I just updated to kernel 6.3.6 via GNOME software and the same issue is still present. When attempting to boot kernel 6.3.6, the system hangs on a black screen with only an underscore (_) visible. I have attempted to reboot the system several times but kernel 6.3.6 does not boot. Kernel 6.2.15 still boots normally.

Comment 6 Sean 2023-06-09 23:06:00 UTC
Created attachment 1970103 [details]
dmesg from kernel 6.2.15

Comment 7 Carlos Guidugli 2023-06-11 00:39:30 UTC
Was having the same problem. After removing rd.driver.pre=vfio-pci it worked again.

Comment 8 Carlos Guidugli 2023-06-11 00:40:16 UTC
Was having the same problem. After removing rd.driver.pre=vfio-pci from /etc/default/grub, it worked again.

Comment 9 Sean 2023-06-11 14:56:13 UTC
(In reply to Carlos Guidugli from comment #8)
> Was having the same problem. After removing rd.driver.pre=vfio-pci from
> /etc/default/grub, it worked again.

Hi Carlos, thanks for your suggestion. Unfortunately that line does not exist in my grub file. Here are the complete contents of my /etc/default/grub file:

GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
GRUB_ENABLE_BLSCFG=true

I would appreciate any other suggestions in modifying this file or another to address my issue. Thanks!

Comment 10 Ernesto 2023-06-13 21:44:51 UTC
I can confirm that in 6.3.7 the problem still persists (tainted = 0, only Iris Xe graphics). 

Kernel 6.2.15 works ok.

Adding module_blacklist=ucsi_acpi to kernel parameters fixes the problem on my Asus Vivobook Alder Lake laptop. Other users report the same on the Fedora Discussion thread, for the same (or similar) model of Asus Vivobook laptop.

I also cannot get any logs from journalctl for a failed boot with 6.3.

- Asus Vivobook S 14X laptop
- Intel Core i7 12700H
- Iris Xe, no DGPU
- Fedora 38
- Kernels 6.3.4, 6.3.5, 6.3.7, 6.4-rc from Rawhide

Comment 11 Sean 2023-06-14 00:36:21 UTC
My original issue persists with kernel 6.3.7. Kernel 6.2.15 continues to be the latest working kernel on my system.

Comment 12 Aleks G 2023-06-14 10:57:45 UTC
I also can confirm that in 6.3.7-100 kernel the problem still exist, even "module_blacklist=ucsi_acpi" is added to bootload conf. 

Kernel 6.2.15-200 works fine.

Comment 13 Christopher Klooz 2023-06-15 17:47:37 UTC
This should be fixed in kernel 6.3.8 -> once you updated to 6.3.8, please try to boot without `module_blacklist=ucsi_acpi` and feel free to let us know if it works.

Thanks Hans & Justin for pushing this so quickly! 

--------------------

FEDORA-2023-0d0eb153c9 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-0d0eb153c9 

FEDORA-2023-0ea5b492c7 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2023-0ea5b492c7

Comment 14 Christopher Klooz 2023-06-15 17:58:11 UTC
To the users whose issues cannot be mitigated by `module_blacklist=ucsi_acpi`: expect that you have a different problem. Please check https://discussion.fedoraproject.org/tag/kernel if one of the topics there correlate to your issue and if not, open a new one. Then let's see there if it can be solved immediately or if it makes sense to open a separated bug report.

Comment 15 Ernesto 2023-06-15 22:43:05 UTC
I can confirm 6.3.8 (FEDORA-2023-0d0eb153c9) works on my Asus Vivobook S 14X laptop (without module_blacklist=ucsi_acpi).

Thanks Chris!

Comment 16 Aleks G 2023-06-16 09:56:07 UTC
I can confirm 6.3.8 (FEDORA-2023-0d0eb153c9) works on my Asus Vivobook S 14X laptop (without module_blacklist=ucsi_acpi).

Thank you Chris, Hans, Justin and other people for solving this issue!

Comment 17 Sean 2023-06-17 04:31:22 UTC
My original issue still persists with kernel 6.3.8. Kernel 6.2.15 continues to be the latest working kernel on my system.

Comment 18 Christopher Klooz 2023-06-19 12:16:28 UTC
Sean, since we now know that the ask.fedora topic and the elaborated problematic behavior refers to three different bugs while this bug report refers to two different bugs, I have opened a new ask.fedora topic to leave the blurred old one focused on the two issues it already contains. The new one is: https://discussion.fedoraproject.org/t/black-screen-only-on-top-left-on-boot-after-grub-menu-only-but-all-6-3-kernels-including-6-3-8-module-blacklist-ucsi-acpi-doesnt-mitigate/84471 . 

Since all logs/system-data in this bugzilla report is from you anyway, I assume it is not necessary to add a new bug report (unless the kernel maintainer team has a different opinion ;) But in the old ask.fedora topic it was no longer comprehensible which data belongs to which issue, and we should focus now on data and testing that starts with kernel 6.3.8 to ensure that the identified bugs' data no longer blurs the data produced by the yet unknown bug, just in case you and/or others have both the ucsi_acpi and the second yet unknown one (your hardware and system configuration seems to widely correlate to the ucsi_acpi affected users).

-------

Obviously, the ucsi_acpi issue has affected all non-nvidia-users who came up with this problem so far except the original author. The ucsi_acpi issue is solved with updating to 6.3.8 based upon all user feedback we have (bugzilla, ask.fedora, bodhi), but the original author of this bug report seems to still have his issue: neither 6.3.8 nor `module_blacklist=ucsi_acpi` solve his issue.

If we can make it in the new ask.fedora topic to get 6.3.8 booted to a level where it produces any output/logs, we will let you know here.


Note You need to log in before you can comment on or make changes to this bug.