Bug 1980928 - Constant "watchdog: BUG: soft lockup - CPU#1" after upgrade from 5.10 to 5.11 kernel
Summary: Constant "watchdog: BUG: soft lockup - CPU#1" after upgrade from 5.10 to 5.11...
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 34
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-07-09 22:26 UTC by Andre Klapper
Modified: 2022-03-19 13:26 UTC (History)
19 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-19 13:26:14 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Kernel log from: journalctl --no-hostname -k > dmesg.txt (128.09 KB, text/plain)
2021-07-09 22:26 UTC, Andre Klapper
no flags Details

Description Andre Klapper 2021-07-09 22:26:32 UTC
Created attachment 1800132 [details]
Kernel log from: journalctl --no-hostname -k > dmesg.txt

=====Problem description:

After kernel update, F33 and F34 show many "soft lockup" errors and get stuck in booting up, running, shutting down; leading to component failures.

=====Did it work previously / kernel version

It does work previously with `5.10.23-200.fc33.x86_64`.
First problematic kernel version is `5.11.7-200.fc33.x86_64`.

=====Reproducibility:

Always

=====Steps to reproduce:

* Upgrade to latest Fedora 33 packages on that machine
* Restart

=====Actual outcome:

* When booting up, see several
`watchdog: BUG: soft lockup - CPU#0 stuck for 26s!` and
`watchdog: BUG: soft lockup - CPU#1 stuck for 26s!`
at random points, often resulting in further issues (random example: `Timed out waiting for device /dev/mapper/fedora_localhost--live-home`)
* If going into bootlevel 3, have quite some lag entering characters at prompt
* If getting into UI, very slow mouse movement and responsiveness in GNOME

=====Hardware:

Acer Aspire 1 A114-32-C6L7
$> uname -m
x86_64
$> uname -r
5.11.21-200.fc33.x86_64
$> sudo dmidecode
System Information
        Manufacturer: Acer
        Product Name: Aspire A114-32
        Version: V1.15
Base Board Information
        Mainfacturer: GLK
        Product Name: Bulbasaur_GL_S
        Version: V1.15
        Type: Motherboard
Processor Information
        Version: Intel(R) Celeron(R) N4000 CPU @ 1.10GHz

=====Reproducibility with other kernel versions:

* Problem also reproducible on F34's latest `kernel-5.12.14-300.fc34.x86_64`.
* Problem also reproducible running `Fedora-Workstation-Live-x86_64-34-1.2.iso` from a USB stick on that machine.

=====Running any modules not shipped with directly Fedora's kernel:

No such modules used.

=====Other information:
* Further info how to debug is welcome.
* Passing boot parameter `rcutree.use_softirq=0` did not help.
* Seems like most similar tickets lack progress or info (e.g. bug 1943866)

Comment 1 Andre Klapper 2021-07-16 13:57:30 UTC
Any hints how to further debug or track down? I'd like to update that machine to F34 at some point.

Comment 2 Justin M. Forbes 2021-07-16 14:43:44 UTC
I might first recommend trying a 5.13 kernel https://fedoraproject.org/wiki/Test_Day:2021-07-11_Kernel_5.13_Test_Week has instructions and links.  You can either try the full f34 ISO or just install the kernel. If that doesn't fix it, it might also be worth trying 5.14-rc1 https://koji.fedoraproject.org/koji/buildinfo?buildID=1780011 and post results. we can go from there.

Comment 3 Andre Klapper 2021-07-21 09:52:47 UTC
To be explicit: I should manually install the latest F35 kernel on an F33 system, correct?

(Currently travelling; will take a few weeks until I get hold of that machine again)

Comment 4 Andre Klapper 2021-09-03 19:03:31 UTC
Still same problem with `5.13.12-100.fc33` (latest kernel offer for F33).

Still same problem with manually installed `5.14.0-60.fc35`.

Comment 5 Andre Klapper 2022-03-19 13:26:14 UTC
Still same problem with latest/last kernel available for F33 `5.14.18-100.fc33`.

No problem anymore when using latest F34 kernel available `5.16.15-101.fc34` on Fedora 33.


Note You need to log in before you can comment on or make changes to this bug.