Bug 1826091 - System freezes after running dnf due to BFQ bug
Summary: System freezes after running dnf due to BFQ bug
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 32
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-20 20:55 UTC by Jordan Russell
Modified: 2021-05-25 17:19 UTC (History)
20 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2021-05-25 17:19:09 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1767539 0 high CLOSED BUG: kernel NULL pointer dereference RIP: 0010:rb_erase+0x1b1/0x370 2023-09-14 05:45:17 UTC

Description Jordan Russell 2020-04-20 20:55:48 UTC
On Fedora 32 Beta with its 5.6 kernel, the system freezes regularly due to unpatched bugs in the default BFQ scheduler. For me, the impact is severe: running basic "dnf install" or "dnf remove" commands results in a total system freeze about 10% of the time.

This issue, and particularly its impact on dnf, was previously noted on the kernel.org tracker by another user (Patrick Dung):
https://bugzilla.kernel.org/show_bug.cgi?id=205447#c70
I posted my experience, including how I'm able to reproduce it consistently, beginning in Comment 72.

As Comment 77 notes, fixes for BFQ were merged into 5.7.0, but they haven't (yet) been backported to 5.6. Please merge those fixes into the Fedora 32 5.6 kernel.

Given the severity of the problem (frequent freezes when running dnf), I believe this ought to be considered a blocking bug for the Fedora 32 final release.

Version-Release number of selected component (if applicable):
5.6.5-300 (and earlier, including 5.6.2)

How reproducible:
For me, 100%

Steps to Reproduce:
1. Create a new VMware VM with 1 CPU assigned.
   (NOTE: The freeze isn't consistently reproducible with >1 CPU.)
2. Install Fedora 32 Beta using the netinstall Server ISO.
   Choose "Minimum" software selection (may not be necessary).
3. Run this script, which installs and removes packages in a loop:

========================================
#!/bin/bash

set -e

while true; do
        dnf --setopt=keepcache=True -y install nano
        sleep 2
        dnf -y remove nano
        sleep 2
done
========================================

Actual results:
The VM consistently freezes in under a minute of running the script.

If I switch to the mq-deadline scheduler -OR- boot with kernel 5.7.0-rc1 (from Fedora 33 repo), the freeze no longer occurs.

Comment 1 Jordan Russell 2020-04-22 20:49:17 UTC
I posted a call trace of the crash here:
https://bugzilla.kernel.org/show_bug.cgi?id=205447#c87
Paolo Valente confirms that it should be fixed by the 5.7 patches (and again, I can't reproduce it on 5.7.0-rc1).


I should also point out that this issue with dnf freezing appears to be NEW in Fedora 32 -- or at least something has made it much worse.
On Fedora 31 with 5.5.17-200, my script never freezes (after 30 minutes of execution).

Comment 2 Jordan Russell 2020-04-24 20:04:23 UTC
The BFQ fixes should be in 5.6.7, released yesterday.
Will re-test when kernel-5.6.7 packages reach updates-testing.

Comment 3 Jordan Russell 2020-04-26 18:13:59 UTC
kernel-5.6.7-300 fixes it for me. No freeze running the same script for 30 minutes.

Comment 4 Andre Robatino 2020-05-14 17:41:35 UTC
I have one F32 machine that experienced the freeze repeatedly on the 5.6.6 kernel. After running the 5.6.7 and later kernels, it has never happened.

Comment 5 Fedora Program Management 2021-04-29 16:52:59 UTC
This message is a reminder that Fedora 32 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '32'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 32 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 6 Ben Cotton 2021-05-25 17:19:09 UTC
Fedora 32 changed to end-of-life (EOL) status on 2021-05-25. Fedora 32 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.