Bug 2445171 - [nfs-utils] rc3 release introduces a performance regression in Cockpit
Summary: [nfs-utils] rc3 release introduces a performance regression in Cockpit
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: nfs-utils
Version: 43
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Steve Dickson
QA Contact: Fedora Extras Quality Assurance
URL: https://github.com/cockpit-project/co...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2026-03-06 12:22 UTC by Jelle van der Waa
Modified: 2026-03-22 01:09 UTC (History)
4 users (show)

Fixed In Version: nfs-utils-2.8.7-0.fc43 nfs-utils-2.8.7-0.fc44 nfs-utils-2.8.7-0.fc42
Clone Of:
Environment:
Last Closed: 2026-03-16 00:59:56 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Jelle van der Waa 2026-03-06 12:22:48 UTC
For the Cockpit storage page we have a test which creates 200 scsi disks to catch issues with kernel <-> udev <-> udisks.

Upgrading to 1:2.8.5-0.rc3.fc43 causes the disks to sequentially load and take a lot longer then our normal making the test fail.

Its hard for me to produce an exact reproducer but my suspicion is that the new `nfsrahead` process is causing udisks to receive udev events a lot later then normal.

Reproducible: Always

Steps to Reproduce:
1. modprobe scsi_debug num_tgts=200
2. notice a nfsrahead process spawned per disk?
3.



Additional Information:
nfsrahead is spawned from udev

[root@fedora-43-127-0-0-2-2201 ~]# pstree -sap 2897
systemd,1 --switched-root --system --deserialize=54
  └─systemd-udevd,648
      └─(udev-worker),2421
          └─nfsrahead,2897 70:64

And tons of nfsrahead processes are spawned

[root@fedora-43-127-0-0-2-2201 ~]# ps aux | grep nfs
root        2466  0.0  0.2   4040  2568 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:80
root        2467  0.0  0.2   4040  2680 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:0
root        2468  0.0  0.2   4040  2620 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:64
root        2469  0.0  0.2   4040  2788 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:32
root        2470  0.0  0.2   4040  2684 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:16
root        2471  0.0  0.2   4040  2788 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:48
root        2472  0.0  0.2   4040  2620 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:96
root        2473  0.0  0.2   4040  2620 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:112
root        2478  0.0  0.2   4040  2792 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:128
root        2481  0.0  0.2   4040  2596 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:144
root        2499  0.0  0.2   6560  2448 pts/0    S+   12:08   0:00 grep --color=auto nfs
[root@fedora-43-127-0-0-2-2201 ~]# ps aux | grep nfs
root        2466  0.0  0.2   4040  2568 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:80
root        2467  0.0  0.2   4040  2680 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:0
root        2468  0.0  0.2   4040  2620 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:64
root        2469  0.0  0.2   4040  2788 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:32
root        2470  0.0  0.2   4040  2684 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:16
root        2471  0.0  0.2   4040  2788 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:48
root        2472  0.0  0.2   4040  2620 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:96
root        2473  0.0  0.2   4040  2620 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:112
root        2478  0.0  0.2   4040  2792 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:128
root        2481  0.0  0.2   4040  2596 ?        S    12:08   0:00 /usr/libexec/nfsrahead 8:144

Comment 1 Fedora Update System 2026-03-08 18:58:02 UTC
FEDORA-2026-fb916a8366 (nfs-utils-2.8.6-0.fc44) has been submitted as an update to Fedora 44.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-fb916a8366

Comment 2 Fedora Update System 2026-03-08 18:58:09 UTC
FEDORA-2026-ebcf8a2337 (nfs-utils-2.8.6-0.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-ebcf8a2337

Comment 3 Fedora Update System 2026-03-08 18:58:15 UTC
FEDORA-2026-e033b6bafe (nfs-utils-2.8.6-0.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-e033b6bafe

Comment 4 Fedora Update System 2026-03-09 00:49:39 UTC
FEDORA-2026-fb916a8366 has been pushed to the Fedora 44 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-fb916a8366`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-fb916a8366

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 5 Fedora Update System 2026-03-09 01:21:37 UTC
FEDORA-2026-e033b6bafe has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-e033b6bafe`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-e033b6bafe

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Fedora Update System 2026-03-09 01:44:54 UTC
FEDORA-2026-ebcf8a2337 has been pushed to the Fedora 42 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-ebcf8a2337`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-ebcf8a2337

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Steve Dickson 2026-03-12 13:54:31 UTC
Here is a scratch build [1] 

Could somebody test this since we (upstream and I) were not able
to reproduce the problem 

tia

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=143290490

Comment 8 Jelle van der Waa 2026-03-12 14:31:35 UTC
I suspect to reproduce it you just need to create a scsi_debug device ie. modprobe scsi_debug num_tgts=1, as I package for Arch Linux I tested the two proposed patches on the mailing list and they seem to resolve the coredump, and I also suspect the performance regression:

https://gitlab.archlinux.org/archlinux/packaging/packages/nfs-utils/-/merge_requests/2/diffs

I've also tested the scratch build and can confirm our scaling test which creates 200 scsi disks passes now and I don't see coredumps anymore.

Comment 9 Steve Dickson 2026-03-12 14:36:34 UTC
(In reply to Jelle van der Waa from comment #8)
> I suspect to reproduce it you just need to create a scsi_debug device ie.
> modprobe scsi_debug num_tgts=1, as I package for Arch Linux I tested the two
> proposed patches on the mailing list and they seem to resolve the coredump,
> and I also suspect the performance regression:
> 
> https://gitlab.archlinux.org/archlinux/packaging/packages/nfs-utils/-/
> merge_requests/2/diffs
> 
> I've also tested the scratch build and can confirm our scaling test which
> creates 200 scsi disks passes now and I don't see coredumps anymore.

Thank you for your time and effort!

Comment 10 Fedora Update System 2026-03-14 17:57:26 UTC
FEDORA-2026-61d402814c (nfs-utils-2.8.7-0.fc44) has been submitted as an update to Fedora 44.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-61d402814c

Comment 11 Fedora Update System 2026-03-14 17:57:27 UTC
FEDORA-2026-a4c9143144 (nfs-utils-2.8.7-0.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-a4c9143144

Comment 12 Fedora Update System 2026-03-14 17:57:29 UTC
FEDORA-2026-49865a242f (nfs-utils-2.8.7-0.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-49865a242f

Comment 13 Fedora Update System 2026-03-15 01:31:00 UTC
FEDORA-2026-61d402814c has been pushed to the Fedora 44 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-61d402814c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-61d402814c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 14 Fedora Update System 2026-03-15 01:55:10 UTC
FEDORA-2026-a4c9143144 has been pushed to the Fedora 42 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-a4c9143144`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-a4c9143144

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 15 Fedora Update System 2026-03-15 02:15:58 UTC
FEDORA-2026-49865a242f has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-49865a242f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-49865a242f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 16 Fedora Update System 2026-03-16 00:59:56 UTC
FEDORA-2026-49865a242f (nfs-utils-2.8.7-0.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 17 Fedora Update System 2026-03-22 00:17:19 UTC
FEDORA-2026-61d402814c (nfs-utils-2.8.7-0.fc44) has been pushed to the Fedora 44 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 18 Fedora Update System 2026-03-22 01:09:23 UTC
FEDORA-2026-a4c9143144 (nfs-utils-2.8.7-0.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.