Bug 2445750

Summary: nfsrahead crashed repeatedly with nfs-utils-2.8.6-0.fc44
Product: [Fedora] Fedora Reporter: Matt Fagnani <matt.fagnani>
Component: nfs-utilsAssignee: Steve Dickson <steved>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 44CC: luk.claes, plumerlis, steved
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: nfs-utils-2.8.7-0.fc43 nfs-utils-2.8.7-0.fc44 nfs-utils-2.8.7-0.fc42 Doc Type: ---
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2026-03-16 01:00:02 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Matt Fagnani 2026-03-09 12:53:43 UTC
I did a dnf offline-upgrade of a Fedora 44 KDE desktop installation with updates-testing enabled. The upgrade included nfs-utils-2.8.6-0.fc44. nfsrahead crashed repeatedly during the removal of one of the nfs-utils-2.8.5-0.rc3.fc44 subpackages. 

Mar 08 21:44:28 dnf5[1226]: [ 2/20] Prepare transaction             100% |  12.0   B/s |  18.0   B |  00m01s
Mar 08 21:44:28 dnf5[1226]: [ 3/20] Upgrading nfs-common-utils-1:2. 100% |   1.9 MiB/s | 390.5 KiB |  00m00s
Mar 08 21:44:28 dnf5[1226]: [ 4/20] Upgrading libnfsidmap-1:2.8.6-0 100% |   1.3 MiB/s | 173.3 KiB |  00m00s
Mar 08 21:44:28 dnf5[1226]: [ 5/20] Upgrading kwin-libs-0:6.6.2-2.f 100% |  38.9 MiB/s |   9.3 MiB |  00m00s
Mar 08 21:44:29 dnf5[1226]: [ 6/20] Upgrading kwin-common-0:6.6.2-2 100% |  17.8 MiB/s |  12.9 MiB |  00m01s
Mar 08 21:44:29 dnf5[1226]: [ 7/20] Upgrading nfsv4-client-utils-1: 100% | 163.0 KiB/s |  43.8 KiB |  00m00s
Mar 08 21:44:29 dnf5[1226]: [ 8/20] Upgrading nfsv3-client-utils-1: 100% | 800.1 KiB/s | 150.4 KiB |  00m00s
Mar 08 21:44:30 dnf5[1226]: [ 9/20] Upgrading nfs-client-utils-1:2. 100% | 855.0   B/s | 124.0   B |  00m00s
Mar 08 21:44:30 dnf5[1226]: [10/20] Upgrading nfs-utils-1:2.8.6-0.f 100% |   2.2 MiB/s | 617.8 KiB |  00m00s
Mar 08 21:44:30 dnf5[1226]: [11/20] Upgrading kwin-0:6.6.2-2.fc44.x 100% |   5.7 MiB/s |   2.0 MiB |  00m00s
Mar 08 21:44:30 dnf5[1226]: [12/20] Removing nfs-utils-1:2.8.5-0.rc 100% | 338.0   B/s |  62.0   B |  00m00s
Mar 08 21:44:30 dnf5[1226]: [13/20] Removing kwin-0:6.6.2-1.fc44.x8 100% |   1.6 KiB/s | 194.0   B |  00m00s
Mar 08 21:44:31 dnf5[1226]: [14/20] Removing nfs-client-utils-1:2.8 100% | 775.0   B/s | 100.0   B |  00m00s
Mar 08 21:44:31 dnf5[1226]: [15/20] Removing kwin-common-0:6.6.2-1. 100% |   6.6 KiB/s |   1.1 KiB |  00m00s
Mar 08 21:44:31 dnf5[1226]: [16/20] Removing nfsv4-client-utils-1:2 100% |  93.0   B/s |  12.0   B |  00m00s
Mar 08 21:44:31 audit[1337]: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:udev_t:s0-s0:c0.c1023 pid=1337 comm="nfsrahead" exe="/usr/libexec/nfsrahead" sig=6 res=1
Mar 08 21:44:31 kernel: audit: type=1701 audit(1773020671.526:50): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:udev_t:s0-s0:c0.c1023 pid=1337 comm="nfsrahead" exe="/usr/libexec/nfsrahead" sig=6 res=1
Mar 08 21:44:31 systemd-coredump[1338]: Process 1337 (nfsrahead) of user 0 terminated abnormally with signal 6/ABRT, processing...
Mar 08 21:44:31 systemd-coredump[1338]: Failed to connect to coredump service: No such file or directory
Mar 08 21:44:31 (udev-worker)[1336]: 251:0: Process '/usr/libexec/nfsrahead 251:0' terminated by signal ABRT.
Mar 08 21:44:31 (udev-worker)[1336]: 251:0: Failed to wait for spawned command '/usr/libexec/nfsrahead 251:0': Input/output error
Mar 08 21:44:31 (udev-worker)[1336]: 251:0: /usr/lib/udev/rules.d/99-nfs.rules:1 PROGRAM="/usr/libexec/nfsrahead %k": Failed to execute "/usr/libexec/nfsrahead 251:0": Input/output error
Mar 08 21:44:31 dnf5[1226]: [17/20] Removing nfsv3-client-utils-1:2 100% | 128.0   B/s |  20.0   B |  00m00s
Mar 08 21:44:31 dnf5[1226]: [18/20] Removing nfs-common-utils-1:2.8 100% | 324.0   B/s |  47.0   B |  00m00s
Mar 08 21:44:31 dnf5[1226]: [19/20] Removing libnfsidmap-1:2.8.5-0. 100% | 258.0   B/s |  23.0   B |  00m00s

coredumpctl showed that nfsrahead crashed five times with an error in glibc "double free or corruption (out)".

Core was generated by `/usr/libexec/nfsrahead 251:0'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0)
    at pthread_kill.c:44
44            return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0;
Missing rpms, try: dnf --enablerepo='*debug*' install libmount-debuginfo-2.41.3-12.fc44.x86_64 libblkid-debuginfo-2.41.3-12.fc44.x86_64 libselinux-debuginfo-3.10-1.fc44.x86_64 pcre2-debuginfo-10.47-1.fc44.1.x86_64
(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0)
    at pthread_kill.c:44
#1  0x00007f51171bba73 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:89
#2  0x00007f511716034e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007f51171477b3 in __GI_abort () at abort.c:77
#4  0x00007f5117148804 in __libc_message_impl (vma_name=vma_name@entry=0x7f51172fbeb1 "glibc: fatal", 
    fmt=fmt@entry=0x7f51172ff4bf "%s\n") at ../sysdeps/posix/libc_fatal.c:138
#5  0x00007f51171c5d4c in __libc_message_wrapper (vmaname=0x7f51172fbeb1 "glibc: fatal", fmt=0x7f51172ff4bf "%s\n")
    at ../include/stdio.h:203
#6  malloc_printerr (str=str@entry=0x7f5117301ea0 "double free or corruption (out)") at malloc.c:5359
#7  0x00007f51171c7750 in _int_free_merge_chunk (av=0x7f5117337ac0 <main_arena>, p=0x5579256f1bc0, size=93978807419008)
    at malloc.c:4438
#8  0x00007f51171c77c9 in _int_free_chunk (av=<optimized out>, p=<optimized out>, size=<optimized out>, 
    have_lock=<optimized out>, have_lock@entry=0) at malloc.c:4386
#9  0x00007f51171ca686 in __GI___libc_free (mem=<optimized out>) at malloc.c:3392
#10 0x00005579256ebd47 in free_device_info (di=0x7fff462e2bf0)
    at /usr/src/debug/nfs-utils-2.8.6-0.fc44.x86_64/tools/nfsrahead/main.c:73
#11 main (argc=<optimized out>, argv=0x7fff462e2d88)
    at /usr/src/debug/nfs-utils-2.8.6-0.fc44.x86_64/tools/nfsrahead/main.c:241

nfsrahead crashed repeatedly with the same types of traces and errors during the next two boots after the upgrade. I don't have any nfs partitions. This problem didn't happen with previous nfs-utils versions.


Reproducible: Sometimes

Steps to Reproduce:
1. In a Fedora 44 KDE desktop installation with nfs-utils-2.8.5-0.rc3.fc44 installed, run an offline upgrade including nfs-utils-2.8.6-0.fc44 with updates-testing enabled by
sudo dnf offline-upgrade download
sudo dnf offline-upgrade reboot
2.
3.
Actual Results:
nfsrahead crashed repeatedly with nfs-utils-2.8.6-0.fc44

Expected Results:
nfsrahead shouldn't have crashed.

Comment 1 plumlis 2026-03-11 02:00:20 UTC
same here

Comment 2 Steve Dickson 2026-03-12 13:25:51 UTC
Here is a scratch build [1] 

Could somebody test this since we (upstream and I) were not able
to reproduce the problem 

tia

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=143290490

Comment 3 Matt Fagnani 2026-03-12 16:00:42 UTC
free_device_info in frame 10 of the trace had a null pointer di->mountpoint which was freed in sfree(di->mountpoint);


(gdb) frame 10
#10 0x00005579256ebd47 in free_device_info (di=0x7fff462e2bf0)
    at /usr/src/debug/nfs-utils-2.8.6-0.fc44.x86_64/tools/nfsrahead/main.c:73
73              sfree(di->mountpoint);
(gdb) p di
$1 = (struct device_info *) 0x7fff462e2bf0
(gdb) p *di
$2 = {device_number = 0x55794423d030 "gssd", dev = 4096, mountpoint = 0x0, fstype = 0x0}
(gdb) p di->mountpoint
$3 = 0x0
(gdb) l
68      }
69
70
71      static void free_device_info(struct device_info *di)
72      {
73              sfree(di->mountpoint);
74              sfree(di->fstype);
75              sfree(di->device_number);
76      }
77

I upgraded to nfs-utils-2.8.6-0.rc1.fc44. I rebooted twice, and the nfsrahead crashes didn't happen as the partitions were mounted during boot as they did with every boot using nfs-utils-2.8.6-0.fc44. Journal messages like "nfsrahead[1038]: skipping non-NFS device 251:0"
were shown instead. Thanks.

Comment 4 Steve Dickson 2026-03-12 20:50:56 UTC
(In reply to Matt Fagnani from comment #3)
> free_device_info in frame 10 of the trace had a null pointer di->mountpoint
> which was freed in sfree(di->mountpoint);
> 
> 
> (gdb) frame 10
> #10 0x00005579256ebd47 in free_device_info (di=0x7fff462e2bf0)
>     at /usr/src/debug/nfs-utils-2.8.6-0.fc44.x86_64/tools/nfsrahead/main.c:73
> 73              sfree(di->mountpoint);
> (gdb) p di
> $1 = (struct device_info *) 0x7fff462e2bf0
> (gdb) p *di
> $2 = {device_number = 0x55794423d030 "gssd", dev = 4096, mountpoint = 0x0,
> fstype = 0x0}
> (gdb) p di->mountpoint
> $3 = 0x0
> (gdb) l
> 68      }
> 69
> 70
> 71      static void free_device_info(struct device_info *di)
> 72      {
> 73              sfree(di->mountpoint);
> 74              sfree(di->fstype);
> 75              sfree(di->device_number);
> 76      }
> 77
> 
> I upgraded to nfs-utils-2.8.6-0.rc1.fc44. I rebooted twice, and the
> nfsrahead crashes didn't happen as the partitions were mounted during boot
> as they did with every boot using nfs-utils-2.8.6-0.fc44. Journal messages
> like "nfsrahead[1038]: skipping non-NFS device 251:0"
> were shown instead. Thanks.

Thank you for your time and effort!

Comment 5 Fedora Update System 2026-03-14 17:57:32 UTC
FEDORA-2026-61d402814c (nfs-utils-2.8.7-0.fc44) has been submitted as an update to Fedora 44.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-61d402814c

Comment 6 Fedora Update System 2026-03-14 17:57:33 UTC
FEDORA-2026-a4c9143144 (nfs-utils-2.8.7-0.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-a4c9143144

Comment 7 Fedora Update System 2026-03-14 17:57:35 UTC
FEDORA-2026-49865a242f (nfs-utils-2.8.7-0.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2026-49865a242f

Comment 8 Fedora Update System 2026-03-15 01:31:06 UTC
FEDORA-2026-61d402814c has been pushed to the Fedora 44 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-61d402814c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-61d402814c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Fedora Update System 2026-03-15 01:55:14 UTC
FEDORA-2026-a4c9143144 has been pushed to the Fedora 42 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-a4c9143144`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-a4c9143144

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Fedora Update System 2026-03-15 02:16:03 UTC
FEDORA-2026-49865a242f has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-49865a242f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2026-49865a242f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Fedora Update System 2026-03-16 01:00:02 UTC
FEDORA-2026-49865a242f (nfs-utils-2.8.7-0.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 12 Fedora Update System 2026-03-22 00:17:24 UTC
FEDORA-2026-61d402814c (nfs-utils-2.8.7-0.fc44) has been pushed to the Fedora 44 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 13 Fedora Update System 2026-03-22 01:09:28 UTC
FEDORA-2026-a4c9143144 (nfs-utils-2.8.7-0.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.