Bug 2458109

Summary: (Emulated) USB boot is broken since kernel-7.0.0-0.rc7.55.fc45 in Fedora-Rawhide-20260409.n.0
Product: [Fedora] Fedora Reporter: Adam Williamson (Red Hat non-Fedora) <awilliam>
Component: kernelAssignee: Justin M. Forbes <jforbes>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: rawhideCC: acaringi, adscvr, airlied, cmdr, hans, hpa, jforbes, kernel-maint, linville, masami256, mchehab, nickolasjcarr, ptalbert, steved, suraj.ghimire7
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: openqa
Fixed In Version: Doc Type: ---
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2026-04-16 22:12:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Adam Williamson (Red Hat non-Fedora) 2026-04-14 01:12:06 UTC
1. Please describe the problem:

Since Fedora-Rawhide-20260409.n.0 , where we went from kernel-7.0.0-0.rc7.260408g3036cd0d3328.56.fc45 to kernel-7.0.0-0.rc7.55.fc45 , all openQA USB boot tests fail on Rawhide. Boot fails in dracut, claiming that /dev/root does not exist. openQA emulates USB boot via qemu, this is the full qemu command:

/usr/bin/qemu-system-x86_64 -device virtio-vga,edid=on,xres=1024,yres=768 -only-migratable -chardev ringbuf,id=serial0,logfile=serial0,logappend=on -serial chardev:serial0 -audiodev none,id=snd0 -device intel-hda -device hda-output,audiodev=snd0 -global isa-fdc.fdtypeA=none -m 4096 -machine q35,smm=on -cpu host -netdev user,id=qanet0,net=172.16.2.0/24 -device virtio-net,netdev=qanet0,mac=52:54:00:12:34:56 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -device qemu-xhci -device usb-tablet -smp 2 -enable-kvm -no-shutdown -vnc :113,share=force-shared -device virtio-serial -chardev pipe,id=virtio_console,path=virtio_console,logfile=virtio_console.log,logappend=on -device virtconsole,chardev=virtio_console,name=org.openqa.console.virtio_console -chardev pipe,id=virtio_console_user,path=virtio_console_user,logfile=virtio_console_user.log,logappend=on -device virtconsole,chardev=virtio_console_user,name=org.openqa.console.virtio_console_user -chardev socket,path=qmp_socket,server=on,wait=off,id=qmp_socket,logfile=qmp_socket.log,logappend=on -qmp chardev:qmp_socket -S -global driver=cfi.pflash01,property=secure,value=on -device virtio-scsi-pci,id=scsi0 -device qemu-xhci,id=xhci0 -blockdev driver=file,node-name=hd0-file,filename=/var/lib/openqa/pool/23/raid/hd0,cache.no-flush=on -blockdev driver=qcow2,node-name=hd0,file=hd0-file,cache.no-flush=on,discard=unmap -device virtio-blk,id=hd0-device,drive=hd0,serial=hd0 -blockdev driver=file,node-name=usbstick-overlay0-file,filename=/var/lib/openqa/pool/23/raid/usbstick-overlay0,cache.no-flush=on -blockdev driver=qcow2,node-name=usbstick-overlay0,file=usbstick-overlay0-file,cache.no-flush=on,discard=unmap -device usb-storage,id=usbstick-device,drive=usbstick-overlay0,bootindex=0 -drive id=pflash-code-overlay0,if=pflash,file=/var/lib/openqa/pool/23/raid/pflash-code-overlay0,unit=0,readonly=on -drive id=pflash-vars-overlay0,if=pflash,file=/var/lib/openqa/pool/23/raid/pflash-vars-overlay0,unit=1

2. What is the Version-Release number of the kernel:

kernel-7.0.0-0.rc7.55.fc45

3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :

Yes, see above. Good: kernel-7.0.0-0.rc7.260408g3036cd0d3328.56.fc45 . Bad: kernel-7.0.0-0.rc7.55.fc45

4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:

openQA reproduces it multiple times a day. I haven't had time to try locally yet.

5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:

Yes.

6. Are you running any modules that not shipped with directly Fedora's kernel?:

No.

7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.

Will see what I can get out of the dracut environment tomorrow, wanted to file this before I forgot.

Comment 1 Adam Williamson (Red Hat non-Fedora) 2026-04-14 01:17:50 UTC
Hmm, might be more than just USB, some other things stopped working in similar ways at the same time, like LVM thinp boot:

https://openqa.fedoraproject.org/tests/4562805#step/_console_wait_login/5

and software RAID boot:

https://openqa.fedoraproject.org/tests/4562753#step/_console_wait_login/11

also ARM disk image boot started failing at the same time which is likely also the same thing, though we have no video output at this point in boot so can't tell for sure:

https://openqa.fedoraproject.org/tests/4581771#step/_console_wait_login/8

Comment 2 Simon de Vlieger 2026-04-14 06:01:16 UTC
I'm seeing the same on hardware; here's an rdsosreport from a user: https://paste.centos.org/view/e7e75e5f

Comment 3 Simon de Vlieger 2026-04-14 07:03:33 UTC
As an aside (it was a red herring in my case) `multipathd` has failed starting with that kernel as well complaining about the kernel module for it not being loaded.

Comment 4 Adam Williamson (Red Hat non-Fedora) 2026-04-15 18:14:30 UTC
jcline suggests this may be the same as https://bugzilla.redhat.com/show_bug.cgi?id=2457183 , that seems plausible. https://bodhi.fedoraproject.org/updates/FEDORA-2026-16d5de8bc9 reverts the glibc change but failed its own gating tests, I'm trying to fix its tests now - https://src.fedoraproject.org/rpms/glibc/pull-request/151

Comment 5 Adam Williamson (Red Hat non-Fedora) 2026-04-16 22:12:31 UTC
Looks like this was indeed that bug; it's fixed in the newest Rawhide compose with the updated glibc.

*** This bug has been marked as a duplicate of bug 2457183 ***