Bug 2131852

Summary: iso-scan boot no longer works in F37 image
Product: [Fedora] Fedora Reporter: Petr Menšík <pemensik>
Component: dracutAssignee: dracut-maint-list
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 37CC: awilliam, bob, dracut-maint-list, fgrose, jamacku, jonathan, knossos456, laszlo.gombos, lnykryn, matthias, pvalena
Target Milestone: ---Keywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: AcceptedFreezeException
Fixed In Version: dracut-059-14.fc39 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-10-09 22:25:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2143447    
Attachments:
Description Flags
rdsosreport none

Description Petr Menšík 2022-10-03 22:44:39 UTC
Description of problem:
I have created a booting of ISO images from grub2 on my flash drive. It uses grub loop feature. I were able to boot this way fedora 36 image, both beta and release. But it does not work for Fedora 37 beta image I have tried, even after modifying correct paths.

It fails with:
[  203.245962] dracut-initqueue[633]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks:
[  203.248709] dracut-initqueue[633]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-label\x2fFedora-WS-Live-36_B-1-5.sh: "[ -e "/dev/disk/by-label/Fedora-WS-Live-36_B-1-5" ]"
[  203.252951] dracut-initqueue[633]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2froot.sh: "[ -e "/dev/root" ]"
[  203.255907] dracut-initqueue[633]: Warning: dracut-initqueue: starting timeout scripts
[  203.800515] dracut-initqueue[633]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks:
[  203.803572] dracut-initqueue[633]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-label\x2fFedora-WS-Live-36_B-1-5.sh: "[ -e "/dev/disk/by-label/Fedora-WS-Live-36_B-1-5" ]"

Here is F37 menuentry:
menuentry "Fedora-Workstation-Live-x86_64-37_Beta" {
  insmod ext2
  set isofile="/ISO/Fedora-Workstation-Live-x86_64-37_Beta-1.5.iso"
  search --fs-uuid --no-floppy --set=isodisk D025-5080
  loopback loop ($isodisk)$isofile
  linux (loop)/images/pxeboot/vmlinuz root=live:CDLABEL=Fedora-WS-Live-37_B-1-5 rd.live.image verbose iso-scan/filename=$isofile
  initrd (loop)/images/pxeboot/initrd.img
}


Version-Release number of selected component (if applicable):
# dracut-cmdline    
dracut-37 (Workstation Edition Prerelease) dracut-057-2.fc37
Using kernel command line parameters:  rd.driver.pre=btrfs   BOOT_IMAGE=(loop)/i
mages/pxeboot/vmlinuz root=live:CDLABEL=Fedora-WS-Live-37_B-1-5 rd.live.image ve
rbose iso-scan/filename=/ISO/Fedora-Workstation-Live-x86_64-37_Beta-1.5.iso cons
ole=tty1 console=ttyS0,115200
livenet: no url handler for CDLABEL=Fedora-WS-Live-37_B-1-5
root was live:CDLABEL=Fedora-WS-Live-37_B-1-5, is now live:/dev/disk/by-label/Fe
dora-WS-Live-37_B-1-5


How reproducible:
reliable

Steps to Reproduce:
1. Save f37 beta image to /ISO folder on flash drive, which has grub2 installed.
2. Use legacy mode bios booting
3. Use grub2 to load loopback and boot from it.
4. Fails with 

Actual results:

[  203.197208] dracut-initqueue[634]: Warning: /lib/dracut/hooks/initqueue/finis
hed/devexists-\x2fdev\x2froot.sh: "[ -e "/dev/root" ]"
[  203.200471] dracut-initqueue[634]: Warning: dracut-initqueue: starting timeou
t scripts
[  203.748847] dracut-initqueue[634]: Warning: dracut-initqueue: timeout, still 
waiting for following initqueue hooks:
[  203.752013] dracut-initqueue[634]: Warning: /lib/dracut/hooks/initqueue/finis
hed/devexists-\x2fdev\x2fdisk\x2fby-label\x2fFedora-WS-Live-37_B-1-5.sh: "[ -e "
/dev/disk/by-label/Fedora-WS-Live-37_B-1-5" ]"
[  203.757295] dracut-initqueue[634]: Warning: /lib/dracut/hooks/initqueue/finis
hed/devexists-\x2fdev\x2froot.sh: "[ -e "/dev/root" ]"
[  203.760706] dracut-initqueue[634]: Warning: dracut-initqueue: starting timeou
t scripts
[  203.762010] dracut-initqueue[634]: Warning: Could not boot.
         Starting dracut-emergency.…ce - Dracut Emergency Shell...
Warning: /dev/disk/by-label/Fedora-WS-Live-37_B-1-5 does not exist
Warning: /dev/root does not exist

Generating "/run/initramfs/rdsosreport.txt"

sh-5.1# ls -l /dev/disk/by-id/
total 0
lrwxrwxrwx 1 root root  9 Oct  3 22:30 usb-Kingston_DataTraveler_3.0_408D5CE5721
4E44179A85CBF-0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 Oct  3 22:30 usb-Kingston_DataTraveler_3.0_408D5CE5721
4E44179A85CBF-0:0-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Oct  3 22:30 usb-Kingston_DataTraveler_3.0_408D5CE5721
4E44179A85CBF-0:0-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 Oct  3 22:30 usb-Kingston_DataTraveler_3.0_408D5CE5721
4E44179A85CBF-0:0-part3 -> ../../sda3
sh-5.1# ls -l /dev/disk/by-label/
total 0
lrwxrwxrwx 1 root root 10 Oct  3 22:30 PhhData -> ../../sda1
lrwxrwxrwx 1 root root 10 Oct  3 22:30 PhhEFI-boot -> ../../sda3
sh-5.1# 

Expected results:
Just like with F36 it should be able to boot. It works fine with previous release image.

menuentry "Fedora-Workstation-Live-x86_64-36" {
  insmod ext2
  set isofile="/ISO/Fedora-Workstation-Live-x86_64-36-1.5.iso"
  search --fs-uuid --no-floppy --set=isodisk D025-5080
  loopback loop ($isodisk)$isofile
  linux (loop)/isolinux/vmlinuz root=live:CDLABEL=Fedora-WS-Live-36-1-5 rd.live.
image verbose iso-scan/filename=$isofile
  initrd (loop)/isolinux/initrd.img
}


Additional info:
# blkid
/dev/loop0: BLOCK_SIZE="2048" UUID="2022-09-06-23-54-46-00" LABEL="Fedora-WS-Liv
e-37_B-1-5" TYPE="iso9660" PTUUID="fc1bf257-f1dd-413d-aef2-f98ba0ff45fc" PTTYPE=
"gpt"
/dev/sda3: SEC_TYPE="msdos" LABEL_FATBOOT="PhhEFI-boot" LABEL="PhhEFI-boot" UUID
="B2EC-4779" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTU
UID="ccad76b3-4f9a-4758-a772-6f045ded28b1"
/dev/sda1: LABEL_FATBOOT="PhhData" LABEL="PhhData" UUID="D025-5080" BLOCK_SIZE="
512" TYPE="vfat" PARTLABEL="PhhName" PARTUUID="29611640-f0a1-4559-8025-f8cfeadcf
ce1"
/dev/sda2: PARTLABEL="BIOS boot partition" PARTUUID="3b19ea75-f92d-49e4-a6e0-729
83fd89a39"

I am not sure this is the best component, but final failure were reported by dracut-initqueue. So I report it here.

Comment 1 Petr Menšík 2022-10-03 22:45:49 UTC
# find /dev/disk
/dev/disk
/dev/disk/by-uuid
/dev/disk/by-uuid/B2EC-4779
/dev/disk/by-uuid/D025-5080
/dev/disk/by-label
/dev/disk/by-label/PhhEFI-boot
/dev/disk/by-label/PhhData
/dev/disk/by-partuuid
/dev/disk/by-partuuid/29611640-f0a1-4559-8025-f8cfeadcfce1
/dev/disk/by-partuuid/ccad76b3-4f9a-4758-a772-6f045ded28b1
/dev/disk/by-partuuid/3b19ea75-f92d-49e4-a6e0-72983fd89a39
/dev/disk/by-partlabel
/dev/disk/by-partlabel/EFI\x20system\x20partition
/dev/disk/by-partlabel/PhhName
/dev/disk/by-partlabel/BIOS\x20boot\x20partition
/dev/disk/by-id
/dev/disk/by-id/usb-Kingston_DataTraveler_3.0_408D5CE57214E44179A85CBF-0:0-part1
/dev/disk/by-id/usb-Kingston_DataTraveler_3.0_408D5CE57214E44179A85CBF-0:0-part3
/dev/disk/by-id/usb-Kingston_DataTraveler_3.0_408D5CE57214E44179A85CBF-0:0-part2
/dev/disk/by-id/usb-Kingston_DataTraveler_3.0_408D5CE57214E44179A85CBF-0:0
/dev/disk/by-diskseq
/dev/disk/by-diskseq/2
/dev/disk/by-diskseq/1
/dev/disk/by-path
/dev/disk/by-path/pci-0000:02:00.0-usb-0:1:1.0-scsi-0:0:0:0-part3
/dev/disk/by-path/pci-0000:02:00.0-usb-0:1:1.0-scsi-0:0:0:0-part1
/dev/disk/by-path/pci-0000:02:00.0-usb-0:1:1.0-scsi-0:0:0:0-part2
/dev/disk/by-path/pci-0000:02:00.0-usb-0:1:1.0-scsi-0:0:0:0

Comment 2 Petr Menšík 2022-10-03 22:47:26 UTC
Created attachment 1915845 [details]
rdsosreport

Comment 3 Petr Menšík 2022-10-03 23:17:23 UTC
Hmm, interesting. In F36 image by-label device is present, unlike in F37. Not sure what is the difference.

$ tree /dev/disk
/dev/disk
├── by-id
│   ├── dm-name-live-base -> ../../dm-1
│   ├── dm-name-live-rw -> ../../dm-0
│   ├── usb-Kingston_DataTraveler_3.0_408D5CE57214E44179A85CBF-0:0 -> ../../sda
│   ├── usb-Kingston_DataTraveler_3.0_408D5CE57214E44179A85CBF-0:0-part1 -> ../../sda1
│   ├── usb-Kingston_DataTraveler_3.0_408D5CE57214E44179A85CBF-0:0-part2 -> ../../sda2
│   └── usb-Kingston_DataTraveler_3.0_408D5CE57214E44179A85CBF-0:0-part3 -> ../../sda3
├── by-label
│   ├── Anaconda -> ../../dm-0
│   ├── Fedora-WS-Live-36-1-5 -> ../../loop0
│   ├── PhhData -> ../../sda1
│   └── PhhEFI-boot -> ../../sda3
├── by-partlabel
│   ├── BIOS\x20boot\x20partition -> ../../sda2
│   ├── EFI\x20system\x20partition -> ../../sda3
│   └── PhhName -> ../../sda1
├── by-partuuid
│   ├── 29611640-f0a1-4559-8025-f8cfeadcfce1 -> ../../sda1
│   ├── 3b19ea75-f92d-49e4-a6e0-72983fd89a39 -> ../../sda2
│   └── ccad76b3-4f9a-4758-a772-6f045ded28b1 -> ../../sda3
├── by-path
│   ├── pci-0000:02:00.0-usb-0:1:1.0-scsi-0:0:0:0 -> ../../sda
│   ├── pci-0000:02:00.0-usb-0:1:1.0-scsi-0:0:0:0-part1 -> ../../sda1
│   ├── pci-0000:02:00.0-usb-0:1:1.0-scsi-0:0:0:0-part2 -> ../../sda2
│   └── pci-0000:02:00.0-usb-0:1:1.0-scsi-0:0:0:0-part3 -> ../../sda3
└── by-uuid
    ├── 2022-05-04-21-36-58-00 -> ../../loop0
    ├── 626a3537-c047-4e1d-b6ad-a10855a3b5f9 -> ../../dm-1
    ├── B2EC-4779 -> ../../sda3
    └── D025-5080 -> ../../sda1

$ ls /run/initramfs/ -l
total 34
drwxr-xr-x. 5 root root 32768 Dec 31  1969 isoscan
lrwxrwxrwx. 1 root root    27 Oct  3 19:09 isoscandev -> /dev/disk/by-uuid/D025-5080
drwxr-xr-x. 1 root root  2048 May  4 17:36 live
lrwxrwxrwx. 1 root root    40 Oct  3 19:09 livedev -> /dev/disk/by-label/Fedora-WS-Live-36-1-5
drwxr-xr-x. 3 root root    60 Oct  3 19:09 log
drwxr-xr-x. 2 root root    40 Oct  3 19:09 squashfs
drwxr-xr-x. 2 root root    40 Oct  3 19:09 url-lib

Anyway, if you know more appropriate component for this issue, please move it to the right component.

Comment 4 Frederick Grose 2022-11-04 14:10:15 UTC
This menuentry booted fine for me:
menuentry 'Live Fedora 37 Beta' --class fedora --class gnu-linux --class gnu --class os {
    insmod part_gpt
    insmod btrfs
    set root='hd1,gpt4'
    set isolabel=Fedora-WS-Live-37_B-1-5
    set isofile=Fedora-Workstation-Live-x86_64-37_Beta-1.5.iso
    loopback loop ($root)/$isofile
    linux (loop)/images/pxeboot/vmlinuz iso-scan/filename=$isofile root=live:LABEL=$isolabel ro rd.live.image rd.live.debug
    initrd (loop)/images/pxeboot/initrd.img
}

Comment 5 bob 2022-12-20 02:10:55 UTC
I've just stumbled over what appears to be the same issue. It seems like iso-scan.sh is racing with udev(?) to access block devices before they've been populated. Testing with QEMU I find that an image only semi-reliably boots from a SATA device and I've yet to observe it boot successfully from a USB device. In both cases, upping the debug logging makes successful boots possible/more reliable

Comment 7 Adam Williamson 2023-10-05 16:07:35 UTC
*** Bug 2181989 has been marked as a duplicate of this bug. ***

Comment 8 Fedora Update System 2023-10-05 16:57:11 UTC
FEDORA-2023-a93b472cd3 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2023-a93b472cd3

Comment 9 Fedora Update System 2023-10-05 16:57:13 UTC
FEDORA-2023-c10e6a6049 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-c10e6a6049

Comment 10 Fedora Update System 2023-10-05 16:57:15 UTC
FEDORA-2023-f71507e281 has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-f71507e281

Comment 11 Fedora Update System 2023-10-05 20:11:14 UTC
FEDORA-2023-f71507e281 has been pushed to the Fedora 39 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-f71507e281`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-f71507e281

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2023-10-06 02:03:51 UTC
FEDORA-2023-a93b472cd3 has been pushed to the Fedora 37 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-a93b472cd3`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-a93b472cd3

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Fedora Update System 2023-10-06 02:54:26 UTC
FEDORA-2023-c10e6a6049 has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-c10e6a6049`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-c10e6a6049

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 14 Matthias Saou 2023-10-06 16:52:11 UTC
I can confirm that the fix worked for me on Fedora 37, 38 & 39 Beta (all affected).

Since the Fedora 37 & 38 live iso files will most probably never be updated, I created a quick and dirty shell script that can be used/adapted to locally "re-spin" them: https://gist.github.com/thias/22efc28196e26fa5ce7fecfc04e189f2

Comment 15 Adam Williamson 2023-10-06 19:15:24 UTC
+6 in https://pagure.io/fedora-qa/blocker-review/issue/1382 , marking accepted FE.

Comment 16 Adam Williamson 2023-10-06 19:16:21 UTC
Note, we do have semi-unofficial respins for the latest release at https://dl.fedoraproject.org/pub/alt/live-respins/ , so the next respins after the F38 update is pushed stable should have this fix.

Comment 17 Adam Williamson 2023-10-06 23:21:00 UTC
BTW, Matthias, could you give the updates positive karma if you tested them? That'll help us get them pushed out. thanks.

Comment 18 Fedora Update System 2023-10-09 01:55:34 UTC
FEDORA-2023-c10e6a6049 has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 19 Fedora Update System 2023-10-09 22:25:45 UTC
FEDORA-2023-f71507e281 has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 20 Fedora Update System 2023-10-21 01:26:02 UTC
FEDORA-2023-a93b472cd3 has been pushed to the Fedora 37 stable repository.
If problem still persists, please make note of it in this bug report.