Bug 2402976

Summary: rescue mode fails to mount /boot and /boot/efi
Product: [Fedora] Fedora Reporter: Chris Murphy <bugzilla>
Component: python-blivetAssignee: Vojtech Trefny <vtrefny>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 43CC: a.badger, alpha, anaconda-maint, awilliam, blivet-maint-list, kkoukiou, kparal, lruzicka, mkolman, robatino, rvykydal, vtrefny, w
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard: AcceptedBlocker
Fixed In Version: python-blivet-3.12.1-9.fc43 Doc Type: ---
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2025-10-17 22:35:14 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2324225    
Attachments:
Description Flags
syslog
none
program.log
none
anaconda.log
none
storage.log
none
lvm.log
none
storage.state
none
packaging.log
none
/etc/fstab none

Description Chris Murphy 2025-10-10 04:49:00 UTC
When choosing to rescue a fedora system (adds inst.rescue), following option 1 to continue, only root is mounted to /mnt/sysimage, neither the boot nor esp are mounted.

Reproducible: Always

Steps to Reproduce:
1. Boot Fedora-Server-dvd-x86_64-43-20251009.n.0.iso in qemu-kvm
2. Automatic installation to an empty disk
3. reboot to the server to test it boots and works ok
4. reboot Fedora-Server-dvd-x86_64-43-20251009.n.0.iso, choose troubleshooting menu, rescue option
5. option 1 to continue

Actual Results:
df, mount, lsblk - shows only the root LV is mounted, boot and esp are not mounted

Expected Results:
/mnt/sysimage/boot
/mnt/sysimage/boot/efi

should show up in df, and mount and lsblk

Comment 1 Chris Murphy 2025-10-10 04:49:38 UTC
Created attachment 2109242 [details]
syslog

Comment 2 Chris Murphy 2025-10-10 04:49:50 UTC
Created attachment 2109243 [details]
program.log

Comment 3 Chris Murphy 2025-10-10 04:50:02 UTC
Created attachment 2109244 [details]
anaconda.log

Comment 4 Chris Murphy 2025-10-10 04:50:15 UTC
Created attachment 2109245 [details]
storage.log

Comment 5 Chris Murphy 2025-10-10 04:50:25 UTC
Created attachment 2109246 [details]
lvm.log

Comment 6 Chris Murphy 2025-10-10 04:50:37 UTC
Created attachment 2109247 [details]
storage.state

Comment 7 Chris Murphy 2025-10-10 04:51:06 UTC
Created attachment 2109248 [details]
packaging.log

Comment 8 Fedora Blocker Bugs Application 2025-10-10 04:53:28 UTC
Proposed as a Blocker for 43-final by Fedora user chrismurphy using the blocker tracking app because:

 The rescue mode of the installer must start successfully and be able to detect and mount any installation performed according to the applicable criteria, and provide a shell with access to utilities capable of performing typical recovery operations. 

https://fedoraproject.org/wiki/Fedora_43_Beta_Release_Criteria#Rescue_mode

Comment 9 Adam Williamson 2025-10-10 05:58:07 UTC
Did it work in previous releases? I'm not actually sure this is implemented.

Comment 10 Adam Williamson 2025-10-10 06:01:09 UTC
This is what we do in rescue mode, and have for a long time:

    def _mount_and_prompt_for_shell(self):
        self._rescue.mount = True
        self._mount_root()
        self._show_result_and_prompt_for_shell()

and _mount_root, as its name indicates, only mounts the root partition (if it finds more than one it asks you which to mount).

I checked anaconda-40.22.3.9 as a random 'pretty old version' and it does the same. So I think this is more of a feature request than a bug.

Comment 11 Chris Murphy 2025-10-10 16:03:08 UTC
Huh. Maybe it never worked on UEFI? I'm certain (possibly certainly wrong) that it worked on BIOS way back when because it was a thing to have anaconda rescue mount all of it, and then run grub2-install grub2-mkconfig to fix the boot loader - which needed everything assembled and in a chroot for all those scripts to work.

Comment 12 Chris Murphy 2025-10-10 16:09:14 UTC
If it's working as designed, then I'd say close as notabug. And not treat the report as a feature request. My Magic 8 Ball proposes a different rescue adventure going forward anyway.

Comment 13 Adam Williamson 2025-10-10 17:10:31 UTC
ooh, wait. Now I dig into it more, I was wrong, I think.

There's a bit I overlooked in the `mount_root` bits in `rescue.py`. `RescueModeSpoke._mount_root` calls `self._rescue.mount_root`, which is `Rescue.mount_root`, which calls `self._device_tree_proxy.MountExistingSystemWithTask`, which - via a frankly *insane* pile of indirection - ultimately winds up as `pyanaconda.modules.storage.devicetree.root.mount_existing_system`, which is *basically* the same code that's existed all the way since e95be86ccde6c4997fd87729cc655c9072f865fd in 2009 , where it was introduced as `mountExistingSystem`. Along the way it got renamed, moved into blivet, then moved *back*, but it's always been around.

It works by parsing /etc/fstab . So...either your /boot and /boot/efi partitions aren't in /etc/fstab on the installed system...or /etc/fstab isn't in the root partition...or the parsing has broken, I guess?

Comment 14 Adam Williamson 2025-10-10 17:12:22 UTC
parse_fstab has some log lines in it, it'd be interesting to see if you can find any trace of these messages:

log.info("cannot open %s for read", fstab_path)
log.debug("blkid.tab devs: %s", list(blkid_tab.devices.keys()))
log_exception_info(log.info, "error parsing blkid.tab")
log.debug("crypttab maps: %s", list(crypt_tab.mappings.keys()))
log_exception_info(log.info, "error parsing crypttab")

Comment 15 Chris Murphy 2025-10-10 17:40:21 UTC
Created attachment 2109307 [details]
/etc/fstab

Comment 16 Chris Murphy 2025-10-10 17:59:48 UTC
(In reply to Adam Williamson from comment #14)
> parse_fstab has some log lines in it, it'd be interesting to see if you can
> find any trace of these messages:

None of those but found this:

storage.log:INFO:blivet:Fstab file '' does not exist, setting fstab read path to None

Comment 17 Adam Williamson 2025-10-10 18:04:59 UTC
hmm, interesting, but it's from blivet, so not sure whether it's relevant, need to trace things out further I guess.

Comment 18 Chris Murphy 2025-10-10 18:07:16 UTC
syslog:04:46:45,544 NOTICE kernel:XFS (vda2): Mounting V5 Filesystem 0207ac84-d5d5-46d5-9cdd-6b50c2c34aa3
syslog:04:46:45,809 NOTICE kernel:XFS (vda2): Unmounting Filesystem 0207ac84-d5d5-46d5-9cdd-6b50c2c34aa3
..
syslog:04:46:51,785 NOTICE kernel:XFS (vda2): Mounting V5 Filesystem 0207ac84-d5d5-46d5-9cdd-6b50c2c34aa3
syslog-04:46:51,814 INFO kernel:XFS (vda2): Ending clean mount

The boot volume is mounted and unmounted twice. None of the logs show an actual mount command though, that's a big missing item.

Comment 19 Kamil Páral 2025-10-13 16:46:11 UTC
Tested with F41 and F42 Server DVD media using the included Rescue boot option. Both F41 and F42 media mount the full filesystem, including /, /home/, /boot and /boot/efi. But the latest F43 image only mounts /, and nothing else. This clearly is a regression.

Comment 20 Lukas Ruzicka 2025-10-13 18:27:51 UTC
AGREED AcceptedFinalBlocker

Discussed at the 2025-10-13 (blocker / freeze exception) review meeting:

This is a clear regression in functionality and intentions. A system that is not fully mounted by the rescue operation is harder to fix. We consider the rescue mode criterion to be violated: https://fedoraproject.org/wiki/Fedora_43_Beta_Release_Criteria#Rescue_mode

https://meetbot-raw.fedoraproject.org//blocker-review_matrix_fedoraproject-org/2025-10-13/f43-blocker-review.2025-10-13-16.00.txt

Comment 21 Katerina Koukiou 2025-10-14 12:13:10 UTC
This appears to be a regression from the fstab handling move from anaconda to blivet in PR #6160 [1].
The issue might be in blivets FSTabManager initialization [2], there is even a comment that we need to change the fstab path in 'next fedora release'.

This would explain why F41/F42 worked (old anaconda fstab handling) but F43 broke (new blivet fstab handling).

[1] https://github.com/rhinstaller/anaconda/pull/6160
[2] https://github.com/storaged-project/blivet/blob/6738e77d7005f3ab2f4503be3482b45f86b42455/blivet/blivet.py#L57

Comment 23 Adam Williamson 2025-10-14 21:19:49 UTC
This fix does seem to work - it causes /boot and /home from the installed system to be mounted. I can't check ESP mounting because I can't use UEFI VMs on Silverblue.

Comment 24 Adam Williamson 2025-10-14 21:26:53 UTC
Unfortunately the patch doesn't easily backport to 3.12.1, so I can't do a quick build with the fix for this. I'd rather not put 3.13.0 in f43 unless it's *really* necessary.

Comment 25 Fedora Update System 2025-10-14 23:04:04 UTC
FEDORA-2025-372f201c05 (python-blivet-3.12.1-9.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-372f201c05

Comment 26 Fedora Update System 2025-10-15 01:18:06 UTC
FEDORA-2025-372f201c05 has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-372f201c05`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-372f201c05

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 27 Kamil Páral 2025-10-16 10:52:08 UTC
With F43 RC1.4 this seems fixed

Comment 28 Fedora Update System 2025-10-17 22:35:14 UTC
FEDORA-2025-372f201c05 (python-blivet-3.12.1-9.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.