Bug 2402976 - rescue mode fails to mount /boot and /boot/efi
Summary: rescue mode fails to mount /boot and /boot/efi
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-blivet
Version: 43
Hardware: Unspecified
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Vojtech Trefny
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker
Depends On:
Blocks: F43FinalBlocker
TreeView+ depends on / blocked
 
Reported: 2025-10-10 04:49 UTC by Chris Murphy
Modified: 2025-10-17 22:35 UTC (History)
13 users (show)

Fixed In Version: python-blivet-3.12.1-9.fc43
Clone Of:
Environment:
Last Closed: 2025-10-17 22:35:14 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
syslog (615.95 KB, text/plain)
2025-10-10 04:49 UTC, Chris Murphy
no flags Details
program.log (967 bytes, text/plain)
2025-10-10 04:49 UTC, Chris Murphy
no flags Details
anaconda.log (36.52 KB, text/plain)
2025-10-10 04:50 UTC, Chris Murphy
no flags Details
storage.log (101.31 KB, text/plain)
2025-10-10 04:50 UTC, Chris Murphy
no flags Details
lvm.log (248.41 KB, text/plain)
2025-10-10 04:50 UTC, Chris Murphy
no flags Details
storage.state (12.00 KB, application/x-sqlite3)
2025-10-10 04:50 UTC, Chris Murphy
no flags Details
packaging.log (3.35 KB, text/plain)
2025-10-10 04:51 UTC, Chris Murphy
no flags Details
/etc/fstab (520 bytes, text/plain)
2025-10-10 17:40 UTC, Chris Murphy
no flags Details

Description Chris Murphy 2025-10-10 04:49:00 UTC
When choosing to rescue a fedora system (adds inst.rescue), following option 1 to continue, only root is mounted to /mnt/sysimage, neither the boot nor esp are mounted.

Reproducible: Always

Steps to Reproduce:
1. Boot Fedora-Server-dvd-x86_64-43-20251009.n.0.iso in qemu-kvm
2. Automatic installation to an empty disk
3. reboot to the server to test it boots and works ok
4. reboot Fedora-Server-dvd-x86_64-43-20251009.n.0.iso, choose troubleshooting menu, rescue option
5. option 1 to continue

Actual Results:
df, mount, lsblk - shows only the root LV is mounted, boot and esp are not mounted

Expected Results:
/mnt/sysimage/boot
/mnt/sysimage/boot/efi

should show up in df, and mount and lsblk

Comment 1 Chris Murphy 2025-10-10 04:49:38 UTC
Created attachment 2109242 [details]
syslog

Comment 2 Chris Murphy 2025-10-10 04:49:50 UTC
Created attachment 2109243 [details]
program.log

Comment 3 Chris Murphy 2025-10-10 04:50:02 UTC
Created attachment 2109244 [details]
anaconda.log

Comment 4 Chris Murphy 2025-10-10 04:50:15 UTC
Created attachment 2109245 [details]
storage.log

Comment 5 Chris Murphy 2025-10-10 04:50:25 UTC
Created attachment 2109246 [details]
lvm.log

Comment 6 Chris Murphy 2025-10-10 04:50:37 UTC
Created attachment 2109247 [details]
storage.state

Comment 7 Chris Murphy 2025-10-10 04:51:06 UTC
Created attachment 2109248 [details]
packaging.log

Comment 8 Fedora Blocker Bugs Application 2025-10-10 04:53:28 UTC
Proposed as a Blocker for 43-final by Fedora user chrismurphy using the blocker tracking app because:

 The rescue mode of the installer must start successfully and be able to detect and mount any installation performed according to the applicable criteria, and provide a shell with access to utilities capable of performing typical recovery operations. 

https://fedoraproject.org/wiki/Fedora_43_Beta_Release_Criteria#Rescue_mode

Comment 9 Adam Williamson 2025-10-10 05:58:07 UTC
Did it work in previous releases? I'm not actually sure this is implemented.

Comment 10 Adam Williamson 2025-10-10 06:01:09 UTC
This is what we do in rescue mode, and have for a long time:

    def _mount_and_prompt_for_shell(self):
        self._rescue.mount = True
        self._mount_root()
        self._show_result_and_prompt_for_shell()

and _mount_root, as its name indicates, only mounts the root partition (if it finds more than one it asks you which to mount).

I checked anaconda-40.22.3.9 as a random 'pretty old version' and it does the same. So I think this is more of a feature request than a bug.

Comment 11 Chris Murphy 2025-10-10 16:03:08 UTC
Huh. Maybe it never worked on UEFI? I'm certain (possibly certainly wrong) that it worked on BIOS way back when because it was a thing to have anaconda rescue mount all of it, and then run grub2-install grub2-mkconfig to fix the boot loader - which needed everything assembled and in a chroot for all those scripts to work.

Comment 12 Chris Murphy 2025-10-10 16:09:14 UTC
If it's working as designed, then I'd say close as notabug. And not treat the report as a feature request. My Magic 8 Ball proposes a different rescue adventure going forward anyway.

Comment 13 Adam Williamson 2025-10-10 17:10:31 UTC
ooh, wait. Now I dig into it more, I was wrong, I think.

There's a bit I overlooked in the `mount_root` bits in `rescue.py`. `RescueModeSpoke._mount_root` calls `self._rescue.mount_root`, which is `Rescue.mount_root`, which calls `self._device_tree_proxy.MountExistingSystemWithTask`, which - via a frankly *insane* pile of indirection - ultimately winds up as `pyanaconda.modules.storage.devicetree.root.mount_existing_system`, which is *basically* the same code that's existed all the way since e95be86ccde6c4997fd87729cc655c9072f865fd in 2009 , where it was introduced as `mountExistingSystem`. Along the way it got renamed, moved into blivet, then moved *back*, but it's always been around.

It works by parsing /etc/fstab . So...either your /boot and /boot/efi partitions aren't in /etc/fstab on the installed system...or /etc/fstab isn't in the root partition...or the parsing has broken, I guess?

Comment 14 Adam Williamson 2025-10-10 17:12:22 UTC
parse_fstab has some log lines in it, it'd be interesting to see if you can find any trace of these messages:

log.info("cannot open %s for read", fstab_path)
log.debug("blkid.tab devs: %s", list(blkid_tab.devices.keys()))
log_exception_info(log.info, "error parsing blkid.tab")
log.debug("crypttab maps: %s", list(crypt_tab.mappings.keys()))
log_exception_info(log.info, "error parsing crypttab")

Comment 15 Chris Murphy 2025-10-10 17:40:21 UTC
Created attachment 2109307 [details]
/etc/fstab

Comment 16 Chris Murphy 2025-10-10 17:59:48 UTC
(In reply to Adam Williamson from comment #14)
> parse_fstab has some log lines in it, it'd be interesting to see if you can
> find any trace of these messages:

None of those but found this:

storage.log:INFO:blivet:Fstab file '' does not exist, setting fstab read path to None

Comment 17 Adam Williamson 2025-10-10 18:04:59 UTC
hmm, interesting, but it's from blivet, so not sure whether it's relevant, need to trace things out further I guess.

Comment 18 Chris Murphy 2025-10-10 18:07:16 UTC
syslog:04:46:45,544 NOTICE kernel:XFS (vda2): Mounting V5 Filesystem 0207ac84-d5d5-46d5-9cdd-6b50c2c34aa3
syslog:04:46:45,809 NOTICE kernel:XFS (vda2): Unmounting Filesystem 0207ac84-d5d5-46d5-9cdd-6b50c2c34aa3
..
syslog:04:46:51,785 NOTICE kernel:XFS (vda2): Mounting V5 Filesystem 0207ac84-d5d5-46d5-9cdd-6b50c2c34aa3
syslog-04:46:51,814 INFO kernel:XFS (vda2): Ending clean mount

The boot volume is mounted and unmounted twice. None of the logs show an actual mount command though, that's a big missing item.

Comment 19 Kamil Páral 2025-10-13 16:46:11 UTC
Tested with F41 and F42 Server DVD media using the included Rescue boot option. Both F41 and F42 media mount the full filesystem, including /, /home/, /boot and /boot/efi. But the latest F43 image only mounts /, and nothing else. This clearly is a regression.

Comment 20 Lukas Ruzicka 2025-10-13 18:27:51 UTC
AGREED AcceptedFinalBlocker

Discussed at the 2025-10-13 (blocker / freeze exception) review meeting:

This is a clear regression in functionality and intentions. A system that is not fully mounted by the rescue operation is harder to fix. We consider the rescue mode criterion to be violated: https://fedoraproject.org/wiki/Fedora_43_Beta_Release_Criteria#Rescue_mode

https://meetbot-raw.fedoraproject.org//blocker-review_matrix_fedoraproject-org/2025-10-13/f43-blocker-review.2025-10-13-16.00.txt

Comment 21 Katerina Koukiou 2025-10-14 12:13:10 UTC
This appears to be a regression from the fstab handling move from anaconda to blivet in PR #6160 [1].
The issue might be in blivets FSTabManager initialization [2], there is even a comment that we need to change the fstab path in 'next fedora release'.

This would explain why F41/F42 worked (old anaconda fstab handling) but F43 broke (new blivet fstab handling).

[1] https://github.com/rhinstaller/anaconda/pull/6160
[2] https://github.com/storaged-project/blivet/blob/6738e77d7005f3ab2f4503be3482b45f86b42455/blivet/blivet.py#L57

Comment 23 Adam Williamson 2025-10-14 21:19:49 UTC
This fix does seem to work - it causes /boot and /home from the installed system to be mounted. I can't check ESP mounting because I can't use UEFI VMs on Silverblue.

Comment 24 Adam Williamson 2025-10-14 21:26:53 UTC
Unfortunately the patch doesn't easily backport to 3.12.1, so I can't do a quick build with the fix for this. I'd rather not put 3.13.0 in f43 unless it's *really* necessary.

Comment 25 Fedora Update System 2025-10-14 23:04:04 UTC
FEDORA-2025-372f201c05 (python-blivet-3.12.1-9.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-372f201c05

Comment 26 Fedora Update System 2025-10-15 01:18:06 UTC
FEDORA-2025-372f201c05 has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-372f201c05`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-372f201c05

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 27 Kamil Páral 2025-10-16 10:52:08 UTC
With F43 RC1.4 this seems fixed

Comment 28 Fedora Update System 2025-10-17 22:35:14 UTC
FEDORA-2025-372f201c05 (python-blivet-3.12.1-9.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.