Bug 2113005 - Live image made with BOOTX64.EFI from latest shim-x64-15.6-2 fails to boot on some boards
Summary: Live image made with BOOTX64.EFI from latest shim-x64-15.6-2 fails to boot on...
Keywords:
Status: POST
Alias: None
Product: Fedora
Classification: Fedora
Component: shim
Version: 36
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Peter Jones
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: https://ask.fedoraproject.org/t/commo...
: 2103785 (view as bug list)
Depends On:
Blocks: 2092065
TreeView+ depends on / blocked
 
Reported: 2022-08-01 16:56 UTC by Gilles Buloz
Modified: 2023-02-06 17:48 UTC (History)
20 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug


Attachments (Terms of Use)
Additional tests (9.49 KB, text/plain)
2022-08-23 14:19 UTC, Gilles Buloz
no flags Details
with debug enabled by mok on shimx64.efi 15.6 (55.77 KB, text/plain)
2022-08-23 15:17 UTC, Gilles Buloz
no flags Details
Auto boot from custom live image with mok debug (26.30 KB, text/plain)
2022-08-23 15:29 UTC, Gilles Buloz
no flags Details
boot from bootx64 from shell on custom live image with mok debug (54.00 KB, text/plain)
2022-08-23 15:31 UTC, Gilles Buloz
no flags Details
Auto boot from official F36 live image with mok debug (boot OK; have to press ENTER in debug messages) (18.20 KB, text/plain)
2022-08-23 15:39 UTC, Gilles Buloz
no flags Details
Test of test of shimx64.efi from shim-unsigned-x64-15.6-505.x86_64.rpm with secure boot disabled (131.99 KB, text/plain)
2022-08-26 17:07 UTC, Gilles Buloz
no flags Details

Description Gilles Buloz 2022-08-01 16:56:07 UTC
Description of problem:
On some boards (?), a custom live image made with /boot/efi/EFI/BOOT/BOOTX64.EFI from shim-x64-15.6-2.x86_64 fails to boot, but replacing the BOOTX64.EFI on this image with the one from shim-x64-15.4-5.x86_64 fixes the problem.
Booting a standard Fedora 36 Live image (tested Fedora-Xfce-Live-x86_64-36-1.5.iso) on such a board works OK, but this image is using shim-x64-15.4-5.x86_64.

Version-Release number of selected component (if applicable):
shim-x64-15.6-2.x86_64

How reproducible:
Easy with the right x86_64 board
Tested with a Kontron VX3040 board (Quad core Intel Core i7-3517UE) with UEFI AMI BIOS

Steps to Reproduce:
1.Rebuild a live image (tested XFce) using livemedia-creator from latest Fedora sources
2.Try to boot the image
3.An error is displayed

Actual results:
Invalid image 
Failed to read header: Unsupported 
Failed to load image: Unsupported 
start_image() returned Unsupported

Expected results:
GRUB successfully loaded

Additional info:
As workaround I can enter the UEFI shell, enter the filesystem (map -r + fs0:), then cd EFI\BOOT, then run GRUB manually with : GRUBX64.EFI <enter>
But running BOOTX64.EFI gives the same errors than when the BIOS is trying to boot automatically from boot list (see 'Actual results")
Strangely, once booted with the workaround above and installed to disk, the disk boots successfully even with this /boot/efi/EFI/BOOT/BOOTX64.EFI from shim-x64-15.6-2.x86_64

Comment 1 Robbie Harwood 2022-08-18 21:16:59 UTC
What if you try to boot shimx64.efi from the UEFI shell?

Comment 2 Gilles Buloz 2022-08-23 08:44:42 UTC
Boot fails also with shimx64.efi :

fs0:\EFI\FEDORA> SHIMX64.EFI
Invalid image
Failed to read header: Unsupported
Failed to load image: Unsupported
start_image() returned Unsupported
start_image() returned Unsupported

fs0:\EFI\FEDORA>

Comment 3 Gilles Buloz 2022-08-23 14:19:15 UTC
Created attachment 1907142 [details]
Additional tests

Comment 4 Gilles Buloz 2022-08-23 14:23:00 UTC
Additional infos : 
- SHIMX64.EFI is not present on EFI partition of live image; only once installed to disk (under EFI\FEDORA)
- Fedora 36 official live image can boot from \EFI\BOOT\BOOTX64.EFI
- Fedora 36 custom live image can not boot from \EFI\BOOT\BOOTX64.EFI but can boot from \EFI\BOOT\GRUBX64.EFI
- Once installed to disk, the custom Fedora 36 live image can not boot from \EFI\FEDORA\SHIMX64.EFI but can boot from \EFI\BOOT\BOOTX64.EFI
See attached f36gruberrdbg_3.txt (attachment 1907142 [details])

Comment 5 Gilles Buloz 2022-08-23 14:31:48 UTC
On Fedora 36 official live image, \EFI\BOOT\BOOTX64.EFI has same checksum and size (13999 / 928592) than the one in shim-x64-15.4-5.x86_64.rpm (fedora release)
On Fedora 36 custom live image and once installed to disk, \EFI\BOOT\BOOTX64.EFI has same checksum and size  (23754 / 946712) than the one in shim-x64-15.6-2.x86_64.rpm (fedora updates)

Comment 6 Robbie Harwood 2022-08-23 14:55:40 UTC
Can you try (in a booted system that works):

mokutil --set-verbosity true
mokutil --set-fallback-verbosity true

and then reboot into the image with 15.6?  This should result in more debug output.

Comment 7 Gilles Buloz 2022-08-23 15:17:51 UTC
Created attachment 1907157 [details]
with debug enabled by mok on shimx64.efi 15.6

Comment 8 Gilles Buloz 2022-08-23 15:18:23 UTC
OK, on a system running from a disk installed from the custom live image, I ran you commands and attempted to reboot from SHIMX64.EFI (from 15.6-2). See attachment 1907157 [details]

Comment 9 Gilles Buloz 2022-08-23 15:29:35 UTC
Created attachment 1907158 [details]
Auto boot from custom live image with mok debug

Comment 10 Gilles Buloz 2022-08-23 15:31:13 UTC
Created attachment 1907159 [details]
boot from bootx64 from shell on custom live image with mok debug

Comment 11 Gilles Buloz 2022-08-23 15:39:52 UTC
Created attachment 1907160 [details]
Auto boot from official F36 live image with mok debug (boot OK; have to press ENTER in debug messages)

Comment 12 Robbie Harwood 2022-08-23 16:12:32 UTC
Thanks.

I may have a fix: https://github.com/rhboot/shim/pull/505 .  If you're able to test unsigned shims, it would be helpful if you could give that a spin.

Comment 13 Gilles Buloz 2022-08-23 16:26:14 UTC
Ok, I would be glad to help for testing, but how can I do that ? Would you send me an updated shimx64.efi ?

Comment 14 Robbie Harwood 2022-08-25 18:47:26 UTC
Well, the tricky part is signing it, not building it.  You either need your own key enrolled to sign with, or secure boot enforcement disabled.

But if it helps, and you feel like running binaries linked by quasi-strangers on the internet, you can extract built, unsigned shims from /usr/share/shim of the RPM at: https://rharwood.fedorapeople.org/shim-505/shim-unsigned-x64-15.6-505.x86_64.rpm

Comment 15 Gilles Buloz 2022-08-26 17:07:59 UTC
Created attachment 1907921 [details]
Test of test of shimx64.efi from shim-unsigned-x64-15.6-505.x86_64.rpm with secure boot disabled

I've extracted the shimx64.efi from your shim-unsigned-x64-15.6-505.x86_64.rpm test package and copied it to an installed disk under EFI/fedora/shimx64_156505.efi. I also copied version 15.6-2 under shimx64_1562.efi and 15.4-5 under shimx64_1545.efi to compare.
Then I've tried to run them manually from EFI shell with secure boto disabled :
shimx64_1545.efi : OK (grub is run)
shimx64_1562.efi : FAILS
shimx64_156505.efi : OK

Comment 16 Robbie Harwood 2022-09-01 19:49:43 UTC
Thanks for the testing.  For anyone hitting this issue before Fedora's signed shim updates: in my opinion, it is better from a security perspective to run the older, signed shim rather than disabling secureboot to run the patched, unsigned shim linked above.  

(If you're able to enroll your own keys, of course you can have the best of both worlds; and if you're just testing fixes, none of this applies anyway.)

Comment 17 Frederick Grose 2022-09-06 21:56:30 UTC
I observe the same error messages booting Fedora 37 images on an Asus Z87-PLUS motherboard.
see https://bugzilla.redhat.com/show_bug.cgi?id=2103785

Substituting the Fedora 36 
Mar 25 20:37 grubx64.efi
May  5  2021 BOOTX64.EFI
files solves the problem.

Comment 18 Adam Williamson 2022-10-25 17:45:20 UTC
*** Bug 2103785 has been marked as a duplicate of this bug. ***

Comment 19 Adam Williamson 2022-10-25 17:46:17 UTC
It's a shame we have a fix for this but are releasing F37 with it :( Sorry, I kinda took my eye off this ball a bit. Robbie, when are we scheduled to do another signed shim build?

Comment 20 Kamil Páral 2022-11-10 11:43:46 UTC
I've created a CommonBugs description here:
https://ask.fedoraproject.org/t/common-issues/28364

Can somebody please proof-read it?
(I only tried to advise workarounds usable for a general user. Approaches like using the UEFI shell can power users easily discover when reading through this bug report).

Most importantly, does this problem only affect booting install media, or does it also affect upgraded systems (F36->F37)? IOW, can a system upgrade make your system unbootable?

Comment 21 Frederick Grose 2022-11-10 12:38:42 UTC
I observe the same error messages booting the latest Rawhide .iso, Fedora-Workstation-Live-x86_64-Rawhide-20221109.n.0.iso
on my Asus Z87-PLUS motherboard.

Comment 22 Adam Williamson 2022-11-10 16:27:59 UTC
I think most likely an upgrade could break it, yes, as the affected files are part of shim-signed and typically Fedora boots with /boot/efi mounted and writeable.

It's probably worth documenting the "replace the files with ones from the older shim package" workaround too, for folks who want to do a UEFI install, or are upgrading a UEFI system.

I wish I'd thought to make this bug a higher priority :| Maybe we could've at least just downgraded to the older working shim for F37, or something, even if we couldn't get a new signed build done in time for release. I hate to do it, but I'm actually gonna throw this on the blocker list for the go/no-go, at least to let people chew it over and raise visibility.

Comment 23 Adam Williamson 2022-11-10 17:33:44 UTC
Just realized I never actually mentioned this, but my Asus H97M-E - which I believe is likely of similar vintage to Frederick's Z87-PLUS - is also affected. So the total list of affected boards so far is:

* Kontron VX3040
* Asus Z87-PLUS
* Asus H97M-E

nirik also found this forum post by another affected person: https://ask.fedoraproject.org/t/f37-invalid-image-error-while-booting/26632/3

Comment 24 Adam Williamson 2022-11-10 18:05:48 UTC
So, good news on the upgrade front. I tested, and this bug does not actually seem to affect installed systems. If you install F36 and update it, you already get the 'bad' shim, because it's in F36 (and F35) updates as well as in F37. But the system still boots fine. If you then upgrade to F37, the system still boots fine.

So it seems, for some reason, this bug only affects the live boot scenario. I don't know why that would be, but it's good news. Robbie, does that make sense to you?

Comment 25 Ben Cotton 2022-11-10 18:19:13 UTC
In today's Go/No-Go meeting, we agreed to reject this on the basis that this bug appears to be limited to specific hardware and may only affect booting live images, so it does not rise to the level of a release blocker

https://meetbot.fedoraproject.org/fedora-meeting/2022-11-10/f37-final-go_no_go-meeting.2022-11-10-17.02.log.html#l-280

Comment 26 Tomas Dolezal 2022-11-12 13:50:02 UTC
Just reporting that my main computer with following mainboard is also affected. Not only live images but also netinst (Everything "variant") has this issue.

Asus Z97-DELUXE/USB 3.1 (bought 2015/07)

Comment 27 Frederick Grose 2022-11-16 13:56:35 UTC
This workaround allowed me to boot a LiveUSB:  (from https://www.reddit.com/r/RockyLinux/comments/r49er0/comment/hs6mll2/?utm_source=share&utm_medium=web2x&context=3)

In the ESP, delete the original BOOTX64.EFI and BOOTIA32.EFI. Then 
Rename grubx64.efi and grubia32.efi to BOOTX64.EFI and BOOTIA32.EFI respectively.

Comment 28 Adam Williamson 2022-11-16 16:42:00 UTC
That's not the best idea, because it will only work with Secure Boot disabled. The best workaround is the one documented at https://ask.fedoraproject.org/t/f37-install-media-dont-boot-in-uefi-mode-on-certain-motherboards/28364 - replacing BOOTX64.EFI with the one from an older package. You don't need to worry about BOOTIA32.EFI.

Comment 29 Gianluca 2022-11-24 18:33:03 UTC
Just for information, I have an MSI H81M-E33 and that is affected too.

Comment 30 Matthew Murrian 2022-12-07 16:35:40 UTC
Also affects to Acer E5-575.

Comment 31 Dan Horák 2022-12-11 13:18:08 UTC
for the record, Lenovo ThinkCentre E73 is also affected

RHEL 9 is suffering from the same issue it seems to me.

Comment 32 Jonathan Teh 2022-12-20 23:32:09 UTC
To add to the list, this also affects Asus AM1I-A (latest AMI UEFI BIOS 0801).

Comment 33 Brandon Nielsen 2022-12-22 19:06:10 UTC
Intel DH87RL is affected.

Comment 34 rx2178 2023-01-21 10:39:33 UTC
Gigabyte GA-6LXSV is affected.  Though, I was trying to boot RHEL 9.1 (dvd and live boot images).  I worked around this by deleteing BOOTX64.efi and renaming GRUBX64.efi in its place in the efi partition.

Comment 35 Maurice 2023-02-06 17:48:33 UTC
Adding to the list:

My DELL G5 5590 with its latest UEFI version 1.22.0 is also affected. Will be installing Fedora 36 and upgrading to 37 as a workaround. Annoying to learn that this is not getting fixed.


Note You need to log in before you can comment on or make changes to this bug.