Bug 1558793 - UEFI fallback fails to boot, system resets
Summary: UEFI fallback fails to boot, system resets
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: shim
Version: rawhide
Hardware: All
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Peter Jones
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
Depends On:
Blocks: ARMTracker
TreeView+ depends on / blocked
 
Reported: 2018-03-21 02:59 UTC by Paul Whalen
Modified: 2018-07-11 10:16 UTC (History)
4 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2018-07-11 10:16:40 UTC


Attachments (Terms of Use)

Description Paul Whalen 2018-03-21 02:59:40 UTC
Description of problem:
UEFI fallback fails to boot, system resets in a loop. 

Version-Release number of selected component (if applicable):
shim-13-5

Additional info:

From OpenQA on aarch64:

FSOpen: Open '\EFI\BOOT\BOOTAA64.EFI' Success
...
[0m[37m[40mSystem BootOrder not found.  Initializing defaults.
FSOpen: Open 'EFI' Success
FSOpen: Open 'fedora' Success
FSOpen: Open 'BOOTAA64.CSV' Success
FSOpen: Open '\EFI\fedora\BOOTAA64.CSV' Success

Reset System

BOOTAA64.CSV is a zero length file, also happens on x86_64.

Full logs:
https://openqa.stg.fedoraproject.org/tests/259978#step/_console_wait_login/22

Comment 1 Adam Williamson 2018-04-04 00:53:52 UTC
So, hum. We're basically poking about here:

https://github.com/rhboot/shim/blob/1ff4a36a23ac5c17144275ccb3e1e1061750a137/fallback.c#L1003

One thing I notice is that we hit this part:

console_print(L"Reset System\n");

and that does this:

gRT->ResetSystem(EfiResetCold, EFI_SUCCESS, 0, NULL);

note that's a *cold* reset. Exactly what that means for edk2-aarch64 I'm having fun trying to figure out, but it seems at least plausible that it does something similar to power off / power on. That's awkward for the scenario we're dealing with here: a VM with no persistent storage for EFI variables, which is expecting to boot successfully from the fallback path. I can see how it *could* lead to a loop like this - the fallback path boots, updates BootOrder, does a cold reset, BootOrder is gone again so we hit the fallback path, repeat ad infinitum...

If ever try_start_first_option() is hit and works, it'd break the loop, but clearly that's *not* happening in this case. I'm not sure if we're detecting the presence of a TPM and thus not even trying it, or trying it but it's failing (we'd need verbose messages to figure that out).

Here's a more detailed log extract:

[Bds]Booting UEFI Misc Device 3
 BlockSize : 512 
 LastBlock : 13FFFFF 
 BlockSize : 512 
 LastBlock : 1FFFFF 
 BlockSize : 512 
 LastBlock : 119B7FF 
FSOpen: Open '\EFI\BOOT\BOOTAA64.EFI' Success
[Bds] DevicePath expand: PciRoot(0x0)/Pci(0x5,0x0) -> PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x088DD077,0x800,0x64000)/\EFI\BOOT\BOOTAA64.EFI
[Security] 3rd party image[0] can be loaded after EndOfDxe: PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x088DD077,0x800,0x64000)/\EFI\BOOT\BOOTAA64.EFI.
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B FABC9040
Loading driver at 0x000F8501000 EntryPoint=0x000F8501120
Loading driver at 0x000F8501000 EntryPoint=0x000F8501120 
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF FABC9E98
FSOpen: Open '\EFI\BOOT\fbaa64.efi' Success
FSOpen: Open '\EFI\BOOT\fbaa64.efi' Success
[0m[37m[40mSystem BootOrder not found.  Initializing defaults.
FSOpen: Open 'EFI' Success
FSOpen: Open 'fedora' Success
FSOpen: Open 'BOOTAA64.CSV' Success
FSOpen: Open '\EFI\fedora\BOOTAA64.CSV' Success

Reset System

(then the whole thing happens again, over and over, till the test gives up and dies).

Comment 2 Adam Williamson 2018-04-04 00:54:55 UTC
For now I'm gonna see if I can possibly tweak the openQA bits such that we run these tests *with* persistent storage for EFI vars, at least across the lifetime of each test.

Comment 3 Adam Williamson 2018-04-04 18:09:15 UTC
Something else I forgot last night: this works on F28. It only fails on Rawhide.

Here's the logs from an F28 test:

[Bds]Booting UEFI Misc Device 3
 BlockSize : 512 
 LastBlock : 13FFFFF 
 BlockSize : 512 
 LastBlock : 1FFFFF 
 BlockSize : 512 
 LastBlock : 119B7FF 
FSOpen: Open '\EFI\BOOT\BOOTAA64.EFI' Success
[Bds] DevicePath expand: PciRoot(0x0)/Pci(0x5,0x0) -> PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x69E22731,0x800,0x64000)/\EFI\BOOT\BOOTAA64.EFI
[Security] 3rd party image[0] can be loaded after EndOfDxe: PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x69E22731,0x800,0x64000)/\EFI\BOOT\BOOTAA64.EFI.
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B FABC9040
Loading driver at 0x000F84FB000 EntryPoint=0x000F84FB120
Loading driver at 0x000F84FB000 EntryPoint=0x000F84FB120 
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF FABC9E98
FSOpen: Open '\EFI\BOOT\fbaa64.efi' Success
FSOpen: Open '\EFI\BOOT\fbaa64.efi' Success
[0m[37m[40mSystem BootOrder not found.  Initializing defaults.
FSOpen: Open 'EFI' Success
FSOpen: Open 'fedora' Success
FSOpen: Open 'BOOTAA64.CSV' Success
FSOpen: Open '\EFI\fedora\BOOTAA64.CSV' Success
Creating boot entry "Boot0001" with label "Fedora" for file "\EFI\fedora\shimaa64.efi"

FSOpen: Open '\EFI\fedora\shimaa64.efi' Success
[Security] 3rd party image[0] can be loaded after EndOfDxe: PciRoot(0x0)/Pci(0x5,0x0)/HD(1,MBR,0x69E22731,0x800,0x64000)/\EFI\fedora\shimaa64.efi.
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B F9260040
Loading driver at 0x000F8422000 EntryPoint=0x000F8422120
Loading driver at 0x000F8422000 EntryPoint=0x000F8422120 
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF F9260E98
FSOpen: Open '\EFI\fedora\grubaa64.efi' Success
[0m[30m[40m[2J[01;01H[0m[37m[40m[21;07HUse the ^ and v keys to change the selection.  

the difference is obvious: after we get to "System BootOrder not found.  Initializing defaults.", we actually log creating a new entry, and then immediately boot it (without the "Reset System" happening, so presumably `try_start_first_option()` ran and worked). For some reason, that's working in F28 but failing in Rawhide.

F28 has shim-aa64-13-4.aarch64 , Rawhide has shim-aa64-13-5.aarch64 . The former I think comes from https://koji.fedoraproject.org/koji/buildinfo?buildID=1054631 - shim-signed-13-4 - the latter from https://koji.fedoraproject.org/koji/buildinfo?buildID=1054372 - shim-13-5 . Both those seem to use the same shimaa64.efi source file (the SHA512 is the same in the `sources` file in both builds), but the BOOTAA64.EFI file winds up being a slightly different size between the two, not sure why.

Comment 4 Adam Williamson 2018-04-04 19:01:44 UTC
pwhalen actually noted what looks like the cause of this, which I missed: in shim-13-5 , BOOTAA64.CSV and BOOTIA32.CSV and BOOTX64.CSV are all 0-length. In shim-signed-13-4 they are not. This can also be seen in the package git repos, of course ('master' branch of shim has zero-length files, 'f28' branch of shim-signed does not). Looks like an easy fix to just put the right files back in the shim package build, but of course only a few people can actually build it, so we need one of them to do it.

Comment 5 Paul Whalen 2018-05-03 16:58:06 UTC
System BootOrder not found.  Initializing defaults.
FSOpen: Open 'EFI' Success
FSOpen: Open 'fedora' Success
FSOpen: Open 'BOOTAA64.CSV' Success
FSOpen: Open '\EFI\fedora\BOOTAA64.CSV' Success
Creating boot entry "Boot0001" with label "Fedora" for file "\EFI\fedora\shimaa64.efi"

FSOpen: Open '\EFI\fedora\shimaa64.efi' Success


shim-signed-15-2 looks to fix the issue, could we get this tagged into f29?

Comment 6 Paul Whalen 2018-05-04 18:00:48 UTC
Wrong package- shim-15-3 also fixes this and is indeed tagged into f29. Thanks pjones!


Note You need to log in before you can comment on or make changes to this bug.