Description of problem: Following a clean default/automatic installation of Fedora 34, the system hangs at the Lenovo splash screen. Booting a USB stick again, efibootmgr shows a bat guano bootorder that is instigated by shim 15.4-4, because the problem doesn't happen upon downgrading to shim 15-8, but immediately reoccurs when upgrading back to 15.4-4. The bootorder is apparently not completely honored by the firmware, FEdora is in the 8th position and yet fallback isn't working well enough to get to that point. So there's certainly a firmware bug here, but it seems shim 15.4-4 is instigating part of this in a way that shim 15-8 wasn't. Version-Release number of selected component (if applicable): shim 15.4-4 How reproducible: Always Steps to Reproduce: 1. Install https://download.fedoraproject.org/pub/fedora/linux/releases/34/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-34-1.2.iso 2. Reboot 3. Actual results: Hang at Lenovo splash screen Expected results: Should boot Additional info: efibootmgr -v following boot from USB stick after failed first boot (of the installed system) BootCurrent: 001F Timeout: 0 seconds BootOrder: 001F,0010,0011,0012,0013,0014,0015,0000,0019,001A,001B,001C,001D,001E,0020,0021,0022,0023 Boot0000* Fedora HD(1,GPT,fb2c442e-2249-4bf8-a6c4-391e52174312,0x800,0x12c000)/File(\EFI\fedora\shimx64.efi) Boot0010 Setup FvFile(721c8b66-426c-4e86-8e99-3457c46ab0b9) Boot0011 Boot Menu FvFile(126a762d-5758-4fca-8531-201a7f57f850) Boot0012 Diagnostic Splash Screen FvFile(a7d8d9a6-6ab0-4aeb-ad9d-163e59a7a380) Boot0013 Lenovo Diagnostics FvFile(3f7e615b-0d45-4f80-88dc-26b234958560) Boot0014 Regulatory Information FvFile(478c92a0-2622-42b7-a65d-5894169e4d24) Boot0015 ThinkShield secure wipe FvFile(3593a0d5-bd52-43a0-808e-cbff5ece2477) Boot0016 Startup Interrupt Menu FvFile(f46ee6f4-4785-43a3-923d-7f786c3c8479) Boot0017 Rescue and Recovery FvFile(665d3f60-ad3e-4cad-8e26-db46eee9f1b5) Boot0018 MEBx Hot Key FvFile(ac6fd56a-3d41-4efd-a1b9-870293811a28) Boot0019* USB CD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,86701296aa5a7848b66cd49dd3ba6a55) Boot001A* USB FDD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,6ff015a28830b543a8b8641009461e49) Boot001B* NVMe0 VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,001c199932d94c4eae9aa0b6e98eb8a400) Boot001C* NVMe1 VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,001c199932d94c4eae9aa0b6e98eb8a401) Boot001D* ATA HDD0 VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f602) Boot001E* ATA HDD1 VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f601) Boot001F* USB HDD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,33e821aaaf33bc4789bd419f88c50803) Boot0020* PXE BOOT VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,78a84aaf2b2afc4ea79cf5cc8f3d3803) Boot0021* LENOVO CLOUD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,ad38ccbbf7edf04d959cf42aa74d3650)/Uri(https://download.lenovo.com/pccbbs/cdeploy/efi/boot.efi) Boot0022 Other CD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,aea2090adfde214e8b3a5e471856a35406) Boot0023 Other HDD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f606) Boot0024* IDER BOOT CDROM PciRoot(0x0)/Pci(0x14,0x0)/USB(11,1) Boot0025* IDER BOOT Floppy PciRoot(0x0)/Pci(0x14,0x0)/USB(11,0) Boot0026* ATA HDD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f6) Boot0027* ATAPI CD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,aea2090adfde214e8b3a5e471856a354) anaconda storage.log shows INFO:program:Running in chroot '/mnt/sysroot'... efibootmgr INFO:program:Running in chroot '/mnt/sysroot'... efibootmgr -c -w -L Fedora -d /dev/nvme0n1 -p 1 -l \EFI\fedora\shimx64.efi INFO:program:Running in chroot '/mnt/sysroot'... efibootmgr INFO:program:Running in chroot '/mnt/sysroot'... efibootmgr -b 0000 -B INFO:program:Running in chroot '/mnt/sysroot'... efibootmgr -c -w -L Fedora -d /dev/nvme0n1 -p 1 -l \EFI\fedora\shimx64.efi
It was possible to assemble the installed system in chroot, downgrade to shim 15-8, and 'efibootmgr --bootorder 0000' and reboot the installed system successfully. Upon updating to shim 15.4-4 though, we're back to a failed boot even though Fedora is first in the bootorder. efivars tar will be attached matching this nvram state: BootCurrent: 001F Timeout: 0 seconds BootOrder: 0000,0019,001A,001B,001C,001D,001E,001F,0020,0021,0022,0023 Boot0000* Fedora HD(1,GPT,fb2c442e-2249-4bf8-a6c4-391e52174312,0x800,0x12c000)/File(\EFI\fedora\shimx64.efi) Boot0010 Setup FvFile(721c8b66-426c-4e86-8e99-3457c46ab0b9) Boot0011 Boot Menu FvFile(126a762d-5758-4fca-8531-201a7f57f850) Boot0012 Diagnostic Splash Screen FvFile(a7d8d9a6-6ab0-4aeb-ad9d-163e59a7a380) Boot0013 Lenovo Diagnostics FvFile(3f7e615b-0d45-4f80-88dc-26b234958560) Boot0014 Regulatory Information FvFile(478c92a0-2622-42b7-a65d-5894169e4d24) Boot0015 ThinkShield secure wipe FvFile(3593a0d5-bd52-43a0-808e-cbff5ece2477) Boot0016 Startup Interrupt Menu FvFile(f46ee6f4-4785-43a3-923d-7f786c3c8479) Boot0017 Rescue and Recovery FvFile(665d3f60-ad3e-4cad-8e26-db46eee9f1b5) Boot0018 MEBx Hot Key FvFile(ac6fd56a-3d41-4efd-a1b9-870293811a28) Boot0019* USB CD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,86701296aa5a7848b66cd49dd3ba6a55) Boot001A* USB FDD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,6ff015a28830b543a8b8641009461e49) Boot001B* NVMe0 VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,001c199932d94c4eae9aa0b6e98eb8a400) Boot001C* NVMe1 VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,001c199932d94c4eae9aa0b6e98eb8a401) Boot001D* ATA HDD0 VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f602) Boot001E* ATA HDD1 VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f601) Boot001F* USB HDD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,33e821aaaf33bc4789bd419f88c50803) Boot0020* PXE BOOT VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,78a84aaf2b2afc4ea79cf5cc8f3d3803) Boot0021* LENOVO CLOUD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,ad38ccbbf7edf04d959cf42aa74d3650)/Uri(https://download.lenovo.com/pccbbs/cdeploy/efi/boot.efi) Boot0022 Other CD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,aea2090adfde214e8b3a5e471856a35406) Boot0023 Other HDD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f606) Boot0024* IDER BOOT CDROM PciRoot(0x0)/Pci(0x14,0x0)/USB(11,1) Boot0025* IDER BOOT Floppy PciRoot(0x0)/Pci(0x14,0x0)/USB(11,0) Boot0026* ATA HDD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f6) Boot0027* ATAPI CD VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,aea2090adfde214e8b3a5e471856a354)
Created attachment 1777572 [details] efivars
Created attachment 1777575 [details] dmesg
*** Bug 1955390 has been marked as a duplicate of this bug. ***
[ 0.000000] DMI: LENOVO 20N2CTO1WW/20N2CTO1WW, BIOS N2IET94W (1.72 ) 02/18/2021
Hi, I'm having this same issue. My laptop is slightly different but behaves the same as above. It's a Thinkpad Yoga 370, so 2 generations older as the T490. `dmesg | grep DMI:` [ 0.000000] DMI: LENOVO 20JJS0VK1F/20JJS0VK1F, BIOS R0HET56W (1.36 ) 08/06/2020 Downgrading the shim works here too. Setting the boot order or creating a new boot item with efibootmgr seems to be reset after rebooting all the time, which by itself doesn't cause any problem. It does indicate that the firmware is being weird. When I set `mokutil --set-verbosity true` and booted the new-broken shim, it showed me the attached output at the point of hanging. I also created a video of the entire boot process here: https://www.youtube.com/watch?v=FBtazoABHYY
Created attachment 1777882 [details] Verbose output before hanging
I played around with this issue some more, and I found out that with the new-broken shim in place, the problem only occurs when secure boot is disabled. In other words. I enabled secure boot. Then I powered off the machine (seems to be required, but rebooting twice also works). Problem gone, I can now successfully boot! To verify, re-disabled secure boot. First boot was fine, the second warm-boot did hang again. Hard poweroff, enabled secure boot again. Warm reboot hanged again, second attempt was fine again. Meanwhile I also recorded a video of a bunch of text appearing after booting via the "ssd" option instead of the "fedora" option: https://www.youtube.com/watch?v=vqKFPMFt25Q Relevant screenshot also attached. This text can sometimes also appear in the hanging situation if you wait for long enough.
Created attachment 1777915 [details] Screenshot of text appearing on screen
(In reply to Peter Hazenberg from comment #9) > Created attachment 1777915 [details] > Screenshot of text appearing on screen This looks very much like the firmware call to HandleProtocol() (shim.c:1104) returned success but gave us back a handle that's not completely populated. Unfortunately that print would be the best clue as to what it's even trying to do when booting the "SSD" option, so I really have no idea what's going on there. That said, it's *probably* unrelated to the original issue.
I just reported this over to the Lenovo folks to give them a heads up.
Firmware 0.1.72 for this ThinkPad, from LVFS, is the latest: https://fwupd.org/lvfs/devices/com.lenovo.ThinkPadN2IETXXP.firmware Might need to wait for an update from Lenovo.
Just a thought: would it matter whether the Secure Boot mode is Standard or Custom?
The workaround that I have found works best was from Chris Murphy's instructions, to download the older 15-8 shim and replace the defective components of the 15.4-4 with the older, working 15-8 components: https://www.reddit.com/r/Fedora/comments/n27212/fedora_wont_boot_after_attempting_update_to_34/gwic9d4/?utm_source=reddit&utm_medium=web2x&context=3 Would be really nice to have some sort of acknowledgement that this is even being worked on. Seems like a huge issue for what's supposed to be a flagship line of laptops for Fedora.
Firmware N2IET95P, version 1.73 was released today via LVFS: https://fwupd.org/lvfs/devices/com.lenovo.ThinkPadN2IETXXP.firmware Can anyone see if this one works out of the box with shim-x64-15.4-4?
I can't find it now but someone on reddit said you can get Fedora 34 to boot by entering the BIOS menu then exit discarding changes. That worked and allowed me to boot into Fedora 34 long enough to install the old version of shim*.rpm. It took me the whole day but I now have a working computer again :) @sethgoldin My firmwares are fully up to date and Fedora 34 still wouldn't boot with shim-x64-15.4.4
*** Bug 1954245 has been marked as a duplicate of this bug. ***
I have checked it with new firmware(1.34) on my Thinkpad T14 and it's still not working.
Hello, I have the same issue on a Acer Aspire E1-731, both Secure Boot enabled and disabled. The boot is stuck on Acer logo and won't go further. I can press F12 to display the UEFI boot list, and I have two entries: 1. HDD: WDC WDxxxxxxxxxxxxxxx 2. Fedora (WDCxxxxxxxxxxxxxx) If I select the second line, the system boots without problem.
Don't think there's actually information waiting on me here. Commenting to remove needinfo request.
Experiencing the same issue with ThinkPad P52. Dual booted with Windows 11. Had Secure Boot enabled earlier, once I disabled it I was unable to boot into a Fedora ISO or Fedora system. System will boot with Ubuntu/GParted-live.
Have a black screen with infinite loop (laptop runs hot) on Lenovo ThinkPad X220, and old laptop without Secure Boot support. No prints, no anything, just black screen. Booting grubx64 using EFI shell works fine.
I can confirm that the MSI GT60 suffers this same issue. Booting with UEFI and CSM, grub does not load. Installed the 15-8 shim-x64 package fixed the problem. This machine did not ship with secure boot enabled and all partitions are MBR instead of GPT. The fix is simpler than shown in the Reddit thread: 1) Boot from fedora installer, choose recovery mode 2) Mount your root volume (option 1) 3) chroot /mnt/sysroot 4) cd /root 5) Obtain the package using curl/wget 6) yum install shim-x64...rpm 7) add the following line to /etc/dnf/dnf.conf: exclude=shim* No need to copy stuff around and muck up the rpm database and risk breaking things.
Correction, my partition tables are GPT, not MBR. Setting from "UEFI with CSM" to just "UEFI" did not fix the issue (there is no specific secure boot setting in the BIOS, so I assume with CSM means insecure boot and without CSM means secure boot).
It's wild that this is still marked as "new," with absolutely no acknowledgement from the RH devs. Anyway, there's a new firmware that was released. Has anyone been able to test a fresh install with this firmware? https://fwupd.org/lvfs/devices/com.lenovo.ThinkPadN2IETXXP.firmware
For me, I downloaded recently the Fedora 34 ISO and installed on my Acer Aspire E1-731 who has the issue, then updated, and there was no problem. Was this issue fixed already?
Incredible. Not only has this not been acknowledged at all by any Red Hat developers, but shim-x64-15.4-5.x86_64.rpm is listed as identical to shim-x64-15.4-4.x86_64.rpm, with the version bumped up just to distinguish that it's for F35. So I don't know why I expected anything different; but upon a clean install of F35, this issue is persisting. So I'm going to have to go replace the guts of shim-x64-15.4-5.x86_64.rpm with the actual components from 15-8 again.
Can those having the problem still report `dmesg | grep DMI:` Thanks
$ dmesg | grep DMI: [ 0.000000] DMI: LENOVO 20N2CTO1WW/20N2CTO1WW, BIOS N2IET96W (1.74 ) 08/10/2021
sudo dmesg | grep DMI: [ 0.000000] DMI: LENOVO 20N20032US/20N20032US, BIOS N2IET96W (1.74 ) 08/10/2021 To be clear I'm on Fedora 35. The problem started with Fedora 34 and persists into Fedora 35. I'm able to "fix" it replacing the shim.rpm with the one from fedora 33 (shim-x64-15-8.x86_64.rpm). My machine now has regular problems waking from sleep but I don't know if that has anything to do with this bug or the fact I'm running an older shim.
DMI: LENOVO 4286CTO/4286CTO, BIOS 8DET76WW (1.46 ) 06/21/2018 In my case, switching off "boot order lock" option in UEFI configuration allowed to start shim, but I ended booting grub directly anyway.
$ sudo dmesg | grep DMI: [ 0.000000] MI: LENOVO 20QNCTO1WW/20QNCTO1WW, BIOS N2NET46W (1.31 ) 06/22/2021
Let's get some shim debugging videos! 0. Prepare to capture video: a dark room to avoid screen glare, test with Terminal first to see if your cell phone or camera needs to have exposure and focus set manually - if you can read the text, developers can't "just figure it out". It needs to be clear. 1. Update to the failing shim 2. sudo mokutil --set-verbosity true 3. Reboot 4. Capture video of the debug scroll 5. Downgrade to working shim (boot off Fedora 33 live CD and just copy the live's shim over the new one on the system's EFI system partition) 6. sudo mokutil --set-verbosity false
Just updated to new firmware today to see if anything might be fixed, but alas, it's not. $ dmesg | grep DMI: [ 0.000000] DMI: LENOVO 20N2CTO1WW/20N2CTO1WW, BIOS N2IET97W (1.75 ) 10/09/2021 Here's a video. There's some glow, but it should be legible in HD. I had already had those working components from the 15-8 version installed, so I reinstalled shim-x64-15.4-5.x86_64.rpm, and then tried rebooting. You can see where it gets stuck, at that `\EFI\fedora\grubx64.efi` step. https://www.youtube.com/watch?v=QfXtJzfFozw
Another thread of users experiencing the issue: https://www.reddit.com/r/Fedora/comments/pprgol/fedora_34_and_35_wont_install_on_new_thinkpads_no/ We can call this a firmware bug and blame Lenovo, but in my opinion, if this didn't happen with 15-8 but is indeed happening with 15.4-4 and 15.4-5, then that's a bug introduced after 15-8.
The two youtube videos in this issue show the hang happening at different points and don't give any clues why they've just stopped where they did. I've opened an upstream shim issue to see whether the next step is to enhance debug output. I've also put out a call on Fedora devel@ mailing list to see if there are any T490 users who aren't having this problem. The working assumption is all T490's are affected but what if this isn't true? It's curious that shim+grub work OK when booted from install media, but not from the laptop's built-in drive following a clean install.
I have Lenovo P14s (gen 2, Intel) and experiencing a similar issue. I can't boot my laptop with _DISABLED secure boot_, it gets stuck at splash screen before grub. And that happens not only when I boot from the laptop's ssd, but also from a USB drive with fedora 35 iso. Everything boots fine with secure boot enabled. I've noticed that because the UEFI Firmware update tool was also getting stuck at splash screen, but with _secure boot ENABLED_. I thought that maybe disabling secure boot will fix that, but then I got the issue described above.
A small update: after my laptop got a firmware update I can't boot Fedora 34 and Fedora 35 anymore from a USB drive with secure boot enabled (haven't tried with secure boot disabled). Laptop: Lenovo P14s (Gen 2, Intel with touchscreen) UEFI Firmware version: 1.46 (N34ET46W) Embedded Controller Firmware version: 1.37 (N34HT37W) Ubuntu 21.10 and the latest openSUSE tumbleweed boot fine. So probably it has something to do with the combination of the firmware and the shim version used in Fedora.
Proposing as a prioritized bug because: (a) it's a regression in at least shim, possibly the device firmware too (b) it affects fairly common hardware (c) other distributions aren't having this problem (yet) (d) it's taking too long to make some forward progress like even "ok we have a shim that spits out more debug info could you try this?"
cc Mark Pearson, can someone at Lenovo help take a look at this?
Thanks Michel - I've forwarded the issue to the team in Asia for their input. Mark
Hi - I've been trying to reproduce this on my T490 to help the FW team out...and haven't been able to. Wanted to check what I was missing. Below are the steps I was trying based on the notes above: T490 Fedora35 - I've used both shim 15.4-4 and 15.4-5, but right now I'm using 15.4-4 BIOS 1.46 Steps - Enable Secure Boot. - Boot to Fedora - Power down - Boot to F1 setup and disable secure boot - F10 save and exit. - Boot to Fedora succesfully. Reboot - Boot to Fedora succesfully The FW team need some pointers as to what has changed in shim and the FW is doing 'wrong'. I had pointed them at this thread, but there hasn't (I don't think) been anything conclusive on what changed in shim that might help them focus on something. If I can reproduce I can do some digging but otherwise I'm a bit stuck. Any pointers/suggestions on next steps would be appreciated mark
Mark, can you try firmware 1.72 or above? 1.72 is the firmware where this was first noticed.
Last time I tried, my P53 with BIOS version 1.34 (N2NET49W) wouldn't even boot from the F35 USB image when I picked it from the boot menu. I could only install it by changing the boot order in the BIOS to boot from USB first, and then apply the work-around of entering the BIOS on boot and exiting without saving changes. Secure Boot is disabled.
I was way behind on that system :) I've updated to the latest (BIOS 1.75 and EC 1.19) and I'm still not reproducing the problem. I'll try a fresh install next - unless anybody has pointers on a reliable set of reproduce steps. Mark
@Mark and @Seth - sounds like the two systems are not in identical states, despite the same firmware revision. Seth should maybe `sudo tar -acf space.tar /sys/firmware/efi/efivars/` for safekeeping, and then do an agreed upon "factory reset" that's intended to get a system's NVRAM in a known state.
Maybe I'm missing something fundamental here. I rebooted to the BIOS settings, hitting Enter and then F1. Then I hit F9 to "reset to defaults." Is this what would get me back to the factory default settings for the T490 firmware? Then I booted up, and reinstalled the shim via `$ dnf reinstall shim`, so that I was truly using shim-x64-15.4-5 on F35. Then I rebooted, and then still, as has been the case since 15.4-4 on F34, the machine hung on the Lenovo splash screen. So I hard reset, booted to my F35 live USB, copied the three files from 15-8 back to my EFI partition, and could get back in again. So I have the guts of the old 15-8 working for me, even though dnf "thinks" I'm running 15.4-5. $ dmesg | grep DMI: [ 0.000000] DMI: LENOVO 20N2CTO1WW/20N2CTO1WW, BIOS N2IET97W (1.75 ) 10/09/2021 I'm able to reproduce this hang at the splash screen 100% of the time. I'm not quite sure what other information I could possibly provide, but it seems like lots of folks are having this issue, so I'd love to help squash this bug once and for all.
Mark, do you have LUKS enabled? I do. Perhaps that has something to do with this whole issue?
>booted to my F35 live USB, copied the three files from 15-8 back to my EFI partition Fedora 33 is the last to have shim 15-8. Fedora 34+ media have shim 15.4. But you can successfully boot either Fedora 34 or Fedora 35 install media from a USB stick? At this stage of boot, the firmware has ostensibly only read the GPT, and the EFI file system. As it executes shim, the firmware doesn't know what other file systems are yet. But could there be something about the GPT, or FAT file system, or in NVRAM, that's resulting in confusion, only with shim 15.4? Hence the idea to wipe NVRAM to get it as stock as possible. But I'm not sure how thorough reset to defaults is when it comes to NVRAM. I guess we could compare evifars between the working and non-working hardware.
Hi, I don't have LUKS enabled - I just have a vanilla install of Fedora with defaults (and it's the only thing on the system currently). I'm hesitant to go that path - it wasn't reported on any of the other linked threads I looked at. I did the BIOS defaults reset myself - didn't seem to make any difference. Did another fresh install and updated to the latest and everything is still working for me :( I'm stumped. I do have a prototype T490 rather than a production unit but I don't think that should make any difference. I'll ask my colleague if he can try on his unit, but he'll need to go and pick it up from the office. Mark
FWIW, I too have LUKS enabled.
In today's Prioritized Bugs meeting, we agreed to accept this as a Prioritized Bug on the grounds it apparently affects multiple popular hardware vendors. https://meetbot.fedoraproject.org/fedora-meeting-1/2022-01-12/fedora_prioritized_bugs_and_issues.2022-01-12-16.00.log.html#l-75
To put it simply: in order for forward progress to occur, someone who can test development builds of shim needs to be able to reproduce (and therefore debug) the issue. Right now, none of us (maintainers and Lenovo) have hardware that can, and no one who can reproduce the issue has built/signed/enrolled/tested a development shim.
Mark, can you try a fresh install of F35 on 1.75 with LUKS enabled on the boot drive? I suspect this is related.
I stuck to this during upgrade from Fedora 33 to 35, my disk was and it is encrypted using LUKS. shim downgrade and lock. ``` [root@x1 dnf]# cat plugins/versionlock.list # Added lock on Tue Dec 7 15:45:25 2021 shim-x64-0:15-8.* ``` Laptop: SKU Number: LENOVO_MT_20KH_BU_Think_FM_ThinkPad X1 Carbon 6th
I have a feeling that the bug is related to UEFI variables state. On X220, I had boot order locked in UEFI settings, and after updating to Fedora 34 shim just hang indefinitely, without any errors. Once I disabled "lock boot order" in UEFI settings, it booted fine, and even if you lock boot order again it will work just as well. Previous shim versions worked fine with boot order locked.
Note that the system does not support Secure Boot and shim did not try to launch fallback .efi file when hung.
(In reply to Robbie Harwood from comment #53) > To put it simply: in order for forward progress to occur, someone who can > test development builds of shim needs to be able to reproduce (and therefore > debug) the issue. Right now, none of us (maintainers and Lenovo) have > hardware that can, and no one who can reproduce the issue has > built/signed/enrolled/tested a development shim. Robbie, does the development version have better debugging than the current shim in Fedora? Is there any documentation for mortal humans to follow how to do this exactly? This is the most recent Fedora doc I can find on this subject, and it's missing steps on how to do what you're asking. https://docs.fedoraproject.org/en-US/Fedora/18/html/UEFI_Secure_Boot_Guide/index.html This looks like it could be adapted for Fedora, but the author got stuck and there are no replies. https://askubuntu.com/questions/1313644/how-to-build-shim-from-source-and-enroll-keys (In reply to Seth Goldin from comment #54) > Mark, can you try a fresh install of F35 on 1.75 with LUKS enabled on the > boot drive? I suspect this is related. Has the problemed laptop been given a truly clean install? e.g. blkdiscard -fv /dev/nvme0n1 and then do an automatic installation with an without LUKS? This would take about 30 minutes to do twice and see if LUKS is a factor, or for that matter some obscure anomaly with the GPT or FAT that's triggering a bug in the firmware. That's a lot more likely than LUKS which neither the firmware nor shim know anything about and will treat it like any other content on the drive it doesn't recognize: ignore it. There's literally no mechanism by which the firmware or shim could care about LUKS.
After 2 hours of debugging with prints, I found that fallback (fbx64.efi) fails on CopyMem inside add_to_boot_list on my Lenovo X220 in this place of fallback.c: CopyMem(newbootorder + 1, bootorder, sizeof (CHAR16) * option); CopyMem(newbootorder + option + 1, bootorder + option + 1, sizeof (CHAR16) * (nbootorder - option - 1)); The fix already present in upstream: https://github.com/rhboot/shim/commit/41319e14c9144063605750baccd5bb332a3cf4fc Kindly asking to apply this patch to finally fix this issue.
Updating shim is not an instantaneous process, but I'm glad to hear we'll have the fix available with 15.5.
(The most helpful thing for making 15.5 happen upstream is for people who can to test the RCs, both in secureboot and non-secureboot configurations. By this I do not mean merely verifying that we have correctly used git, but actually booting it on your hardware.)
Thanks ValdikSS and Robbie! Where can I get a signed shim to test against? I'm happy to run it against a number of our platforms in various configurations. Just wondering: there have been a couple of fairly impactful shim issues in the last 12 months (there was another issue breaking FW updates early last year) - is there something we can do at Lenovo to help with testing etc? Admittedly I personally struggled to reproduce either of the issues...but I'd like to try and help out where we can and if there's a way we can throw proposed updates at a bunch of HW to sanity check it that might be useful? Caveat - we're working on improving some internal testing resources we have to make things like this possible so it might take a while to get to the truly useful stage...but seems like a good target :) If there's any legs to this email me (markpearson at lenovo dot com) rather than dragging this bug too much sideways. mark
> Where can I get a signed shim to test against? I'm happy to run it against a number of our platforms in various configurations. Only the distro shims get signed by MS. We're talking about a pre-release version from upstream, so if you want to test, you either need to enroll your own CA into the UEFI db, or disable verification for testing purposes. (How to do this varies between vendors and machines.) > Just wondering: there have been a couple of fairly impactful shim issues in the last 12 months (there was another issue breaking FW updates early last year) - is there something we can do at Lenovo to help with testing etc? Admittedly I personally struggled to reproduce either of the issues...but I'd like to try and help out where we can and if there's a way we can throw proposed updates at a bunch of HW to sanity check it that might be useful? Caveat - we're working on improving some internal testing resources we have to make things like this possible so it might take a while to get to the truly useful stage...but seems like a good target :) If there's any legs to this email me (markpearson at lenovo dot com) rather than dragging this bug too much sideways. Thanks, I'll follow up via email.
Same here on Lenovo p15v Gen 1 type 20TQ. Upgraded from FC33 to FC35 (disk LUKS encrypted), same issue. Tried a fresh install, same. Downgraded to shim 15-8 works as expected. Keeping a look on updates to see if fixes the issue.
Same here on Lenovo T480 with enabled EFI + secure boot. Disk not encrypted. Everything start with upgrade from Fedora 33 -> Fedora 34 -> Fedora 35 and after reboot - stuck on Lenovo logo. If I restert system again, any time system stuck on Lenovo logo and does not show grub. To success boot I need to going to bios settings and disable secure boot and trying to boot again ;) + shutdown from power buotton (long press) and enable secure boot from bios. After that operation I able to see grub menu. When I download shim-x64-15-8.x86_64.rpm and install it and exclude shim package in dnf.conf - everithing works again without problem. That is a bug and all of us that has that problem will be happy if rh developers fix that. Thanks...
@mpe
(Gah, Bugzilla, why?! Sorry for the spam, everyone) Mark, was your testing successful?
Ha - I have that issue with bugzilla all the time too. Yes it was - I tested a bunch of platforms with no issues found. Shared the results with the RH team. Not sure what the next steps are but I'm assuming they're looking at it. Mark
src.fedoraproject.org is no longer hosting the working Fedora 33 shim (shim-x64-15-8.x86_64.rpm). So I've uploaded it in case anyone needs it but forgot to back it up. I realize there's a security concern with downloading from me, a random internet person. Hopefully someone else can show us an official way to get the rpm. sha256sum 0ff0897d7fd548a435345944ad3faece662ae8fdbf8c7e625991ce3ccfd54eef https://mega.nz/file/JZYWGRDY#R8rGHeys4VLbAbN9Mrmfy6jRF3YTE_R3nwsn6kH8Bwk
Yeah so don't do that. Built rpms don't live on src.fedoraproject.org. If you want shim-x64-15-8.x86_64.rpm you can get it from koji: https://koji.fedoraproject.org/koji/buildinfo?buildID=1149359
I'm still not seeing any new package with the fix: https://koji.fedoraproject.org/koji/packageinfo?buildOrder=-completion_time&packageID=14502&tagOrder=name&tagStart=0#buildlist Couldn't this patch just be backported? https://github.com/rhboot/shim/commit/41319e14c9144063605750baccd5bb332a3cf4fc What's the hold-up?
shim-unsigned 15.5 was just built today, this is not signed for UEFI Secure Boot though. But it is the first step in a multistep process to get a signed shim packaged per https://github.com/rhboot/shim-review https://koji.fedoraproject.org/koji/buildinfo?buildID=1932202 Once it's posted for review, I figure it's a few weeks for it to get signed. In the meantime it should be possible to sign it yourself and enroll the key with mokutil.
Hello! Are there any updates on this? Looks like it hasn't been posted for review yet
I'm checking with the maintainer to see if a timeline is available.
Currently, the maintainers expect to send the updated shim for signing in early June.
What is the reason for waiting until June before sending it to be signed? For someone who are not that familiar with the shim development, it seems a bit strange that this bug is marked as a prioritized bug on the F36 blockers page, the unsigned binaries are built, and yet it will not be sent for signing until after 3 month after the binaries were built. I'm tracking this bug since the current shim also prevents firmware upgrades on some Lenovo laptops and have been so for about a year (which can cause security issues), so there are more than one reason for this to be prioritized.
(In reply to mattias.eriksson from comment #76) > What is the reason for waiting until June before sending it to be signed? There's other work going into it that I can't comment on publicly, unfortunately. shim is an unusual package due to the signing process, so it's generally better to batch up the work and go through the process once instead of several times in short succession.
Hi, What's the impact on f36 use/upgrade path for Lenovo users? The bug is reported against f34, is it safe to assume that systems that are running f35 are OK to upgrade? what about systems that successfully upgraded f34->f35?
*** Bug 1987018 has been marked as a duplicate of this bug. ***
I would also like to know the answer to Jan's question. If I upgrade or clean install Fedora 36, is the flawed shim-x64-15.4-5.x86_64.rpm still shipping with it, and will we all have to go through the convoluted workaround yet again?
The problems started from F34. It seems not to be fixed in F36 yet. But if you are running F34 or F35 without workarounds you should be safe to upgrade. If you are affected the workaround is possible also for F36.
This message is a reminder that Fedora Linux 34 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 34 on 2022-06-07. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '34'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 34 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
I tried booting Fedora 36 on my T490. I can confirm the bug is still present. Sadly my BIOS now won't remember the boot order I set more than one boot before resetting making it impossible to even use the workaround.
Hi Peter, are you able to update this for F36 as well? Still present in 36, and I don't want the bug to close before it is.
Maybe Peter or Ben can updated the version to FC35/36 ? Is still present on both FC35 and FC36 and should not be lost until the update is released.
This bug is in Rawhide so setting it to rawhide. I expect it needs to be fixed there first before we'd see it get to F35. I see a shim review request for rhel 9 back in March, but I don't see one for Fedora, so I don't know what or when then plan is.
FEDORA-2022-98830efc68 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-98830efc68
f35, f36, and rawhide eventually all need the same build, so test with the one above, and I'll work with rel-eng on tagging when it's okay to do so.
FEDORA-2022-98830efc68 has been pushed to the Fedora 35 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-98830efc68` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-98830efc68 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2022-98830efc68 has been pushed to the Fedora 35 stable repository. If problem still persists, please make note of it in this bug report.
(In reply to Fedora Update System from comment #90) > FEDORA-2022-98830efc68 has been pushed to the Fedora 35 stable repository. > If problem still persists, please make note of it in this bug report. This shim (shim-x64-15.6-1.x86_64) has a regression, it does not allow to work with third-party enrolled certificates or binary file hashes. It always shows security violation screen, regardless what is enrolled into moklist.
(In reply to ValdikSS from comment #91) > This shim (shim-x64-15.6-1.x86_64) has a regression, it does not allow to > work with third-party enrolled certificates or binary file hashes. It always > shows security violation screen, regardless what is enrolled into moklist. Please file a new bug with exact reproduce steps. I think it's pretty urgent we determine if this is reproducible because it's planned to go stable soon. This would be a really bad regression if folks can't use their own signed EFI programs or kernel modules, so I'm not sure 2 weeks is enough testing let alone 5 days. I think we ought to keep it in Rawhide for a while longer before moving it back to Fedora 35/36. Could the original posters with the problem *this* bug is about please test this new shim in koji and confirm your problem is fixed?
FWIW I can't reproduce the problem mentioned in c91 $ openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -sha256 -days 365 $ openssl x509 -in cert.pem -inform PEM -out test_cert.cer -outform DER $ sudo mokutil --import test_cert.cer (reboot, I can enroll the key, following key enrollment I can boot, nothing unexpected appears including a security violation screen; deleting the enrolled key works as expected too)
The new shim version works just fine for me. I was finally able to install firmware updates with no issues. I also have self-signed kernel modules and haven't faced the issue described by VladikSS. Didn't notice any security violation screen.
(In reply to Chris Murphy from comment #92) > Please file a new bug with exact reproduce steps. I think it's pretty urgent > we determine if this is reproducible because it's planned to go stable soon. Sorry, it seems this regression was introduced not at this version, but in shim-x64-15.4-1.x86_64, it's the first version that does not load third-party files with security violation. shim-x64-15-8.x86_64 works as intended. Bug: https://bugzilla.redhat.com/show_bug.cgi?id=2099380
Updated here on FC35 and works ok, thanks! Also with self signed modules no issues, everything continues running ok. Finally I can remove shim from versionlocked dnf list ;)