Bug 1861977
Summary: | RHSA-2020:3216 grub2 security update renders system unbootable | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 8 | Reporter: | are-support | ||||
Component: | shim | Assignee: | Bootloader engineering team <bootloader-eng-team> | ||||
Status: | CLOSED ERRATA | QA Contact: | Release Test Team <release-test-team-automation> | ||||
Severity: | urgent | Docs Contact: | |||||
Priority: | urgent | ||||||
Version: | 8.2 | CC: | ajb, akborder, anazmy, aperotti, apm118, asamir, awilliam, beat, berend.de.schouwer, bhoefer, b, bodik, bootloader-eng-team, bubrown, cdonnell, chref, Colin.Simpson, cperry, crxssi, cyberrider, cye, daniell1, dev, dmaley, dornelas, dwojewod, ealcaniz, emcnabb, farid_ghoreyshi, fedoraproject, fiezzi, fmartine, fweimer, gbailey, glaubitz, hakon.gislason, hartsjc, h.hoogendoorn, jbittner, jcsible, jeff, jpazdziora, jrhbgz16a, jscheibe, jstancek, jstodola, jsuchane, jwedgeco, jwkwan2000, kas, knoha, kwalker, lee.jnk, ludwigit, manuel.wolfshant, mark, mbanas, mbenatto, mburman, mfalz, mgarciac, mhofmann, miabbott, mironov.ivan, momran, mpoole, murray.j.blakeman, mvanderw, ngompa13, nk, obockows, ofalk, oktaya, pasteur, perobins, peter, petr, phil, phil.randal, pjones, prjagtap, ptalbert, pvlasin, pzatko, rboza89, redhat-bugzilla, redhat, rmaity, rmetrich, robert.scheck, rstrode, ryankr, sbonazzo, smstewa4, sonny, sreber, steevithak, stefw, tbblake, therman, thoger, toracat, twaugh, vkadlcik, w.deborger, wmealing, x.wolverine.x, yuokada, z8sergey8z | ||||
Target Milestone: | rc | Keywords: | ZStream | ||||
Target Release: | 8.0 | ||||||
Hardware: | x86_64 | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | shim-15-15.el8 | Doc Type: | If docs needed, set a value | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | |||||||
: | 1862230 1862231 1862232 (view as bug list) | Environment: | |||||
Last Closed: | 2020-11-04 02:06:11 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | |||||||
Bug Blocks: | 1862230, 1862231, 1862232 | ||||||
Attachments: |
|
Description
are-support
2020-07-30 05:15:35 UTC
Hit the same issue right now on my work laptop You may be able to work around this by disabling secure boot in the BIOS. Not ideal.. for sure but if you can't see grub.. sounds like something is up in the secure boot chain. I booted my system with the installation ISO in troubleshoot mode, chrooted to /mnt/sysimage, regenerated the grub.cfg with grub2-mkconfig and executed grub2-install. Now I can go past grub but got stuck to emergency shell failing swtching root. (In reply to Sandro Bonazzola from comment #6) > I booted my system with the installation ISO in troubleshoot mode, chrooted > to /mnt/sysimage, regenerated the grub.cfg with grub2-mkconfig and executed > grub2-install. > Now I can go past grub but got stuck to emergency shell failing swtching > root. solved this by actually copying the grubenv file to /boot/grub2 instead of relying on the symlink to efi. Hi, If you are Red Hat customers, please do open a support case with Red Hat Support. This will allow our support engineers to gather debugging data that may help to understand what is happening in these individual situations. Thanks, Cliff See also BZ 1862045 on RHEL7. Happens on HPE hardware with secure boot disabled. Solution is to downgrade grub2 / shim / mokutil Seen on HPE ProLiant XL230k Gen10 with secure boot disabled. I hit it as well and resolved by 1) booting to rhel-8 (centos-8) troubleshooting mode 2) chrooted to /mnt/sysimage 3) setup networking 4) yum downgrade shim-x64 grub2\* 5) reboot Switching boot mode to legacy in bios does not help. (In reply to Jaroslav Suchanek from comment #15) > I hit it as well and resolved by > > 1) booting to rhel-8 (centos-8) troubleshooting mode > 2) chrooted to /mnt/sysimage > 3) setup networking > 4) yum downgrade shim-x64 grub2\* > 5) reboot > > Switching boot mode to legacy in bios does not help. Thanks a lot for the information. Could you please try to downgrade them individually to confirm which package is causing the issue? Since if I understood correctly the problem happens when Secure Boot is disabled, that should be possible. Solution tells to protect yum from upgrading I got new laptop from RHEL8.2 have not yum upgrade system. But I might need to run `yum update` in order to install some dependencies would this solution of protecting yum from upgrading is enough for me ? Protect yum from upgrading the packages by adding the following entry in /etc/yum.conf Raw exclude=grub2* shim* mokutil The problem is in grub2-efi-x64, specifically in the grubx64.efi. You can have everything else up-to-date but use grubx64.efi from the previous RPM, and the system can boot again (assuming that SecureBoot is turned off, of course). (In reply to Christof Efkemann from comment #21) > The problem is in grub2-efi-x64, specifically in the grubx64.efi. You can > have everything else up-to-date but use grubx64.efi from the previous RPM, > and the system can boot again (assuming that SecureBoot is turned off, of > course). Just so I've got clear data - which version is "the previous RPM" for you? (In reply to Peter Jones from comment #23) > Just so I've got clear data - which version is "the previous RPM" for you? The previously installed RPM was grub2-efi-x64-1:2.02-81.el8.x86_64 One additional observation: Like I said, we were able to boot the new kernel (4.18.0-193.14.2.el8_2.x86_64) with the grubx64.efi from the 2.02-81.el8.x86_64 RPM. However, this seems to work only when bypassing shim altogether. I. e.: UEFI -> shim -> grubx64.efi (2.02-87) -> *crash* UEFI -> shim -> grubx64.efi (2.02-81) -> *crash* UEFI -> grubx64.efi (2.02-87) -> *crash* UEFI -> grubx64.efi (2.02-81) -> kernel -> *success* (In reply to Christof Efkemann from comment #28) > One additional observation: > Like I said, we were able to boot the new kernel > (4.18.0-193.14.2.el8_2.x86_64) with the grubx64.efi from the > 2.02-81.el8.x86_64 RPM. > However, this seems to work only when bypassing shim altogether. > > I. e.: > UEFI -> shim -> grubx64.efi (2.02-87) -> *crash* > UEFI -> shim -> grubx64.efi (2.02-81) -> *crash* > UEFI -> grubx64.efi (2.02-87) -> *crash* > UEFI -> grubx64.efi (2.02-81) -> kernel -> *success* This is with shim-x64-15-14.el8_2.x86_64 right? What happens if you downgrade the shim package? (In reply to Javier Martinez Canillas from comment #29) > (In reply to Christof Efkemann from comment #28) > > One additional observation: > > Like I said, we were able to boot the new kernel > > (4.18.0-193.14.2.el8_2.x86_64) with the grubx64.efi from the > > 2.02-81.el8.x86_64 RPM. > > However, this seems to work only when bypassing shim altogether. > > > > I. e.: > > UEFI -> shim -> grubx64.efi (2.02-87) -> *crash* > > UEFI -> shim -> grubx64.efi (2.02-81) -> *crash* > > UEFI -> grubx64.efi (2.02-87) -> *crash* > > UEFI -> grubx64.efi (2.02-81) -> kernel -> *success* > > This is with shim-x64-15-14.el8_2.x86_64 right? What happens if you > downgrade the shim package? No, that's with shim-x64-15-13.el8.x86_64. However, after writing the last comment, I got some doubts about my previous observations -- being unable to boot grubx64.efi version 2.02-87 without shim resulting in a crash. Turns out, when I tried to reproduce this crash just now, I wasn't able to. So the revised list looks like this: UEFI -> shim -> grubx64.efi (2.02-87) -> *crash* UEFI -> shim -> grubx64.efi (2.02-81) -> *crash* UEFI -> grubx64.efi (2.02-87) -> kernel -> *success* UEFI -> grubx64.efi (2.02-81) -> kernel -> *success* Therefore, my initial assessment (grubx64.efi is the culprit) clearly seems wrong -- the problem seems to lie with shim-x64-15-13.el8.x86_64. (NB: shim-x64-15-14.el8_2.x86_64 isn't available on my system -- is that from AppStream?) Created attachment 1702984 [details]
shimx64.efi
Please test with the shim EFI binary attached.
It has to be copied to /boot/efi/EFI/redhat/
(In reply to Javier Martinez Canillas from comment #32) > Created attachment 1702984 [details] > shimx64.efi > > Please test with the shim EFI binary attached. > > It has to be copied to /boot/efi/EFI/redhat/ This worked for me on CentOS 8. (In reply to Javier Martinez Canillas from comment #32) > Created attachment 1702984 [details] > shimx64.efi > > Please test with the shim EFI binary attached. > > It has to be copied to /boot/efi/EFI/redhat/ Works well on lenovo t470s with rhel-8.2. (In reply to Ludwig from comment #36) > (In reply to Javier Martinez Canillas from comment #32) > > Created attachment 1702984 [details] > > shimx64.efi > > > > Please test with the shim EFI binary attached. > > > > It has to be copied to /boot/efi/EFI/redhat/ > > This worked for me on CentOS 8. Same here, works on CentOS 8.2 (of course, I didn't try SecureBoot - I assume that the signature check would fail in this case (RHEL shim/CentOS GRUB)). Thank you Sandro Bonazzola for that tip on grubenv. Also tried to fix this from the troubleshooter (centos 8) and had the same switch root fail/emergency console. Banging my head all day but at least I'll sleep good tonight. Tomorrow I'll see if there are fixed packages. Turning off updates for now. Cheers (In reply to Sandro Bonazzola from comment #6) > I booted my system with the installation ISO in troubleshoot mode, chrooted > to /mnt/sysimage, regenerated the grub.cfg with grub2-mkconfig and executed > grub2-install. > Now I can go past grub but got stuck to emergency shell failing swtching > root. Did the same, but disabled secure boot in BIOS. Ended up with a system that boots to `grub>` prompt. Entering `normal` at it resumes usual OS boot with image menu, etc. For us (non-secure-boot environment), replacing /boot/efi/EFI/redhat/shimx64.efi (from shim-x64-15-14.el8_2.x86_64) by /boot/efi/EFI/redhat/grubx64.efi (from grub2-efi-x64-2.02-87.el8_2.x86_64) lead us back to a bootable system. Opened case 02717116 at the Red Hat customer portal anyway, but I guess it's related to this RHBZ. (In reply to Wade Mealing from comment #3) > You may be able to work around this by disabling secure boot in the BIOS. > Not ideal.. for sure but if you can't see grub.. sounds like something is up > in the secure boot chain. My UEFI configuration already has SecureBoot disabled. The CentOS 7 server (CentOS 7.8.2003) locked up at the point it was loading the shimx64.efi module; grub never even got a chance to display its menu. Fortunately, I had another box with the prior GRUB2 EFI modules as I'd not updated it yet, mainly because I was distrustful of new shim modules! After booting the dead host from USB/ISO rescue and mounting the boot/efi file system, I copied all the old efi modules to their correct locations. No change to grubenv. Now I'm running with the older grub2 efi modules and the latest kernel (3.10.0-1128.18.2.el7) I can confirm that CentOS 7 is also broken by the latest grub/shim update. With grub2-2.02-0.86.el7.centos.x86_64 and shim-x64-15-7.el7_9.x86_64 it does not boot for me, grub2-2.02-0.81.el7.centos.x86_64 and shim-x64-15-2.el7.centos.x86_64 the system is booting OK. (In reply to Javier Martinez Canillas from comment #32) > Created attachment 1702984 [details] > shimx64.efi > > Please test with the shim EFI binary attached. > > It has to be copied to /boot/efi/EFI/redhat/ Yesterday I spend several hours ending up booting in recovery mode and perform a roll back to previous version as described in https://access.redhat.com/solutions/3486741, today I tried the grub update again to find out the systems did fail again to boot. On CentOS 8.2 copied the EFI file into /boot/efi/EFI/centos/ and the server is back to normal. Thanks for this fix. (In reply to Javier Martinez Canillas from comment #32) > Created attachment 1702984 [details] > shimx64.efi > > Please test with the shim EFI binary attached. > > It has to be copied to /boot/efi/EFI/redhat/ I am on an HP ML350G10 & CENTOS8.2. Same problem as everyone else- after the update and reboot, the server froze on the "HP Enterprise" logo. I booted rescue media and was unable to rollback the affected packages as suggested by https://access.redhat.com/solutions/5272311 I just get a bunch of "lowest version already installed, cannot downgrade it." And I was unable to locate a previous version (or even name of) shimx64-15-13.el8.x86_64. That is when I decided to just give your shimx64.efi a try. I renamed the broken one, then copied the supplied shimx64.efi to /boot/efi/EFI/centos/ and rebooted the system. Grub came up normally. Choose newest kernel, late in the boot it complained about "Warning -- SELinux targeted policy relabel is required", it worked on that a while, then it rebooted itself again. After that reboot, I have 4.18.0-193.14.2.el8_2.x86_64 up and running again, apparently normal. Not sure where you got the file, but it works great. Thank you! Now we just wait for updated packages (once they figure out what went wrong). For sure I will be more prepared for the next attempt... This issue just hit mainstream: https://linux.slashdot.org/story/20/07/31/158212/red-hat-security-update-renders-systems-unbootable Can confirm on a Centos 7 server. I had to boot with a recovery USB stick and then downgraded grub, shim and also mokutil as yum dependency. Then the system rebooted without problems I did a "yum update" this afternoon on our CentOS 8.2 server and it seems to be bricked now. I get the Dell logo and then a black screen, no grub no CentOS. I see some people above saying they've fixed it by replacing a "shim" file - can anyone point me to detailed instructions on how to do this with a non-bootable server? Is there a bootable CD or flash stick image that has the fix on it? (In reply to Nerijus from comment #42) > (In reply to Sandro Bonazzola from comment #6) > > I booted my system with the installation ISO in troubleshoot mode, chrooted > > to /mnt/sysimage, regenerated the grub.cfg with grub2-mkconfig and executed > > grub2-install. > > Now I can go past grub but got stuck to emergency shell failing swtching > > root. > > Did the same, but disabled secure boot in BIOS. Ended up with a system that > boots to `grub>` prompt. Entering `normal` at it resumes usual OS boot with > image menu, etc. I am seeing the same thing. How did you fix it? Update: I found this page at RedHat that is supposed to solve the problem: https://access.redhat.com/solutions/5272311 I tried applying to my CentOS 8.2 but ran into a couple of problems: 1. All the steps are in clear text except Step 2, which is critical to the solution. Step 2 is pay-walled and you have pay a support fee to access it which seems like bit of a jerk move for a critical issue like this. But I am using CentOS and don't pay for support so I guess I can't complain too much. Without step 2, you can't easily start networking after booting to the troubleshooting mode of the install DVD. I googled and found enough info bring up the ethernet interface, give it an ip, and set a route, so I got minimal networking. So far so good. 2. The solution as described in the above document says to fix the problem by using "yum downgrade shim\* grub2\* mokutil" but that doesn't work on my CentOS 8.2, I just get a yum error saying I already have the oldest version and there's nothing to downgrade to. If anyone has successfully fixed this on a CentOS 8.2. and could post a step-by-step guide for what they did, it would be appreciated. I have a feeling it's going to be long night... :( Centos 8.2 said earlier versions of the packages weren't available when I tried the yum downgrade from the chrooted environment. I just rpm -qa shim-x64\* grub2\* and noted the packages I had. Then manually downloaded the previous versions from: http://mirror.centos.org/centos-8/8.2.2004/BaseOS/x86_64/os/Packages/ Then set an ip in the chroot and scp'd these packages onto this box. Used rpm -Uvh --force to install them. I then rebooted, and came up , it did and SELinux relabel and rebooted again.then all fine. https://lists.centos.org/pipermail/centos-devel/2020-July/055953.html Other people seem to be doing the same, there maybe a neater way someone knows. Work around that gets the system to boot (doesn't fix it, just allowed me to boot, without secure boot) 1. drop to boot menu, then to uefi shell (hold F11 during boot on my system) 2. navigate to the efi folder (uefi shell is just like dos, for me the commands where:) --- FS0: cd EFI cd CENTOS --- 3. directly load grubx86.efi --- grubx64.efi --- I just wanted to post a brief note that updated shim packages are now available and can be used in conjunction with previously released grub2, fwupd, and fwupdate packages. Both https://access.redhat.com/security/vulnerabilities/grub2bootloader and https://access.redhat.com/solutions/5272311 have been updated with links to the new shim package bugfix errata. Red Hat fixes are available. https://access.redhat.com/errata/RHBA-2020:3265 https://access.redhat.com/errata/RHBA-2020:3262 wait for CentOS. (In reply to Steevithak from comment #54) > Update: I found this page at RedHat that is supposed to solve the problem: > > https://access.redhat.com/solutions/5272311 > > I tried applying to my CentOS 8.2 but ran into a couple of problems: > > 1. All the steps are in clear text except Step 2, which is critical to the > solution. Step 2 is pay-walled and you have pay a support fee to access it > which seems like bit of a jerk move for a critical issue like this. But I am > using CentOS and don't pay for support so I guess I can't complain too much. > Without step 2, you can't easily start networking after booting to the > troubleshooting mode of the install DVD. I googled and found enough info > bring up the ethernet interface, give it an ip, and set a route, so I got > minimal networking. So far so good. > > 2. The solution as described in the above document says to fix the problem > by using "yum downgrade shim\* grub2\* mokutil" but that doesn't work on my > CentOS 8.2, I just get a yum error saying I already have the oldest version > and there's nothing to downgrade to. > > If anyone has successfully fixed this on a CentOS 8.2. and could post a > step-by-step guide for what they did, it would be appreciated. I have a > feeling it's going to be long night... :( I am not sure if you fixed the problem. I had same issue when I tried to downgraded shim\* etc and it did not help at all. What I did is follow the other thread that provided a shimx64.efi attachment. I have to save it to a usb from another computer. Once I booted into the trouble shoot and recovery command prompt, I have to mount usb and copied the shim file to the /boot/efi/EFI/centos. This method was tested by few people. One issue I have now is instead of seeing the boot option, it started with 'Grub>' prompt and I have to type 'normal' to enter the boot option. I am still trying to figure out how to fix this one. An alternate work-around to get bootable again is to get a recovery/live image that has a writable grub.conf, and insert only the boot entries from the broken system's grub.conf into the recovery system's grub.conf. This avoids many issues that must otherwise be resolved before one can work on a chrooted system. For example, we were blocked by the fact that previous CentOS 7 recovery images and live USBs were too old to load some kernel modules needed to connect and downgrade, and recent CentOS 8 images could not load others. Surprisingly, the VFAT module was one of these. For CentOS 8 just install the new shim-x64-15-15.el8_2.x86_64.rpm (http://mirror.centos.org/centos-8/8.2.2004/BaseOS/x86_64/os/Packages/shim-x64-15-15.el8_2.x86_64.rpm). Works like a charm. yum list grub2\* shim\* Installed Packages grub2-common.noarch 1:2.02-87.el8_2 @BaseOS grub2-efi-x64.x86_64 1:2.02-87.el8_2 @BaseOS grub2-pc.x86_64 1:2.02-87.el8_2 @BaseOS grub2-pc-modules.noarch 1:2.02-87.el8_2 @BaseOS grub2-tools.x86_64 1:2.02-87.el8_2 @BaseOS grub2-tools-efi.x86_64 1:2.02-87.el8_2 @BaseOS grub2-tools-extra.x86_64 1:2.02-87.el8_2 @BaseOS grub2-tools-minimal.x86_64 1:2.02-87.el8_2 @BaseOS shim-x64.x86_64 15-15.el8_2 @@commandline (In reply to Javier Martinez Canillas from comment #32) > Created attachment 1702984 [details] > shimx64.efi > > Please test with the shim EFI binary attached. > > It has to be copied to /boot/efi/EFI/redhat/ Hi I encountered this bug just a few days ago trying a clean install of CentOS 8.2 and did a yum update. I was like (WTF :/ What just happened) [Kidding !] and tried rescuing grub and yet no luck but thanks to this attached file I was able to successfully boot the OSs I have installed. Thanks again. Hope this fixture gets released on the stream so anyone who is experiencing this issue can fix it without having any difficulty. Worked On Lenovo G50-80 with bios version "b0cna0ww" (In reply to Logan from comment #65) > For CentOS 8 just install the new shim-x64-15-15.el8_2.x86_64.rpm > (http://mirror.centos.org/centos-8/8.2.2004/BaseOS/x86_64/os/Packages/shim- > x64-15-15.el8_2.x86_64.rpm). Works like a charm. Confirmed, I was able to do nothing but a "yum update" and got the package on my dead, then hacked-to-work system. It replaced the shimx64.efi file I downloaded from this bug report. I rebooted and it is fine and up-to-date. According to the time stamp, the file was updated on the mirrors on Saturday (Aug 01, 2020). Fast! I assume the same was/is true for RHEL repos. If so, I propose the bug report can now be resolved/closed. I can confirm that a RHEL 8.2 system with shim-x64-15-15.el8_2.x86_64.rpm installed and a RHEL 7.8 system with shim-x64-15-8.el7_8.x86_64.rpm installed both successfully reboot now. Hi, I noticed that after updating to: grub2 1:2.02-87 shim 15-15 mokutil --list-enrolled would output: MokListRT is empty This prevents kmod-drivers such as kmod-nvidia from loading in 8.2 and 7.8 and boot up doesn't complete. This is 100% reproducible for me at least. Fresh system install with either centos 7.8 or centos 8.2 with secure boot enabled. Using kmod-nvidia drivers from ELRepo on a Quadro K4200. It worked fine before this bug, I can see the keys before applying the update. (In reply to Clifford Perry from comment #57) > I just wanted to post a brief note that updated shim packages are now > available and can be used in conjunction with previously released grub2, > fwupd, and fwupdate packages. Both > https://access.redhat.com/security/vulnerabilities/grub2bootloader and > https://access.redhat.com/solutions/5272311 have been updated with links to > the new shim package bugfix errata. I wonder what happens if you try to install the ELRepo's SB key again. (In reply to Akemi Yagi from comment #71) > I wonder what happens if you try to install the ELRepo's SB key again. I tried that, and what happens is that it lets me enroll the key again even though it actually already exists but mokutil --list-enrolled continues to output MokListRT is empty. However upon trying on a fresh 8.2 before the bug, I see the key was duplicated. So not only the original key was always there, it duplicated the key without acknowledging it already existed. (In reply to R. Boza from comment #72) > (In reply to Akemi Yagi from comment #71) > > I wonder what happens if you try to install the ELRepo's SB key again. > > I tried that, and what happens is that it lets me enroll the key again even > though it actually already exists but mokutil --list-enrolled continues to > output MokListRT is empty. However upon trying on a fresh 8.2 before the > bug, I see the key was duplicated. So not only the original key was always > there, it duplicated the key without acknowledging it already existed. Strange indeed. What do you see with a "keyctl list %:.system_keyring" ? (In reply to Akemi Yagi from comment #73) > (In reply to R. Boza from comment #72) > > (In reply to Akemi Yagi from comment #71) > > > I wonder what happens if you try to install the ELRepo's SB key again. > > > > I tried that, and what happens is that it lets me enroll the key again even > > though it actually already exists but mokutil --list-enrolled continues to > > output MokListRT is empty. However upon trying on a fresh 8.2 before the > > bug, I see the key was duplicated. So not only the original key was always > > there, it duplicated the key without acknowledging it already existed. > > Strange indeed. What do you see with a "keyctl list %:.system_keyring" ? I was only able to test from 8.2: (it looks like keyctl list %:.system_keyring is now keyctl list %:.builtin_trusted_keys. https://bugzilla.redhat.com/show_bug.cgi?id=1509714 documentation error reported for Fedora 26+ fixed now) 3 keys in keyring: Centos Linux kernel Centos Linux kpatch Centos Linux Driver update So it is missing ELRepo, and a few other keys that would normally be there. I think something in the update is preventing those keys from being read. (In reply to R. Boza from comment #74) > (In reply to Akemi Yagi from comment #73) > > (In reply to R. Boza from comment #72) > > > (In reply to Akemi Yagi from comment #71) > > > > I wonder what happens if you try to install the ELRepo's SB key again. > > > > > > I tried that, and what happens is that it lets me enroll the key again even > > > though it actually already exists but mokutil --list-enrolled continues to > > > output MokListRT is empty. However upon trying on a fresh 8.2 before the > > > bug, I see the key was duplicated. So not only the original key was always > > > there, it duplicated the key without acknowledging it already existed. > > > > Strange indeed. What do you see with a "keyctl list %:.system_keyring" ? > > I was only able to test from 8.2: (it looks like keyctl list > %:.system_keyring is now keyctl list %:.builtin_trusted_keys. > https://bugzilla.redhat.com/show_bug.cgi?id=1509714 documentation error > reported for Fedora 26+ fixed now) > 3 keys in keyring: > Centos Linux kernel > Centos Linux kpatch > Centos Linux Driver update > > So it is missing ELRepo, and a few other keys that would normally be there. > I think something in the update is preventing those keys from being read. The output matches the one of a system with UEFI and Secure Boot disabled. However secureboot is enabled and I have doubled checked via BIOS settings and mokutil --sb-state. it should be something like: ...asymmetric: Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87... ...asymmetric: Red Hat Secure Boot (CA key 1): 4016841644ce3a810408050766e8f8a29... ...asymmetric: Microsoft Corporation UEFI CA 2011: 13adbf4309bd82709c8cd54f316ed... ...asymmetric: Microsoft Windows Production PCA 2011: a92902398e16c49778cd90f99e... ...asymmetric: Red Hat Enterprise Linux kernel signing key: 4249689eefc77e95880b... ...asymmetric: Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b7... ...ElRepo signing key... I believe it is supposed to display: asymmetric: The ELRepo Project (http://elrepo.org): ELRepo.org Secure Boot Key: f365ad3481a7b20e3427b61b2a26635b83fe427b This is something that needs investigation. Thank you for your input. Thank you for the above report! I have opened the following bug to address the MOK list failure. 1866107 – Following 1861977, the MOK list is inaccessible with "Couldn't get UEFI MokListRT" visible in the logs https://bugzilla.redhat.com/show_bug.cgi?id=1866107 This bug has also affected my RHEL 8.2 system. After POST, the GRUB menu is not displayed. I have SecureBoot disabled by default in my UEFI BIOS settings. The bug can affect you whether or not SB is enabled. Please follow the steps in https://access.redhat.com/solutions/5272311 to recover now. (In reply to James K from comment #53) > (In reply to Nerijus from comment #42) > > (In reply to Sandro Bonazzola from comment #6) > > > I booted my system with the installation ISO in troubleshoot mode, chrooted > > > to /mnt/sysimage, regenerated the grub.cfg with grub2-mkconfig and executed > > > grub2-install. > > > Now I can go past grub but got stuck to emergency shell failing swtching > > > root. > > > > Did the same, but disabled secure boot in BIOS. Ended up with a system that > > boots to `grub>` prompt. Entering `normal` at it resumes usual OS boot with > > image menu, etc. > > I am seeing the same thing. How did you fix it? Haven't found how to fix it. Still living with `grub>` prompt at boot time. The issue is still reproducible on my Macmini3,1 (Late 2009) machine with CentOS 8.2 installed. The system boots normally with shim-x64-15-11.el8, but doesn't boot with shim-x64-15-13.el8 or shim-x64-15-15.el8_2. In all cases grub2-efi-x64-2.02-87.el8_2 is installed. Still having this issue on a virtual machine running Centos 7.8.2003 running under a libvirt/kvm machine with the same centos version. I have not rebooted the server itself but the virtual machine does not boot (black screen after grub) with the latest kernel even though it already had the fixed shim package. (In reply to oktaya from comment #86) > Still having this issue on a virtual machine running Centos 7.8.2003 > running under a libvirt/kvm machine with the same centos version. I have > not rebooted the server itself but the virtual machine does not boot (black > screen after grub) with the latest kernel even though it already had the > fixed shim package. Hi! Yes, there seems to be still some issue. I'd suggest to watch the following solution article, which we'll keep updated: https://access.redhat.com/solutions/5385031 Consider this *not* solved, until this RHBZ has been closed! I hope this information helps! Regards, Oliver Reproduced the issue in VM using shim-x64-15-14.el8 Verified using shim-x64-15-15.el8 (and grub2-efi-x64-2.02-90) as I managed to reboot without any problems and can't reproduce the issue anymore, moving to Verified. @Siarhei since we don't have a Macmini to try to test the issue in your exact environment and we can't reproduce it, please submit a new bug if you continue to have problems the with the issue @oktaya your issue (black screen after grub) seems to be different than the one described: 'System hangs after POST and the grub menu never loads'. Please submit a new bug if needed. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (shim bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4574 |