Bug 1862045 - Grub or Shim dies since updating to grub2-2.02-0.86.el7_8 / shim-x64-15-7.el7_8
Summary: Grub or Shim dies since updating to grub2-2.02-0.86.el7_8 / shim-x64-15-7.el7_8
Keywords:
Status: MODIFIED
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: shim
Version: 7.8
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: Bootloader engineering team
QA Contact: Release Test Team
URL:
Whiteboard:
Depends On:
Blocks: 1862280 1862346
TreeView+ depends on / blocked
 
Reported: 2020-07-30 09:36 UTC by Renaud Métrich
Modified: 2020-08-12 14:03 UTC (History)
53 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1862280 1862346 (view as bug list)
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)
1st error message (21.95 KB, image/png)
2020-07-30 09:39 UTC, Renaud Métrich
no flags Details
2nd error message (23.29 KB, image/png)
2020-07-30 09:40 UTC, Renaud Métrich
no flags Details
dmidecode of HPE hardware hitting this (54.64 KB, text/plain)
2020-07-30 10:05 UTC, Renaud Métrich
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 5272311 None None None 2020-07-30 10:51:55 UTC

Internal Links: 1861977

Description Renaud Métrich 2020-07-30 09:36:29 UTC
Description of problem:

A customer sees a screen of death in Grub or Shim on HP xl230k hardware since he rebooted after updating to latest kernel + Grub2 + Shim fixing Boot Hold Vulnerability:

  https://access.redhat.com/security/vulnerabilities/grub2bootloader

I don't know if Secure Boot is enabled, I will get a sosreport soon.

We were able to recover by keeping latest kernel but downgrading grub2, shim and mokutil packages to previous release.

Screenshots attached.


Version-Release number of selected component (if applicable):

grub2-2.02-0.86.el7_8 / shim-x64-15-7.el7_8


How reproducible:

Always for customer

Comment 2 Renaud Métrich 2020-07-30 09:39:38 UTC
Created attachment 1702918 [details]
1st error message

Comment 3 Renaud Métrich 2020-07-30 09:40:02 UTC
Created attachment 1702919 [details]
2nd error message

Comment 4 Renaud Métrich 2020-07-30 09:50:18 UTC
Looks like BZ 1861977 is the same for RHEL8

Comment 5 Renaud Métrich 2020-07-30 10:05:38 UTC
Created attachment 1702922 [details]
dmidecode of HPE hardware hitting this

Comment 6 Renaud Métrich 2020-07-30 10:07:19 UTC
Hit on HPE (see attached dmidecode) with SecureBoot disabled.

Comment 9 Larry Fahnoe 2020-07-30 17:30:22 UTC
Although running CentOS 7, my systems were also impacted by this issue. For reference my hardware is:

Dell T110-II
BIOS 2.10.0
Secure Boot off (I think but did not find it explicitly listed in the BIOS settings)
UEFI boot
Updates applied via yum-cron

My steps to a temporary work-around:
1) boot rescue disk
2) chroot /mnt/sysimage
3) ip address add...
4) ip route add...
5) update /etc/resolv.conf if necessary
6) yum downgrade shim-x64 mokutil grub2\*
7) restore resolve.conf if updated
8) exit
9) reboot

System now boots and runs with the following:
grub2-common-2.02-0.81.el7.centos.noarch
grub2-efi-x64-2.02-0.81.el7.centos.x86_64
grub2-tools-2.02-0.81.el7.centos.x86_64
grub2-tools-extra-2.02-0.81.el7.centos.x86_64
grub2-tools-minimal-2.02-0.81.el7.centos.x86_64
mokutil-15-2.el7.centos.x86_64
shim-x64-15-2.el7.centos.x86_64

Should probably also blacklist the above packages until this is properly resolved.

--Larry

Comment 20 Wayne Feick 2020-07-31 20:15:38 UTC
I had problems running the downgrade command on CentOS, failing with a complaint about already having the minimum version. I manually downloaded the older versions of the packages from the base repository and specified those downloaded files on the yum downgrade command line. After that, I successfully rebooted, waited for the selinux relabel and reboot, and then things came up fine.

Comment 21 uli9000 2020-08-01 01:29:37 UTC
Problem: Cannot boot past bios after update. No select kernel options.

Workaround: Boot from live usb. install timeshift from epel. revert to pre-existing, pre-update snapshot. remove live usb. reboot.

OS: centos 7

Bios info:
	Vendor: American Megatrends Inc.
	Version: 2304
	Release Date: 10/09/2014
	Address: 0xF0000
	Runtime Size: 64 kB
	ROM Size: 8192 kB
	Characteristics:
		PCI is supported
		APM is supported
		BIOS is upgradeable
		BIOS shadowing is allowed
		Boot from CD is supported
		Selectable boot is supported
		BIOS ROM is socketed
		EDD is supported
		5.25"/1.2 MB floppy services are supported (int 13h)
		3.5"/720 kB floppy services are supported (int 13h)
		3.5"/2.88 MB floppy services are supported (int 13h)
		Print screen service is supported (int 5h)
		8042 keyboard services are supported (int 9h)
		Serial services are supported (int 14h)
		Printer services are supported (int 17h)
		ACPI is supported
		USB legacy is supported
		BIOS boot specification is supported
		Targeted content distribution is supported
		UEFI is supported
	BIOS Revision: 4.6

BIOS Language Information
	Language Description Format: Long
	Installable Languages: 3
		en|US|iso8859-1
		fr|FR|iso8859-1
		zh|CN|unicode
	Currently Installed Language: en|US|iso8859-1


hardware info:

description: Motherboard
       product: Z97-C
       vendor: ASUSTeK COMPUTER INC.
       physical id: 0
       version: Rev X.0x
       serial: 140425266205696
       slot: To be filled by O.E.M.
     *-firmware
          description: BIOS
          vendor: American Megatrends Inc.
          physical id: 0
          version: 2304
          date: 10/09/2014
          size: 64KiB
          capacity: 8MiB
          capabilities: pci apm upgrade shadowing cdboot bootselect socketedrom edd int13floppy1200 int13floppy720 int13floppy2880 int5printscreen int9keyboard int14serial int17printer acpi usb biosbootspecification uefi


Observations:
Received a warning re epel on previous (or near to it) attempt to update.

Comment 22 Clifford Perry 2020-08-01 14:46:39 UTC
I just wanted to post a brief note that updated shim packages are now available and can be used in conjunction with previously released grub2, fwupd, and fwupdate packages. Both https://access.redhat.com/security/vulnerabilities/grub2bootloader and https://access.redhat.com/solutions/5272311 have been updated with links to the new shim package bugfix errata.

Comment 23 Olimp Bockowski 2020-08-03 09:38:09 UTC
@Clifford
What about CentOS?

Comment 24 Clifford Perry 2020-08-03 15:19:00 UTC
(In reply to Olimp Bockowski from comment #23)
> @Clifford
> What about CentOS?

Released over the weekend as well. 
For EL 7 - the -8 packages released to address the bug within the -7. 

http://mirror.centos.org/centos/7/updates/x86_64/Packages/
 - shim-x64-15-8.el7.x86_64.rpm	2020-07-31 23:30	680K	
 

For EL8 - the -15 packages released to address the bug within the -14. 

http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/
 - shim-x64-15-15.el8_2.x86_64.rpm	2020-08-01 02:35	666K	  

Thanks,
Cliff


Note You need to log in before you can comment on or make changes to this bug.