Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2040307

Summary: grub2 2.06-13.el9 update breaks RHEL 9 secureboot
Product: Red Hat Enterprise Linux 9 Reporter: Oliver Ilian <oliver>
Component: grub2Assignee: Bootloader engineering team <bootloader-eng-team>
Status: CLOSED CURRENTRELEASE QA Contact: Release Test Team <release-test-team-automation>
Severity: high Docs Contact:
Priority: high    
Version: 9.0CC: rharwood, sbarcomb
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-17 21:27:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2037084    
Bug Blocks:    

Description Oliver Ilian 2022-01-13 12:39:10 UTC
Description of problem:
After updating an RHEL 9 Beta install (UEFI / SecureBoot enabled) with the latest content from Satellite, the machine is no longer booting

Version-Release number of selected component (if applicable):
grub2: 2.06-13.el9
kernel-5.14.0-1.7.1.el9.x86_64.rpm

How reproducible:
always

Steps to Reproduce:
1. Enable UEFI and SecureBoot
2. Install an RHEL 9 Beta system (I used rhel-baseos-9.0-beta-0-x86_64-dvd.iso)
3. subscribe system to Satellite with updated RHEL 9 beta repos
4. update grub2* via yum

==================================================================================================
 Package                  Arch      Version          Repository                            Size
==================================================================================================
Upgrading:
 grub2-common             noarch    1:2.06-13.el9    rhel-9-for-x86_64-baseos-beta-rpms    936 k
 grub2-efi-x64            x86_64    1:2.06-13.el9    rhel-9-for-x86_64-baseos-beta-rpms    1.3 M
 grub2-tools              x86_64    1:2.06-13.el9    rhel-9-for-x86_64-baseos-beta-rpms    1.9 M
 grub2-tools-minimal      x86_64    1:2.06-13.el9    rhel-9-for-x86_64-baseos-beta-rpms    609 k
Installing dependencies:
 grub2-tools-efi          x86_64    1:2.06-13.el9    rhel-9-for-x86_64-baseos-beta-rpms    543 k
 grub2-tools-extra        x86_64    1:2.06-13.el9    rhel-9-for-x86_64-baseos-beta-rpms    847 k

Transaction Summary
==================================================================================================
Install  2 Packages
Upgrade  4 Packages

Total download size: 6.1 M

Actual results:
- after the grub2 update, the system is no longer rebooting and shows the message right after boot:
     Verification failed: (0x1A) Security Violation

Expected results:
System boots in secure boot


Additional info:
- I tested the issue in a KVM machine. A customer also sees the issue in VCenter.
- Disabling SecureBoot allows the system to boot again

Comment 1 Robbie Harwood 2022-01-13 14:47:26 UTC
-13 is a version that is known to not be signed with the right keys.  Please retest with the latest version - i.e., >= -16.

Comment 2 Oliver Ilian 2022-01-13 16:16:40 UTC
Thanks for the info. -16 indeed works.

Do you know when -16 will be available, so it can be synced into the customer's Satellite server?

Also.. I think that might be a kernel topic, but after subscribing directly to subscription.rhsm.redhat.com and updating I get kernel 5.14.0-39.el9. I manually install grub2 *-16 and on reboot, the new kernel is not booting (the previous kernel works)

Error:

error:../,,/grub-core/kern/efi/sb.c:150:bad shim signature
error:../,,/grub-core/loader/i386/efi/linux.c:208:you need to load the kernel first.
Press any key to continue

I checked for a newer shim version, but could not find any (currently shim-x64-15-16.el8

Thanks for the feedback

Comment 3 Robbie Harwood 2022-01-13 19:41:06 UTC
> Do you know when -16 will be available, so it can be synced into the customer's Satellite server?

You'd need to ask a release engineering type person that - from engineering's perspective, -16 is ready to go for GA, and -11 was the last version slated for Beta.

> Also.. I think that might be a kernel topic, but after subscribing directly to subscription.rhsm.redhat.com and updating I get kernel 5.14.0-39.el9. I manually install grub2 *-16 and on reboot, the new kernel is not booting (the previous kernel works)

Interesting, does kernel >= 5.14.0-40.el9 work?  (With grub >= 2.06-16.el9)

Comment 4 Oliver Ilian 2022-01-14 08:22:09 UTC
(In reply to Robbie Harwood from comment #3)
> > Do you know when -16 will be available, so it can be synced into the customer's Satellite server?
> 
> You'd need to ask a release engineering type person that - from
> engineering's perspective, -16 is ready to go for GA, and -11 was the last
> version slated for Beta.

Thanks.. Understood. Now I am wondering if -11 was the last version meant for the beta, how did -12 made it into the beta

> 
> > Also.. I think that might be a kernel topic, but after subscribing directly to subscription.rhsm.redhat.com and updating I get kernel 5.14.0-39.el9. I manually install grub2 *-16 and on reboot, the new kernel is not booting (the previous kernel works)
> 
> Interesting, does kernel >= 5.14.0-40.el9 work?  (With grub >= 2.06-16.el9)

Nope, does not work with 5.14.0-40.el9 or 5.14.0-42.el9

Comment 5 Robbie Harwood 2022-01-14 19:14:32 UTC
> Thanks.. Understood. Now I am wondering if -11 was the last version meant for the beta, how did -12 made it into the beta

Unsure, but based on the kernel versions you're seeing, I suspect you're not actually running beta - they aren't targeted for beta either.

In any case, I think this needs to wait on https://bugzilla.redhat.com/show_bug.cgi?id=2037084