Bug 2002499

Summary: Please build the RHEL 9 Beta kernel against the internal pesign targets
Product: Red Hat Enterprise Linux 9 Reporter: Brian Stinson <bstinson>
Component: kernelAssignee: Jan Stancek <jstancek>
kernel sub component: Packaging QA Contact: Jeff Bastian <jbastian>
Status: CLOSED CURRENTRELEASE Docs Contact: Sagar Dubewar <sdubewar>
Severity: unspecified    
Priority: unspecified CC: alexanderzz16122, dhoward, hkrzesin, jbastian, jstancek, mcermak, pvlasin, sdubewar, shangsong2
Version: 9.0Keywords: Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: kernel-5.14.0-1.6.1.el9 Doc Type: Enhancement
Doc Text:
.RHEL 9 Beta kernels signed with trusted SecureBoot certificates Previously, RHEL Beta releases required users to enroll a separate Beta public key using the Machine Owner Key (MOK) facility. Starting with RHEL 9 Beta, kernels are signed with trusted SecureBoot certificates, hence users no longer need to enroll a separate Beta public key to use the beta versions on systems having UEFI Secure Boot enabled.
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-11 17:42:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1951031    

Description Brian Stinson 2021-09-09 02:30:16 UTC
We are ready to build the kernel with the following certs:

x86_64: redhatsecureboot501
aarch64: redhatsecureboot501
ppc64le: redhatsecureboot601
s390x: redhatsecureboot302

We can embed these certificates in dist-git for now while we continue the single spec CentOS Stream/RHEL solution during the GA cycle.

Comment 10 Jeff Bastian 2021-09-22 20:16:10 UTC
Jan, I tried Secure Booting your non-scratch build from comment 9 on a Dell PowerEdge R7425 system, but it failed to boot with an error:

  Verification failed: (0x1A) Security Violation

The same system is successful with Secure Boot of RHEL-8.4.0 GA, though.  The Dell firmware is pre-loaded with Microsoft's and VMware's Secure Boot keys.

I noticed, however, that I don't even get to the grub menu when the "Verification failed" error screen pops up.  I wonder, do we have the shim and grub properly signed in these early RHEL9 builds?

Comment 11 Jeff Bastian 2021-09-22 20:29:08 UTC
I think the shim is ok since it's actually the RHEL8 version.  The RHEL9 grub binaries, however, are signed with a test certificate.  Is the test certificate expected to work?  I'll see if I can use the RHEL-8.4.0 grubx64.efi binary with RHEL9.


[root@dell-per7425-04 ~]# pesign --in /boot/efi/EFI/redhat/shimx64.efi --show-signature
---------------------------------------------
certificate address is 0x7f447eaf0c08
Content was not encrypted.
Content is detached; signature cannot be verified.
The signer's common name is Microsoft Windows UEFI Driver Publisher
No signer email address.
No signing time included.
There were certs or crls included.
---------------------------------------------


[root@dell-per7425-04 ~]# pesign --in /boot/efi/EFI/redhat/grubx64.efi --show-signature
---------------------------------------------
certificate address is 0x7fccc4eae008
Content was not encrypted.
Content is detached; signature cannot be verified.
The signer's common name is Red Hat Test Certificate
No signer email address.
Signing time: Mon May 03, 2021
There were certs or crls included.
---------------------------------------------
certificate address is 0x7fccc4eaeb00
Content was not encrypted.
Content is detached; signature cannot be verified.
The signer's common name is Red Hat Test Certificate
No signer email address.
Signing time: Mon May 03, 2021
There were certs or crls included.
---------------------------------------------

Comment 12 Jan Stancek 2021-09-22 20:35:22 UTC
Buildroot signing has been fixed only recently and there is only one grub2 build since then (grub2-2.06-2.el9_b), but looking at logs it doesn't seem to be picking correct cert still. I could try to rebuild grub2 or we can try with rhel8 version.

Comment 14 Jeff Bastian 2021-09-22 20:44:10 UTC
RHEL-8.4.0 grub2-efi-x64-2.02-99.el8.x86_64.rpm looks better and the system successfully Secure Boots to a grub> prompt.

[root@dell-per7425-04 redhat]# pesign --in grubx64.efi -S
---------------------------------------------
certificate address is 0x7f0170cfdc08
Content was not encrypted.
Content is detached; signature cannot be verified.
The signer's common name is Red Hat Secure Boot (signing key 1)
The signer's email address is secalert
Signing time: Thu Feb 25, 2021
There were certs or crls included.
---------------------------------------------
certificate address is 0x7f0170cfe5c0
Content was not encrypted.
Content is detached; signature cannot be verified.
The signer's common name is Red Hat Secure Boot Signing 502
The signer's email address is secalert
Signing time: Thu Feb 25, 2021
There were certs or crls included.
---------------------------------------------

[root@dell-per7425-04 redhat]# mv /boot/efi/EFI/redhat/grubx64.efi{,.ORIG}

[root@dell-per7425-04 redhat]# cp grubx64.efi /boot/efi/EFI/redhat/

[root@dell-per7425-04 redhat]# reboot
...


However, the RHEL-8.4.0 grub fails to read the config file due to an error:

error: ../../grub-core/kern/fs.c:120:unknown filesystem

I suspect this is due to bug 1977236 which enables some new XFS features and makes /boot un-mountable with older kernel and grub versions, and also Petitboot on OpenPOWER systems.

Let me try rebuilding /boot as ext4 next.

Comment 15 Jeff Bastian 2021-09-22 21:14:46 UTC
Using grub2-efi-x64-2.02-99.el8 and making /boot an ext4 filesystem did the trick: I was able to successfully SecureBoot into Jan's kernel from comment 9.

[root@dell-per7425-04 ~]# uname -r
5.14.0-1.onesig.el9.x86_64

[root@dell-per7425-04 ~]# journalctl --no-hostname --output short-monotonic --dmesg | grep -i -e secure -e lockdown
[    0.000000] kernel: secureboot: Secure boot enabled
[    0.000000] kernel: Kernel is locked down from EFI Secure Boot mode; see man kernel_lockdown.7
[    0.007409] kernel: secureboot: Secure boot enabled
[    9.416195] kernel: Lockdown: swapper/0: hibernation is restricted; see man kernel_lockdown.7
[   10.712855] kernel: megaraid_sas 0000:61:00.0: Secure JBOD support        : No
[   11.857939] kernel: Lockdown: systemd-hiberna: hibernation is restricted; see man kernel_lockdown.7
[   39.917123] kernel: Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7


Do you want me to test aarch64, ppc64le, and s390x before setting Verified:Tested?  I have not tried SecureBoot before on the non-x86 architectures so it may take me a few days to study the docs.  I know IBM has firmware ready for Secure Boot based on some meetings I've attended.  As for ARM, I randomly picked ampere-mtsnow-altra-04 from Beaker and poked through the UEFI menus and discovered it has an Ampere Platform Key installed, along with "Microsoft Corporation KEK CA 2011", and Authorized Signatures for Windows, SUSE, Canonical, and Fedora.  Presumably RHEL should work since the shim is signed with Microsoft's key.  I'll give it a try tomorrow.

Comment 29 Robbie Harwood 2021-10-04 18:54:37 UTC
*** Bug 2006898 has been marked as a duplicate of this bug. ***

Comment 33 Jeff Bastian 2021-10-05 20:53:31 UTC
x86_64 Secure Boot looks good with kernel-5.14.0-1.6.1.el9 (and a grub2-efi-x64 package which is awaiting QA testing).

[root@dell-per7425-04 ~]# rpm -q kernel grub2-efi-x64 shim-x64
kernel-5.14.0-1.6.1.el9.x86_64
grub2-efi-x64-2.06-5.el9.x86_64
shim-x64-15-16.el8.x86_64

[root@dell-per7425-04 ~]# uname -r
5.14.0-1.6.1.el9.x86_64

[root@dell-per7425-04 ~]# journalctl --no-hostname --output short-monotonic --dmesg | grep -i -e secure -e lockdown
[    0.000000] kernel: secureboot: Secure boot enabled
[    0.000000] kernel: Kernel is locked down from EFI Secure Boot mode; see man kernel_lockdown.7
[    0.006694] kernel: secureboot: Secure boot enabled
[    9.293660] kernel: integrity: Loaded X.509 cert 'VMware, Inc.: VMware Secure Boot Signing: 04597f3e1ffb240bba0ff0f05d5eb05f3e15f6d7'
[    9.321057] kernel: integrity: Loaded X.509 cert 'Red Hat Secure Boot CA 5: cc6fa5e72868ba494e939bbd680b9144769a9f8f'
[    9.460531] kernel: Lockdown: swapper/0: hibernation is restricted; see man kernel_lockdown.7
[   11.024745] kernel: megaraid_sas 0000:61:00.0: Secure JBOD support        : No
[   11.967170] kernel: Lockdown: systemd-hiberna: hibernation is restricted; see man kernel_lockdown.7

Comment 36 Jan Stancek 2021-10-19 06:39:02 UTC
*** Bug 1964807 has been marked as a duplicate of this bug. ***

Comment 42 Dimas Alexander 2021-12-22 22:42:06 UTC
Hello, from the release notes, I've seen "Starting with RHEL 9 Beta, kernels are signed with trusted SecureBoot certificates, hence users no longer need to enroll a separate Beta public key to use the beta versions on systems having UEFI Secure Boot enabled", but i still can't get secure boot worked on my machine before / after enrolled key. 

I've verified that key is listed, but still got "Verification failed: (0x1A) Security Violation"

[root@amnesia redhat]# rpm -q kernel grub2-efi-x64 shim-x64
kernel-5.14.0-21.el9.x86_64
grub2-efi-x64-2.06-13.el9.x86_64
shim-x64-15-16.el8.x86_64

and btw i'm dual boot with win11, win11 can boot but not with red hat, any idea?