Bug 2030940

Summary:	Booting with grub2-2.06-9.fc35 and UEFI Secure Boot enabled resulted in Error: Verification Failed: (0x1A) Security Violation
Product:	[Fedora] Fedora	Reporter:	Matt Fagnani <matt.fagnani>
Component:	grub2	Assignee:	Javier Martinez Canillas <fmartine>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	35	CC:	accts, fmartine, ken, lkundrak, pgnet.dev, pjones, pmendezh, rharwood, sanjay.ankur, thomas
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	grub2-2.06-11.fc36 grub2-2.06-10.fc35 grub2-2.06-9.fc34	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-12-12 01:10:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Matt Fagnani 2021-12-10 02:46:55 UTC

Description of problem:

I updated a Fedora 35 KDE Plasma installation with updates-testing enabled on an hp laptop with UEFI Secure Boot enabled using
sudo dnf offline-upgrade download
sudo dnf offline-upgrade reboot

The update included grub2-2.06-9.fc35 and about 250 other rpms. The update completed normally, and the system rebooted.

Error: Verification Failed: (0x1A) Security Violation was shown on a blue screen which I guess is a UEFI message before grub would have normally appeared. Pressing enter for OK on that screen showed a screen with
Shim UEFI Key Management
Press any key to perform MOK management

The following error messages were also shown
Failed to load image: Security Policy Violation
start_image() returned Security Policy Violation

After these errors happened on each of the next few boots, I disabled UEFI Secure Boot from within the BIOS settings. grub appeared normally with UEFI Secure Boot disabled and the system booted properly. I downgraded to grub2*-2.06-8.fc35 with sudo dnf downgrade grub2* I rebooted, and I enabled UEFI Secure Boot. The system booted normally with grub2-common-2.06-8.fc35 and UEFI Secure Boot.

Version-Release number of selected component (if applicable):
grub2-2.06-9.fc35

How reproducible:
The boot errors occurred 4/4 boots with grub2-2.06-9.fc35

Steps to Reproduce:
1. Boot a Fedora 35 KDE Plasma installation with UEFI Secure Boot enabled
2. Start Plasma on Wayland
3. Start konsole
4. update the system with updates-testing enabled using
sudo dnf offline-upgrade download
sudo dnf offline-upgrade reboot

Actual results:

Booting with grub2-2.06-9.fc35 and UEFI Secure Boot enabled resulted in Error: Verification Failed: (0x1A) Security Violation

Expected results:
The system would boot normally.

Additional info:

Comment 1 Ken Fallon 2021-12-10 10:54:48 UTC

I've been running a version of Fedora 35 and after todays upgrade I started getting the error Verification failed: (0x1A) Security Violation. 

Removed secure boot allowed the system to boot.

Comment 2 Ankur Sinha (FranciscoD) 2021-12-10 12:30:26 UTC

I can confirm this bug too. Just upgraded and ran into it.

Increasing severity since this is going to break systems for lots of users. From the dnf transaction log:



    Upgrade       grub2-common-1:2.06-9.fc35.noarch                             @updates-testing                                        
    Upgraded      grub2-common-1:2.06-8.fc35.noarch                             @@System                                                
    Upgrade       grub2-efi-ia32-1:2.06-9.fc35.x86_64                           @updates-testing                                        
    Upgraded      grub2-efi-ia32-1:2.06-8.fc35.x86_64                           @@System                                                
    Upgrade       grub2-efi-ia32-cdboot-1:2.06-9.fc35.x86_64                    @updates-testing                                        
    Upgraded      grub2-efi-ia32-cdboot-1:2.06-8.fc35.x86_64                    @@System                                                
    Upgrade       grub2-efi-x64-1:2.06-9.fc35.x86_64                            @updates-testing                                        
    Upgraded      grub2-efi-x64-1:2.06-8.fc35.x86_64                            @@System                                                
    Upgrade       grub2-efi-x64-cdboot-1:2.06-9.fc35.x86_64                     @updates-testing                                        
    Upgraded      grub2-efi-x64-cdboot-1:2.06-8.fc35.x86_64                     @@System                                                
    Upgrade       grub2-pc-1:2.06-9.fc35.x86_64                                 @updates-testing                                        
    Upgraded      grub2-pc-1:2.06-8.fc35.x86_64                                 @@System                                                
    Upgrade       grub2-pc-modules-1:2.06-9.fc35.noarch                         @updates-testing                                        
    Upgraded      grub2-pc-modules-1:2.06-8.fc35.noarch                         @@System                                                
    Upgrade       grub2-tools-1:2.06-9.fc35.x86_64                              @updates-testing                                        
    Upgraded      grub2-tools-1:2.06-8.fc35.x86_64                              @@System                                                
    Upgrade       grub2-tools-efi-1:2.06-9.fc35.x86_64                          @updates-testing                                        
    Upgraded      grub2-tools-efi-1:2.06-8.fc35.x86_64                          @@System                                                
    Upgrade       grub2-tools-extra-1:2.06-9.fc35.x86_64                        @updates-testing                                        
    Upgraded      grub2-tools-extra-1:2.06-8.fc35.x86_64                        @@System                                                
    Upgrade       grub2-tools-minimal-1:2.06-9.fc35.x86_64                      @updates-testing                                        
    Upgraded      grub2-tools-minimal-1:2.06-8.fc35.x86_64                      @@System

Comment 3 Fedora Update System 2021-12-10 18:07:54 UTC

FEDORA-2021-8dbf0a81c0 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-8dbf0a81c0

Comment 4 Fedora Update System 2021-12-10 18:07:57 UTC

FEDORA-2021-73d63662b0 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-73d63662b0

Comment 5 Fedora Update System 2021-12-11 01:16:05 UTC

FEDORA-2021-8dbf0a81c0 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-8dbf0a81c0`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-8dbf0a81c0

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Fedora Update System 2021-12-11 01:58:57 UTC

FEDORA-2021-73d63662b0 has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-73d63662b0`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-73d63662b0

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2021-12-12 01:10:18 UTC

FEDORA-2021-8dbf0a81c0 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 8 Tom Gugel 2021-12-12 21:17:31 UTC

Hi, I found this report and thought I might add that here, as it seems a closely related problem.
Since the update I cannot boot into a single btrfs snapshot anymore.

This is not fixable, not through a reinstall of all shim-* and/or grub2-* packages, not with recreating grub.cfg.

All the snapshots are dead essentially with the same error "Unknown TPM error..."
Only the active version can be booted, if you want to boot the active version after selecting a snapshot in grub, and getting the above unknown TPM error, also the active version cannot be booted anymore, resulting in the same error.

After that only a system reset makes the active version bootable again. All snapshots stay dead anyways.

Snapshot handling seems completely broken as also newly created snapshots cannot be booted into. The system has to be reinstalled as it seems.

I read about similar issues already in the past on Fedora, they all had to do with updated packages of grub or shim.

Are there any ideas regarding that? Because if the complete snapshot system is messed up everytime something happens with grub or shim I will have to switch away from Fedora for my production workstations.

Thanks

Comment 9 Tom Gugel 2021-12-12 21:43:31 UTC

The exact error is error: ../../grub-core/commands/efi/tpm.c:148:Unknown TPM error
The exact error is error: ../../grub-core/loader/i386/efi/linux.c:208:you need to load the kernel first 
So for some reason it seems to not find the kernel anymore
This also happens with the working active version as said if you want to boot a snapshot before and you would have to completely reset the system.
It worked with 2.06-8

Comment 10 Matt Fagnani 2021-12-12 22:42:41 UTC

(In reply to Tom Gugel from comment #8)
> Hi, I found this report and thought I might add that here, as it seems a
> closely related problem.
> Since the update I cannot boot into a single btrfs snapshot anymore.
> 
> This is not fixable, not through a reinstall of all shim-* and/or grub2-*
> packages, not with recreating grub.cfg.
> 
> All the snapshots are dead essentially with the same error "Unknown TPM
> error..."
> Only the active version can be booted, if you want to boot the active
> version after selecting a snapshot in grub, and getting the above unknown
> TPM error, also the active version cannot be booted anymore, resulting in
> the same error.
> 
> After that only a system reset makes the active version bootable again. All
> snapshots stay dead anyways.
> 
> Snapshot handling seems completely broken as also newly created snapshots
> cannot be booted into. The system has to be reinstalled as it seems.
> 
> I read about similar issues already in the past on Fedora, they all had to
> do with updated packages of grub or shim.
> 
> Are there any ideas regarding that? Because if the complete snapshot system
> is messed up everytime something happens with grub or shim I will have to
> switch away from Fedora for my production workstations.
> 
> Thanks

The grub2-2.06-10.fc35 update fixed the boot failure with Error: Verification Failed: (0x1A) Security Violation for me and others https://bodhi.fedoraproject.org/updates/FEDORA-2021-8dbf0a81c0 The problem occurred due to an issue with the update of the Fedora Secure Boot builders according to https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/BGL4W2RLKKSLTGD4GRXIIV4Q6RLHLN24/ Your problem looks like it might be different from this one as it involves btrfs snapshots. I don't use any btrfs partitions. My / and /boot are ext4. If your problem still happens with grub2-2.06-10.fc35, I'd suggest that you report your problem as a new bug against grub2.

Comment 11 Tom Gugel 2021-12-13 05:48:53 UTC

Thanks for you reply. This is the thing, it has nothing to do with the grub2 version per se, it has something to do with what happens a new version of grub2 installed, as it already happened in the past. That is, if you reinstall the whole system it works without any flaws, also with 2.06-10, as it did with 2.06-8. Until you install/update the fedora grub-* packages to the next version. It is unfixable broken then. There must be something happening on Fedora when doing this, maybe it has also to do with the specific MB/CPU combination (AMD Firmware TPM, TR3960X) and what is done on Fedora specifically. As this renders the whole concept of snapshots unusable and a system reinstall because of a package upgrade is unbearable I hope it is found out what causes this. It not only affects me. I will file a new bug. Thanks

Comment 12 Tom Gugel 2021-12-13 06:21:34 UTC

This is now described in https://bugzilla.redhat.com/show_bug.cgi?id=2031640

Comment 13 Fedora Update System 2021-12-26 01:09:25 UTC

FEDORA-2021-73d63662b0 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.