Bug 2187644

Summary:	SEV-es guest failed to find the boot disk when reseting nvram
Product:	Red Hat Enterprise Linux 9	Reporter:	zixchen
Component:	edk2	Assignee:	Gerd Hoffmann <kraxel>
Status:	CLOSED NOTABUG	QA Contact:	zixchen
Severity:	medium	Docs Contact:
Priority:	medium
Version:	9.3	CC:	berrange, jinzhao, kraxel, pbonzini, vgoyal, virt-maint, xuwei, zhguo
Target Milestone:	rc	Keywords:	CustomerScenariosInitiative, Triaged
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-06-27 03:56:58 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description zixchen 2023-04-18 09:58:43 UTC

Description of problem:
Virsh start a SEV VM with --reset-nvram failed to find the booting disk. 

Version-Release number of selected component (if applicable):
edk2-ovmf-20230301gitf80f052277c8-1.el9.noarch
kernel-5.14.0-295.el9.x86_64
qemu-kvm-7.2.0-14.el9_2.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Boot a sev/sev-es VM with libvirt, check sev/sev-es enabled then destroy the guest.
2. virsh start $name --reset-nvram


Actual results:
VM boot failed with boot option restoration and then VM quits.
 
Expected results:
Nvram can be reset.

Additional info:
after Step 1, remove sev configs in xml then #virsh start $name --reset-nvram, vm fails with the same issue.

Comment 2 Gerd Hoffmann 2023-04-18 10:19:07 UTC

Relevant upstream discussion:
https://edk2.groups.io/g/devel/message/103023

Seems the root cause is memory encryption attributes for variable flash can be wrong.

Comment 3 Gerd Hoffmann 2023-04-18 10:20:59 UTC

(In reply to Gerd Hoffmann from comment #2)
> Relevant upstream discussion:

Oops, wrong thread, here is the real one:
https://edk2.groups.io/g/devel/message/102098

Comment 5 Gerd Hoffmann 2023-06-01 08:45:30 UTC

Status update: edk2-stable202305 released without upstream fix.
Increased risk this will miss the 9.3 boat.

Comment 6 Gerd Hoffmann 2023-06-01 09:17:17 UTC

Testbuild:
https://kojihub.stream.centos.org/koji/taskinfo?taskID=2275934
Any change in behavior?

Comment 7 zixchen 2023-06-02 04:00:06 UTC

(In reply to Gerd Hoffmann from comment #6)
> Testbuild:
> https://kojihub.stream.centos.org/koji/taskinfo?taskID=2275934
> Any change in behavior?

No change for this,  virsh start $name --reset-nvram still failed with boot option restoration and then VM quits.

Version:
kernel-5.14.0-319.el9.x86_64
qemu-kvm-8.0.0-4.el9.x86_64
edk2-ovmf-20230301gitf80f052277c8-4.el9.bz2187644.20230601.1109.noarch

Upload a firmware log: nvram_reset.log

Comment 9 Gerd Hoffmann 2023-06-22 08:49:29 UTC

Is this a regression?
What happens if you try boot the VM twice, first time with --reset-nvram and second time without?

Comment 10 zixchen 2023-06-25 09:18:38 UTC

(In reply to Gerd Hoffmann from comment #9)
> Is this a regression?
It is not a regression, I test edk2-ovmf-20210527gite1999b264f1f-2.el9.noarch, and the issue reproduces. 

> What happens if you try boot the VM twice, first time with --reset-nvram and
> second time without?
The second time boot with #virsh start $guest_name, the VM can boot. 


After reset nvram of sev-es VM, the guests boot to "Boot Option Restoration" screen, if you select 'system reset' or wait for timeout to reset system, the VM will quit. If you select continue boot, then the guest van boots to the system. Sev-es doesn't support reboot, is that the reason why VM quit after resting nvram? Does this mean sev-es VM doesn't support reset nvram?

Comment 11 Gerd Hoffmann 2023-06-26 11:44:44 UTC

(In reply to zixchen from comment #10)
> (In reply to Gerd Hoffmann from comment #9)
> > Is this a regression?
> It is not a regression, I test
> edk2-ovmf-20210527gite1999b264f1f-2.el9.noarch, and the issue reproduces. 
> 
> > What happens if you try boot the VM twice, first time with --reset-nvram and
> > second time without?
> The second time boot with #virsh start $guest_name, the VM can boot. 
> 
> 
> After reset nvram of sev-es VM, the guests boot to "Boot Option Restoration"
> screen, if you select 'system reset' or wait for timeout to reset system,
> the VM will quit. If you select continue boot, then the guest van boots to
> the system. Sev-es doesn't support reboot, is that the reason why VM quit
> after resting nvram?

I think this is exactly what is happening.  guest restores boot options.
guest wants reboot, but because sev-es doesn't support that the vm is
powered off instead.  When starting it again it boots fine (using the
boot options restored on first start).

> Does this mean sev-es VM doesn't support reset nvram?

resetting vmram works.  But you need to start the vm twice because
sev-es does not support reboots.

I think you should update the test case accordingly.

Comment 12 zixchen 2023-06-27 03:56:58 UTC

Thanks Gerd for confirming, I update the reset nvram case, and close this as not a bug.