Bug 2090464

Summary: AMD-SEV enabled guest fails to schedule due to UEFINotSupported exception
Product: Red Hat OpenStack Reporter: James Parker <jparker>
Component: openstack-novaAssignee: OSP DFG:Compute <osp-dfg-compute>
Status: CLOSED NOTABUG QA Contact: OSP DFG:Compute <osp-dfg-compute>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 17.0 (Wallaby)CC: alifshit, dasmith, eglynn, igallagh, jhakimra, kchamart, pgrist, sbaker, sbauza, sgordon, vromanso
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-06-08 15:22:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2090752    
Bug Blocks:    

Description James Parker 2022-05-25 19:51:01 UTC
Description of problem: Scheduling an amd-sev guest fails due to UEFINotSupported. Specifically the failure happens when iteration over:

[root@computeamdsev-0 firmware]# pwd
/usr/share/qemu/firmware
[root@computeamdsev-0 firmware]# ls
40-edk2-ovmf-sb.json  50-edk2-ovmf-amdsev.json	50-edk2-ovmf-cc.json  50-edk2-ovmf.json

from [1].  The machine returned in get_loader [2] is pc-q35-rhel9.0.0, but the machines in the qemu firmware directory are either pc-q35-rhel8.5.0 or pc-q35-*. For the machines returned with the pattern pc-q35-* their features include secure_boot. Since the request to get_loaders has secure_boot as false no loaders are found and the exception is raised.

[root@computeamdsev-0 firmware]# grep -r pc-q35-* *
40-edk2-ovmf-sb.json:                "pc-q35-*"
50-edk2-ovmf-amdsev.json:                "pc-q35-rhel8.5.0"
50-edk2-ovmf-cc.json:                "pc-q35-rhel8.5.0"
50-edk2-ovmf.json:                "pc-q35-*"
[root@computeamdsev-0 firmware]# grep -rl secure-boot *
40-edk2-ovmf-sb.json
50-edk2-ovmf.json

Version-Release number of selected component (if applicable):
RHOS-17.0-RHEL-9-20220517.n.1

How reproducible:
100%

Steps to Reproduce:
1. Attempt to create an amd sev guest on a supported deployment
2.
3.

Actual results:
Guest fails to schedule with Instance failed to spawn: nova.exception.UEFINotSupported: UEFI is not supported

Expected results:
Guest is scheduled without any issues

Additional info:
Bed can be made available upon request
[1] https://github.com/openstack/nova/blob/c3ad968c875ae7fa67e7a1f8d321f1fc622897bc/nova/virt/libvirt/host.py#L1764
[2] https://github.com/openstack/nova/blob/c3ad968c875ae7fa67e7a1f8d321f1fc622897bc/nova/virt/libvirt/host.py#L1762

Comment 1 James Parker 2022-05-25 20:36:51 UTC
Quick note, image is using 'q35'

(overcloud) [stack@undercloud-0 ~]$ openstack image show cirros-0.5.2-x86_64-disk.img
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field            | Value                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checksum         | b874c39491a2377b8490f5f1e89761a4                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| container_format | bare                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| created_at       | 2022-05-23T21:31:24Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| disk_format      | qcow2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| file             | /v2/images/659c0249-29f4-5ae7-9295-cacf1176422f/file                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| id               | 659c0249-29f4-5ae7-9295-cacf1176422f                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| min_disk         | 0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| min_ram          | 0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| name             | cirros-0.5.2-x86_64-disk.img                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| owner            | a2552c854ca74e80afa16e00aaaab68d                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| properties       | direct_url='swift+config://ref1/glance/659c0249-29f4-5ae7-9295-cacf1176422f', hw_firmware_type='uefi', hw_machine_type='q35', os_hash_algo='sha512', os_hash_value='6b813aa46bb90b4da216a4d19376593fa3f4fc7e617f03a92b7fe11e9a3981cbe8f0959dbebe36225e5f53dc4492341a4863cac4ed1ee0909f3fc78ef9c3e869', os_hidden='False', owner_specified.openstack.md5='', owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk.img', owner_specified.openstack.sha256='', stores='default_backend' |
| protected        | False                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| schema           | /v2/schemas/image                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| size             | 16300544                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| status           | active                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| tags             |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| updated_at       | 2022-05-23T21:31:35Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| virtual_size     | 117440512                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| visibility       | public                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Changing the image to use  pc-q35-rhel8.5.0 instead of just q35 results in the below failure: (If this needs to be tracked in a separate BZ I can open one)

nova/nova-compute.log:2022-05-25 20:30:47.428 2 DEBUG oslo_concurrency.lockutils [req-79627bb6-1e33-4799-9952-27a5e608efbc 09699a87ba594d518aa8d3b76f297536 a2552c854ca74e80afa16e00aaaab68d - default default] Lock "ddb52a03-8e92-4387-a7d9-5b580e03729a" released by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 2.619s inner /usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py:367
[root@computeamdsev-1 containers]# grep -r nvram-template *
nova/nova-compute.log:2022-05-25 20:30:46.726 2 ERROR nova.compute.manager [req-79627bb6-1e33-4799-9952-27a5e608efbc 09699a87ba594d518aa8d3b76f297536 a2552c854ca74e80afa16e00aaaab68d - default default] [instance: ddb52a03-8e92-4387-a7d9-5b580e03729a] Instance failed to spawn: KeyError: 'nvram-template'
nova/nova-compute.log:2022-05-25 20:30:46.726 2 ERROR nova.compute.manager [instance: ddb52a03-8e92-4387-a7d9-5b580e03729a]     loader['mapping']['nvram-template']['filename'],
nova/nova-compute.log:2022-05-25 20:30:46.726 2 ERROR nova.compute.manager [instance: ddb52a03-8e92-4387-a7d9-5b580e03729a] KeyError: 'nvram-template'
nova/nova-compute.log:2022-05-25 20:30:46.880 2 ERROR nova.compute.manager [req-79627bb6-1e33-4799-9952-27a5e608efbc 09699a87ba594d518aa8d3b76f297536 a2552c854ca74e80afa16e00aaaab68d - default default] [instance: ddb52a03-8e92-4387-a7d9-5b580e03729a] Failed to build and run instance: KeyError: 'nvram-template'
nova/nova-compute.log:2022-05-25 20:30:46.880 2 ERROR nova.compute.manager [instance: ddb52a03-8e92-4387-a7d9-5b580e03729a]     loader['mapping']['nvram-template']['filename'],
nova/nova-compute.log:2022-05-25 20:30:46.880 2 ERROR nova.compute.manager [instance: ddb52a03-8e92-4387-a7d9-5b580e03729a] KeyError: 'nvram-template'
nova/nova-compute.log:2022-05-25 20:30:46.881 2 DEBUG nova.compute.manager [req-79627bb6-1e33-4799-9952-27a5e608efbc 09699a87ba594d518aa8d3b76f297536 a2552c854ca74e80afa16e00aaaab68d - default default] [instance: ddb52a03-8e92-4387-a7d9-5b580e03729a] Build of instance ddb52a03-8e92-4387-a7d9-5b580e03729a was re-scheduled: 'nvram-template' _do_build_and_run_instance /usr/lib/python3.9/site-packages/nova/compute/manager.py:2254

Comment 2 Artom Lifshitz 2022-06-08 15:22:20 UTC
New OSP 17 Jira doc ticket will be open to document known issue until RHEL fix is released.

Comment 3 Irina 2022-06-09 08:38:05 UTC
Doc tracker: https://issues.redhat.com/browse/RHOSPDOC-955