Description of problem: It would be nice if the variable store will be persistent not just for the approach of using the OVMF setup TUI in order to enroll key from file, but for "mokutil --import" and nvidia_installer signed keys approach as well. Currently, oVirt always re-creates the varstore from OVMF_VARS.secboot.fd varstore template, and oVirt would not preserve the varstore for the domain between boot-ups. Relevant use cases: This issue affects RHEL 7.5 VMs and NVIDIA drivers installed on VM with VFIO or mdev: When importing files using mokutil (relevant for RHEL7.5 workaround, see https://bugzilla.redhat.com/show_bug.cgi?id=1649270) or Signing NVIDIA Kernel module using imported keys or generated keys by NVIDIA installer and afterward powering off an running the VM again, these keys will not be part of system keyring anymore. Version-Release number of selected component (if applicable): ovirt-engine-4.3.0-0.8.master.20190122102235.git7a1ef10.el7 OVMF-20180508-3.gitee3198e672e2.el7.noarch vdsm-4.30.6-19.git754a02d.el7.x86_64 qemu-kvm-ev-2.12.0-18.el7_6.1.1.x86_64 libvirt-client-4.5.0-10.el7_6.3.x86_64
An alternative could be to write a systemd service that loads custom pubkeys (certificates) from regular files with keyctl into the system keyring. This should happen early enough, before the nvidia proprietary modules are loaded. The service and its dependencies (certificate files) might have to be built into the initrd as well (because I assume the nvidia modules end up in the initrd too). Thanks.
The following procedure allows to install NVidia driver with SecureBoot: 1. Start the VM. 2. Download and install NVidia driver. During the installation, ask the installer to generate the signing keys and to sign the module. 3. Import the public key to the system keyring: mokutil --import /usr/share/nvidia/nvidia-modsign-crt-*.der 4. Reboot the VM from the console (do not poweroff). 5. Finish the key enrolment process from the menu, that will be displayed when booting. 6. Continue booting and ensure that the driver loads normally. 7. While the VM is still running, copy its NVRAM image, located at /var/lib/libvirt/qemu/nvram/<VM_ID>.fd (VM ID is displayed in the VM details in WebAdmin). 8. Copy the image to /usr/share/OVMF/OVMF_VARS.secboot.fd (you may want to backup the original file) on all hosts where the VM or its copies may be running. 9. If you want to install the driver in other VMs, ask the installer on step 2 to preserve the private key used for signing. Then pass the same keys to the installer in other VMs using --module-signing-secret-key= and --module-signing-public-key= options. In this case you will be able to use the same NVRAM image everywhere.
(In reply to Shmuel Melamud from comment #2) > 7. While the VM is still running, copy its NVRAM image, located at > /var/lib/libvirt/qemu/nvram/<VM_ID>.fd (VM ID is displayed in the VM > details in WebAdmin). I don't recommend this. Same as we don't snapshot a read-write mounted, not fs-frozen disk image without also snapshotting guest RAM, processor, and device state. Capturing a varstore file for use as a template is a valid technique (we use it in the build script of the edk2 package), but the originating domain must be powered down first. > 8. Copy the image to /usr/share/OVMF/OVMF_VARS.secboot.fd (you may > want to backup the original file) on all hosts where the VM or its > copies may be running. I don't recommend this either. It is possible to offer new variable store template files without corrupting existent package contents. And the new varstore template can be exposed to libvirtd and other management applications through additional firmware metadata descriptor files. (See examples in "/usr/share/qemu/firmware/".) The new files (varstore template and matching descriptor) can be collected into a new RPM (to be installed on RHV hosts). The new descriptor may be assigned such a double-digit priority prefix that causes it to take priority over the descriptor files from the edk2-ovmf package. This will instantiate the varstore files of newly defined UEFI domains from the extra varstore template that carries the nvidia module signing certificate. Additionally, when defining a new domain, the @template attribute in the nvram element can still be pointed, like before, at a specific varstore template file. Then libvirtd will create the domain's own varstore from that specific template. The above steps are acceptable as a proof of concept, but please don't use them in production. Overwriting installed package contents (except config files) makes debugging very difficult. Thanks.
(In reply to Laszlo Ersek from comment #3) > Capturing a varstore file for use as a template is a valid technique (we > use it in the build script of the edk2 package), but the originating > domain must be powered down first. It is not possible, because the varstore file is removed immediately after VM shutdown.
(In reply to Shmuel Melamud from comment #4) > (In reply to Laszlo Ersek from comment #3) > > Capturing a varstore file for use as a template is a valid technique (we > > use it in the build script of the edk2 package), but the originating > > domain must be powered down first. > > It is not possible, because the varstore file is removed immediately after > VM shutdown. Libvirt will execute a hook script if you have it on domain shutdown - that might be a good place to copy the NVRAM store to a safe location. https://www.libvirt.org/hooks.html
Would be great to have a unified mechanism to report both NVRAM store and TPM data [1] from the host to the engine [1] https://github.com/oVirt/ovirt-site/pull/2298
*** Bug 1727987 has been marked as a duplicate of this bug. ***
(In reply to Michal Privoznik from comment #5) > Libvirt will execute a hook script if you have it on domain shutdown - that > might be a good place to copy the NVRAM store to a safe location. > > https://www.libvirt.org/hooks.html thanks Michal, is the NVRAM store cleared on domain shutdown or when the domain is undefined?
(In reply to Arik from comment #8) > thanks Michal, is the NVRAM store cleared on domain shutdown or when the > domain is undefined? The hook script is ran on domain shutdown, and removal of the NVRAM file is done on domain undefine.
The Vdsm part is done. I'm going to implement the Engine part now, so taking over the bug.
The changes were done in 4.4.5 already, will be verified on 4.4.6
Verified: ovirt-engine-4.4.6.6-0.10.el8ev vdsm-4.40.60.6-1.el8ev.x86_64 qemu-kvm-5.2.0-15.module+el8.4.0+10650+50781ca0.x86_64 libvirt-daemon-7.0.0-13.module+el8.4.0+10604+5608c2b4.x86_64 host Nvidia drivers: NVIDIA-vGPU-rhel-8.4-460.73.02.x86_64 VM Nvidia drivers(for Windows and Linux): GRID 12.0 GA Verification scenario: Polarion test case added to "Links"
This bugzilla is included in oVirt 4.4.6 release, published on May 4th 2021. Since the problem described in this bug report should be resolved in oVirt 4.4.6 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.