Bug 2186383

Summary: Failed to migrate uefi vm with firmware XML config [rhel-9.2.0.z]
Product: Red Hat Enterprise Linux 9 Reporter: RHEL Program Management Team <pgm-rhel-tools>
Component: libvirtAssignee: Andrea Bolognani <abologna>
libvirt sub component: General QA Contact: Meina Li <meili>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: abologna, fjin, jdenemar, jsuchane, juzhou, lmen, mdeng, meili, tyan, tzheng, virt-maint, xiaohli, xuwei, xuzhang, yafu, ymankad, zhang.hongzhou
Version: 9.2Keywords: Regression, Triaged, ZStream
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-9.0.0-10.1.el9_2 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2184966 Environment:
Last Closed: 2023-05-09 11:04:43 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2184966    
Bug Blocks:    

Comment 2 Meina Li 2023-04-17 05:42:27 UTC 
Test Version:
libvirt-9.0.0-10.1.el9_2.x86_64
qemu-kvm-7.2.0-14.el9_2.x86_64

Test Steps:
S1: Migration guest with secure boot enabled.
1. Prepare a guest with the following os xml.
<os firmware="efi">
  <type arch="x86_64" machine="pc-q35-rhel9.2.0">hvm</type>
  <firmware>
    <feature enabled="yes" name="enrolled-keys"/>
    <feature enabled="yes" name="secure-boot"/>
  </firmware>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>
2. Start the guest.
# virsh start rhel
Domain 'rhel' started
# virsh dumpxml rhel --xpath //os
<os>
  <type arch="x86_64" machine="pc-q35-rhel9.2.0">hvm</type>
  <firmware>
    <feature enabled="yes" name="enrolled-keys"/>
    <feature enabled="yes" name="secure-boot"/>
  </firmware>
  <loader readonly="yes" secure="yes" type="pflash">/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader>
  <nvram template="/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd">/var/lib/libvirt/qemu/nvram/rhel_VARS.fd</nvram>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>
3. Migrate to the target host.
# virsh migrate rhel qemu+ssh://[target_host]/system --live --verbose --p2p
Migration: [100 %]
# virsh dumpxml rhel --xpath //os
<os>
  <type arch="x86_64" machine="pc-q35-rhel9.2.0">hvm</type>
  <loader readonly="yes" secure="yes" type="pflash">/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader>
  <nvram template="/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd">/var/lib/libvirt/qemu/nvram/rhel_VARS.fd</nvram>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>
4. Migrate backe to the source host.
# virsh migrate rhel qemu+ssh://[source_host]/system --live --verbose --p2p
Migration: [100 %]
# virsh dumpxml rhel --xpath //os
<os>
  <type arch="x86_64" machine="pc-q35-rhel9.2.0">hvm</type>
  <loader readonly="yes" secure="yes" type="pflash">/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader>
  <nvram template="/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd">/var/lib/libvirt/qemu/nvram/rhel_VARS.fd</nvram>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>
5. Destroy the guest and check.
# virsh destroy rhel
Domain 'rhel' destroyed
# virsh dumpxml rhel --xpath //os
<os firmware="efi">
  <type arch="x86_64" machine="pc-q35-rhel9.2.0">hvm</type>
  <firmware>
    <feature enabled="yes" name="enrolled-keys"/>
    <feature enabled="yes" name="secure-boot"/>
  </firmware>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>

S2: Migration guest with secure boot disabled.
Repeat the S1 steps but with the following os xml:
<os firmware="efi">
  <type arch="x86_64" machine="pc-q35-rhel9.2.0">hvm</type>
  <firmware>
    <feature enabled="no" name="enrolled-keys"/>
    <feature enabled="yes" name="secure-boot"/>
  </firmware>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>

S3: Cross migration
Source:
libvirt-9.0.0-10.1.el9_2.x86_64
qemu-kvm-7.2.0-14.el9_2.x86_64

Target:
libvirt-8.0.0-5.6.module+el8.6.0+17751+d6559882.x86_64
qemu-kvm-6.2.0-11.module+el8.6.0+17576+33ee06a8.7.x86_64

1. Prepare a running guest with the following os xml:
# virsh dumpxml rhel --inactive --xpath //os
<os>
  <type arch="x86_64" machine="pc-q35-rhel8.6.0">hvm</type>
  <firmware>
    <feature enabled="yes" name="enrolled-keys"/>
    <feature enabled="yes" name="secure-boot"/>
  </firmware>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>
2. Migrate to the target host.
# virsh migrate rhel qemu+ssh://[target_host]/system --live --verbose --p2p
Migration: [100 %]
# virsh dumpxml rhel | grep /os -B7
  <os>
    <type arch='x86_64' machine='pc-q35-rhel8.6.0'>hvm</type>
    <loader readonly='yes' secure='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader>
    <nvram template='/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd'>/var/lib/libvirt/qemu/nvram/rhel_VARS.fd</nvram>
    <boot dev='hd'/>
    <bootmenu enable='yes' timeout='6000'/>
    <bios useserial='yes' rebootTimeout='3000'/>
  </os>
3. Migrate back to the source host.
# virsh migrate rhel qemu+ssh://[source_host]/system --live --verbose --p2p
Migration: [100 %]
# virsh dumpxml rhel --xpath //os
<os>
  <type arch="x86_64" machine="pc-q35-rhel8.6.0">hvm</type>
  <loader readonly="yes" secure="yes" type="pflash">/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader>
  <nvram template="/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd">/var/lib/libvirt/qemu/nvram/rhel_VARS.fd</nvram>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>

S4: Save/managedsave the guest with firmware features.
1. Prepare a running guest with the following os xml:
# virsh dumpxml rhel --inactive --xpath //os
<os>
  <type arch="x86_64" machine="pc-q35-rhel9.2.0">hvm</type>
  <firmware>
    <feature enabled="yes" name="enrolled-keys"/>
    <feature enabled="yes" name="secure-boot"/>
  </firmware>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>
2. Save the guest and then restore.
# virsh save rhel rhel.save
Domain 'rhel' saved to rhel.save
# virsh list --all
 Id   Name             State
---------------------------------
 -    rhel             shut off
# virsh restore rhel.save 
Domain restored from rhel.save
# virsh dumpxml rhel --xpath //os
<os>
  <type arch="x86_64" machine="pc-q35-rhel9.2.0">hvm</type>
  <loader readonly="yes" secure="yes" type="pflash">/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader>
  <nvram template="/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd">/var/lib/libvirt/qemu/nvram/rhel_VARS.fd</nvram>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>
3. Managedsave the guest.
# virsh managedsave rhel
Domain 'rhel' state saved by libvirt
# virsh start rhel
Domain 'rhel' started
# virsh dumpxml rhel --xpath //os
<os>
  <type arch="x86_64" machine="pc-q35-rhel9.2.0">hvm</type>
  <loader readonly="yes" secure="yes" type="pflash">/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader>
  <nvram template="/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd">/var/lib/libvirt/qemu/nvram/rhel_VARS.fd</nvram>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>
# virsh dumpxml rhel --inactive --xpath //os
<os firmware="efi">
  <type arch="x86_64" machine="pc-q35-rhel8.6.0">hvm</type>
  <firmware>
    <feature enabled="yes" name="enrolled-keys"/>
    <feature enabled="yes" name="secure-boot"/>
  </firmware>
  <boot dev="hd"/>
  <bootmenu enable="yes" timeout="6000"/>
  <bios useserial="yes" rebootTimeout="3000"/>
</os>

All the test scenarios are expected.
Comment 6 Meina Li 2023-04-18 10:02:27 UTC 
Test with all the scenarios in comment 2, the result are expected.
Comment 10 errata-xmlrpc 2023-05-09 11:04:43 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (libvirt bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2632