Bug 1829691 - Engine adding PCI-E elements on XML of i440FX SeaBIOS VM created from Q35 Template
Summary: Engine adding PCI-E elements on XML of i440FX SeaBIOS VM created from Q35 Tem...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.3.9
Hardware: x86_64
OS: Linux
low
low
Target Milestone: ovirt-4.4.3
: 4.4.3
Assignee: Shmuel Melamud
QA Contact: Nisim Simsolo
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-30 06:10 UTC by Germano Veit Michel
Modified: 2023-09-07 23:01 UTC (History)
4 users (show)

Fixed In Version: ovirt-engine-4.4.3.8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-11-24 13:09:21 UTC
oVirt Team: Virt
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
vdsm1.log.xz, see 2020-05-03 09:38:19,370+0300 ERROR (197.51 KB, application/x-xz)
2020-05-03 07:01 UTC, Nisim Simsolo
no flags Details
vdsm2.log.xz, see 2020-05-03 09:34:19,315+0300 (505.55 KB, application/x-xz)
2020-05-03 07:02 UTC, Nisim Simsolo
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 5036751 0 None None None 2020-04-30 06:29:20 UTC
Red Hat Product Errata RHSA-2020:5179 0 None None None 2020-11-24 13:09:46 UTC
oVirt gerrit 95157 0 None MERGED engine: Ability to set biosType for a cluster 2021-02-01 19:05:07 UTC
oVirt gerrit 111690 0 master MERGED core: Clear device addresses if VM's chipset differs from template's 2021-02-01 19:04:22 UTC

Description Germano Veit Michel 2020-04-30 06:10:16 UTC
Description of problem:

If a VM is created from a Q35 Template but is set to i440FX at the time of 'New VM', the vm_devices are left behind with several PCI elements and addresses that are not compatible.
And the VM fails to start because it's XML is inconsistent:

2020-04-30 15:50:25,055+1000 ERROR (vm/49395e3c) [virt.vm] (vmId='49395e3c-1dd1-4144-9d48-87fbe64662f1') The vm start process failed (vm:934)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 868, in _startUnderlyingVm
    self._run()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2892, in _run
    dom = self._connection.defineXML(self._domain.xml)
  File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3733, in defineXML
    if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self)
libvirtError: XML error: The device at PCI address 0000:00:02.0 cannot be plugged into the PCI controller with index='0'. It requires a controller that accepts a pcie-root-port.

The full XML generated by the engine is in the "Additional Info" section.

NOTE: This does not seem to happen if a VM is switched from Q35 to i440FX after creation, the change needs to be during the 'New VM' when creating from Template.
From a quick research, I think it could be because engine is not clearing the devices if the BIOS change is done during VM creaion, like it does on UpdateVm:
https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/UpdateVmCommand.java#L522

Version-Release number of selected component (if applicable):
rhvm-4.3.9.4-11.el7.noarch
vdsm-4.30.44-1.el7ev.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create a dummy VM with Q35 chipset (no OS install needed)
2. Create Template from VM
3. Create VM2 from Template, while setting BIOS Type to i440FX Seabios (Default) in the New VM dialog.
4. Run VM2

Actual results:
VM fails to start

Expected results:
VM starts with proper i440FX definitions and devices

Additional info:

2020-04-30 15:50:24,908+10 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-95) [18b31f59-ba59-44d1-a122-894e7dbe30a6] VM <?xml version="1.0" encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0">
  <name>TEST_VM</name>
  <uuid>49395e3c-1dd1-4144-9d48-87fbe64662f1</uuid>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <iothreads>1</iothreads>
  <maxMemory slots="16">4194304</maxMemory>
  <vcpu current="1">16</vcpu>
  <sysinfo type="smbios">
    <system>
      <entry name="manufacturer">Red Hat</entry>
      <entry name="product">OS-NAME:</entry>
      <entry name="version">OS-VERSION:</entry>
      <entry name="serial">HOST-SERIAL:</entry>
      <entry name="uuid">49395e3c-1dd1-4144-9d48-87fbe64662f1</entry>
    </system>
  </sysinfo>
  <clock offset="variable" adjustment="0">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <features>
    <acpi/>
  </features>
  <cpu match="exact">
    <model>Skylake-Client</model>
    <feature name="spec-ctrl" policy="require"/>
    <feature name="ssbd" policy="require"/>
    <feature name="md-clear" policy="require"/>
    <topology cores="1" threads="1" sockets="16"/>
    <numa>
      <cell id="0" cpus="0" memory="1048576"/>
    </numa>
  </cpu>
  <cputune/>
  <devices>
    <input type="tablet" bus="usb"/>
    <channel type="unix">
      <target type="virtio" name="ovirt-guest-agent.0"/>
      <source mode="bind" path="/var/lib/libvirt/qemu/channels/49395e3c-1dd1-4144-9d48-87fbe64662f1.ovirt-guest-agent.0"/>
    </channel>
    <channel type="unix">
      <target type="virtio" name="org.qemu.guest_agent.0"/>
      <source mode="bind" path="/var/lib/libvirt/qemu/channels/49395e3c-1dd1-4144-9d48-87fbe64662f1.org.qemu.guest_agent.0"/>
    </channel>
    <controller type="virtio-serial" index="0" ports="16">
      <alias name="ua-034639a5-b812-43c1-a3d5-732ec631b8ee"/>
      <address bus="0x03" domain="0x0000" function="0x0" slot="0x00" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="8">
      <address bus="0x00" domain="0x0000" function="0x7" slot="0x02" type="pci"/>
    </controller>
    <video>
      <model type="qxl" vram="8192" heads="1" ram="65536" vgamem="16384"/>
      <alias name="ua-082d5003-30d6-4abf-b009-7d57298d1ddf"/>
    </video>
    <controller type="pci" model="pcie-root-port" index="16">
      <address bus="0x00" domain="0x0000" function="0x7" slot="0x03" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="14">
      <address bus="0x00" domain="0x0000" function="0x5" slot="0x03" type="pci"/>
    </controller>
    <graphics type="vnc" port="-1" autoport="yes" passwd="*****" passwdValidTo="1970-01-01T00:00:01" keymap="en-us">
      <listen type="network" network="vdsm-ovirtmgmt"/>
    </graphics>
    <rng model="virtio">
      <backend model="random">/dev/urandom</backend>
      <alias name="ua-1c052397-a709-4362-a4c0-e085b23631e8"/>
    </rng>
    <controller type="pci" model="pcie-root-port" index="2">
      <address bus="0x00" domain="0x0000" function="0x1" slot="0x02" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="3">
      <address bus="0x00" domain="0x0000" function="0x2" slot="0x02" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="12">
      <address bus="0x00" domain="0x0000" function="0x3" slot="0x03" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="15">
      <address bus="0x00" domain="0x0000" function="0x6" slot="0x03" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="13">
      <address bus="0x00" domain="0x0000" function="0x4" slot="0x03" type="pci"/>
    </controller>
    <graphics type="spice" port="-1" autoport="yes" passwd="*****" passwdValidTo="1970-01-01T00:00:01" tlsPort="-1">
      <channel name="main" mode="secure"/>
      <channel name="inputs" mode="secure"/>
      <channel name="cursor" mode="secure"/>
      <channel name="playback" mode="secure"/>
      <channel name="record" mode="secure"/>
      <channel name="display" mode="secure"/>
      <channel name="smartcard" mode="secure"/>
      <channel name="usbredir" mode="secure"/>
      <listen type="network" network="vdsm-ovirtmgmt"/>
    </graphics>
    <controller type="pci" model="pcie-root-port" index="5">
      <address bus="0x00" domain="0x0000" function="0x4" slot="0x02" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="7">
      <address bus="0x00" domain="0x0000" function="0x6" slot="0x02" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="6">
      <address bus="0x00" domain="0x0000" function="0x5" slot="0x02" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="9">
      <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci" multifunction="on"/>
    </controller>
    <memballoon model="virtio">
      <stats period="5"/>
      <alias name="ua-9f8436d3-7291-4fbf-b5e0-3f85a8487e36"/>
      <address bus="0x04" domain="0x0000" function="0x0" slot="0x00" type="pci"/>
    </memballoon>
    <controller type="scsi" model="virtio-scsi" index="0">
      <driver iothread="1"/>
      <alias name="ua-ac24a3c9-6ce0-49e0-b811-4583ec285619"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="1">
      <address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci" multifunction="on"/>
    </controller>
    <controller type="sata">
      <address bus="0x00" domain="0x0000" function="0x2" slot="0x1f" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="11">
      <address bus="0x00" domain="0x0000" function="0x2" slot="0x03" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="4">
      <address bus="0x00" domain="0x0000" function="0x3" slot="0x02" type="pci"/>
    </controller>
    <controller type="pci" model="pcie-root-port" index="10">
      <address bus="0x00" domain="0x0000" function="0x1" slot="0x03" type="pci"/>
    </controller>
    <controller type="usb" model="piix3-uhci" index="0"/>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
    </channel>
    <disk type="file" device="cdrom" snapshot="no">
      <driver name="qemu" type="raw" error_policy="report"/>
      <source file="" startupPolicy="optional">
        <seclabel model="dac" type="none" relabel="no"/>
      </source>
      <target dev="hdc" bus="ide"/>
      <readonly/>
      <alias name="ua-ba85d4d1-8143-469f-bb98-61dc020f7c25"/>
      <address bus="0" controller="0" unit="2" type="drive" target="0"/>
    </disk>
    <disk snapshot="no" type="file" device="disk">
      <target dev="sda" bus="scsi"/>
      <source file="/rhev/data-center/61facf2e-82aa-11ea-964e-52540019c104/0c384a7c-ae10-470e-82ae-a1f10728c156/images/5e398581-aa2c-4919-ac9b-bfd14aee5441/3f555307-e7bd-47f3-b9ca-ee1bbdfe88c6">
        <seclabel model="dac" type="none" relabel="no"/>
      </source>
      <driver name="qemu" io="threads" type="qcow2" error_policy="stop" cache="none"/>
      <alias name="ua-5e398581-aa2c-4919-ac9b-bfd14aee5441"/>
      <address bus="0" controller="0" unit="0" type="drive" target="0"/>
      <boot order="1"/>
      <serial>5e398581-aa2c-4919-ac9b-bfd14aee5441</serial>
    </disk>
  </devices>
  <pm>
    <suspend-to-disk enabled="no"/>
    <suspend-to-mem enabled="no"/>
  </pm>
  <os>
    <type arch="x86_64" machine="pc-i440fx-rhel7.6.0">hvm</type>
    <smbios mode="sysinfo"/>
  </os>
  <metadata>
    <ovirt-tune:qos/>
    <ovirt-vm:vm>
      <ovirt-vm:minGuaranteedMemoryMb type="int">1024</ovirt-vm:minGuaranteedMemoryMb>
      <ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion>
      <ovirt-vm:custom/>
      <ovirt-vm:device devtype="disk" name="sda">
        <ovirt-vm:poolID>61facf2e-82aa-11ea-964e-52540019c104</ovirt-vm:poolID>
        <ovirt-vm:volumeID>3f555307-e7bd-47f3-b9ca-ee1bbdfe88c6</ovirt-vm:volumeID>
        <ovirt-vm:imageID>5e398581-aa2c-4919-ac9b-bfd14aee5441</ovirt-vm:imageID>
        <ovirt-vm:domainID>0c384a7c-ae10-470e-82ae-a1f10728c156</ovirt-vm:domainID>
      </ovirt-vm:device>
      <ovirt-vm:launchPaused>false</ovirt-vm:launchPaused>
      <ovirt-vm:resumeBehavior>auto_resume</ovirt-vm:resumeBehavior>
    </ovirt-vm:vm>
  </metadata>
</domain>

Comment 1 Ryan Barry 2020-05-01 00:15:56 UTC
Q35 is a technical preview in 4.3

Even in 4.4, creating a VM from a Q35 template and changing the biostype won't be supportable. However, Nisim, can you check if this is reproducible on 4.4?

Comment 2 Germano Veit Michel 2020-05-01 00:41:35 UTC
(In reply to Ryan Barry from comment #1)
> Q35 is a technical preview in 4.3

Yup, customer is aware too :)

But it would be nice to have it working.

And the workaround is to edit the VM, change to Q35 and then back i440FX.
With that it clears the devices and gets new addresses once the VM starts.

I think we just need to clear the devices when creating from template and the target chipset is different from the source chipset, like we do on UpdateVm().

> Even in 4.4, creating a VM from a Q35 template and changing the biostype won't be supportable. However, Nisim, can you check if this is reproducible on 4.4?
So we need to document this...

Comment 3 Nisim Simsolo 2020-05-03 07:00:23 UTC
(In reply to Ryan Barry from comment #1)
> Q35 is a technical preview in 4.3
> 
> Even in 4.4, creating a VM from a Q35 template and changing the biostype
> won't be supportable. However, Nisim, can you check if this is reproducible
> on 4.4?

Is it reproducible? Yes and no.
- The relation between BIOS type and Custom emulated machine should be blocked by the UI if incorrect options are selected (for example: 'pc-i440fx' & 'Q35 chipset with legacy BIOS').
- Following https://bugzilla.redhat.com/show_bug.cgi?id=1829691#c0 steps to reproduce:
1. Issue is reproducible if selecting custom emulated machine pc-i440fx and keeping BIOS type with "Q35 chipset with legacy BIOS" after VM creation from template (see vdsm1.log.xz, 2020-05-03 09:38:19,370+0300 ERROR).
2. Issue is not reproducible if selecting custom emulated machine pc-i440fx and changing BIOS type to legacy after VM creation from template (see vdsm2.log.xz, 2020-05-03 09:34:19,315+0300).

Comment 4 Nisim Simsolo 2020-05-03 07:01:57 UTC
Created attachment 1684375 [details]
vdsm1.log.xz, see 2020-05-03 09:38:19,370+0300 ERROR

Comment 5 Nisim Simsolo 2020-05-03 07:02:33 UTC
Created attachment 1684376 [details]
vdsm2.log.xz, see 2020-05-03 09:34:19,315+0300

Comment 6 Arik 2020-08-26 08:26:12 UTC
Actually it still doesn't work - we change the devices but for some reason don't clear the device addresses in this flow.

Comment 9 Nisim Simsolo 2020-10-27 15:24:37 UTC
Verified:
ovirt-engine-4.4.3.7-0.22.el8ev
vdsm-4.40.34-1.el8ev.x86_64
libvirt-daemon-6.6.0-6.module+el8.3.0+8125+aefcf088.x86_64
qemu-kvm-5.1.0-13.module+el8.3.0+8382+afc3bbea.x86_64

Comment 13 errata-xmlrpc 2020-11-24 13:09:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: Red Hat Virtualization security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5179

Comment 14 meital avital 2022-05-11 08:49:28 UTC
Due to QE capacity we are not going to cover this issue in our automation


Note You need to log in before you can comment on or make changes to this bug.