Description of problem: If a VM is created from a Q35 Template but is set to i440FX at the time of 'New VM', the vm_devices are left behind with several PCI elements and addresses that are not compatible. And the VM fails to start because it's XML is inconsistent: 2020-04-30 15:50:25,055+1000 ERROR (vm/49395e3c) [virt.vm] (vmId='49395e3c-1dd1-4144-9d48-87fbe64662f1') The vm start process failed (vm:934) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 868, in _startUnderlyingVm self._run() File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2892, in _run dom = self._connection.defineXML(self._domain.xml) File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3733, in defineXML if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self) libvirtError: XML error: The device at PCI address 0000:00:02.0 cannot be plugged into the PCI controller with index='0'. It requires a controller that accepts a pcie-root-port. The full XML generated by the engine is in the "Additional Info" section. NOTE: This does not seem to happen if a VM is switched from Q35 to i440FX after creation, the change needs to be during the 'New VM' when creating from Template. From a quick research, I think it could be because engine is not clearing the devices if the BIOS change is done during VM creaion, like it does on UpdateVm: https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/UpdateVmCommand.java#L522 Version-Release number of selected component (if applicable): rhvm-4.3.9.4-11.el7.noarch vdsm-4.30.44-1.el7ev.x86_64 How reproducible: Always Steps to Reproduce: 1. Create a dummy VM with Q35 chipset (no OS install needed) 2. Create Template from VM 3. Create VM2 from Template, while setting BIOS Type to i440FX Seabios (Default) in the New VM dialog. 4. Run VM2 Actual results: VM fails to start Expected results: VM starts with proper i440FX definitions and devices Additional info: 2020-04-30 15:50:24,908+10 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-95) [18b31f59-ba59-44d1-a122-894e7dbe30a6] VM <?xml version="1.0" encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"> <name>TEST_VM</name> <uuid>49395e3c-1dd1-4144-9d48-87fbe64662f1</uuid> <memory>1048576</memory> <currentMemory>1048576</currentMemory> <iothreads>1</iothreads> <maxMemory slots="16">4194304</maxMemory> <vcpu current="1">16</vcpu> <sysinfo type="smbios"> <system> <entry name="manufacturer">Red Hat</entry> <entry name="product">OS-NAME:</entry> <entry name="version">OS-VERSION:</entry> <entry name="serial">HOST-SERIAL:</entry> <entry name="uuid">49395e3c-1dd1-4144-9d48-87fbe64662f1</entry> </system> </sysinfo> <clock offset="variable" adjustment="0"> <timer name="rtc" tickpolicy="catchup"/> <timer name="pit" tickpolicy="delay"/> <timer name="hpet" present="no"/> </clock> <features> <acpi/> </features> <cpu match="exact"> <model>Skylake-Client</model> <feature name="spec-ctrl" policy="require"/> <feature name="ssbd" policy="require"/> <feature name="md-clear" policy="require"/> <topology cores="1" threads="1" sockets="16"/> <numa> <cell id="0" cpus="0" memory="1048576"/> </numa> </cpu> <cputune/> <devices> <input type="tablet" bus="usb"/> <channel type="unix"> <target type="virtio" name="ovirt-guest-agent.0"/> <source mode="bind" path="/var/lib/libvirt/qemu/channels/49395e3c-1dd1-4144-9d48-87fbe64662f1.ovirt-guest-agent.0"/> </channel> <channel type="unix"> <target type="virtio" name="org.qemu.guest_agent.0"/> <source mode="bind" path="/var/lib/libvirt/qemu/channels/49395e3c-1dd1-4144-9d48-87fbe64662f1.org.qemu.guest_agent.0"/> </channel> <controller type="virtio-serial" index="0" ports="16"> <alias name="ua-034639a5-b812-43c1-a3d5-732ec631b8ee"/> <address bus="0x03" domain="0x0000" function="0x0" slot="0x00" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="8"> <address bus="0x00" domain="0x0000" function="0x7" slot="0x02" type="pci"/> </controller> <video> <model type="qxl" vram="8192" heads="1" ram="65536" vgamem="16384"/> <alias name="ua-082d5003-30d6-4abf-b009-7d57298d1ddf"/> </video> <controller type="pci" model="pcie-root-port" index="16"> <address bus="0x00" domain="0x0000" function="0x7" slot="0x03" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="14"> <address bus="0x00" domain="0x0000" function="0x5" slot="0x03" type="pci"/> </controller> <graphics type="vnc" port="-1" autoport="yes" passwd="*****" passwdValidTo="1970-01-01T00:00:01" keymap="en-us"> <listen type="network" network="vdsm-ovirtmgmt"/> </graphics> <rng model="virtio"> <backend model="random">/dev/urandom</backend> <alias name="ua-1c052397-a709-4362-a4c0-e085b23631e8"/> </rng> <controller type="pci" model="pcie-root-port" index="2"> <address bus="0x00" domain="0x0000" function="0x1" slot="0x02" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="3"> <address bus="0x00" domain="0x0000" function="0x2" slot="0x02" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="12"> <address bus="0x00" domain="0x0000" function="0x3" slot="0x03" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="15"> <address bus="0x00" domain="0x0000" function="0x6" slot="0x03" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="13"> <address bus="0x00" domain="0x0000" function="0x4" slot="0x03" type="pci"/> </controller> <graphics type="spice" port="-1" autoport="yes" passwd="*****" passwdValidTo="1970-01-01T00:00:01" tlsPort="-1"> <channel name="main" mode="secure"/> <channel name="inputs" mode="secure"/> <channel name="cursor" mode="secure"/> <channel name="playback" mode="secure"/> <channel name="record" mode="secure"/> <channel name="display" mode="secure"/> <channel name="smartcard" mode="secure"/> <channel name="usbredir" mode="secure"/> <listen type="network" network="vdsm-ovirtmgmt"/> </graphics> <controller type="pci" model="pcie-root-port" index="5"> <address bus="0x00" domain="0x0000" function="0x4" slot="0x02" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="7"> <address bus="0x00" domain="0x0000" function="0x6" slot="0x02" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="6"> <address bus="0x00" domain="0x0000" function="0x5" slot="0x02" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="9"> <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci" multifunction="on"/> </controller> <memballoon model="virtio"> <stats period="5"/> <alias name="ua-9f8436d3-7291-4fbf-b5e0-3f85a8487e36"/> <address bus="0x04" domain="0x0000" function="0x0" slot="0x00" type="pci"/> </memballoon> <controller type="scsi" model="virtio-scsi" index="0"> <driver iothread="1"/> <alias name="ua-ac24a3c9-6ce0-49e0-b811-4583ec285619"/> </controller> <controller type="pci" model="pcie-root-port" index="1"> <address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci" multifunction="on"/> </controller> <controller type="sata"> <address bus="0x00" domain="0x0000" function="0x2" slot="0x1f" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="11"> <address bus="0x00" domain="0x0000" function="0x2" slot="0x03" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="4"> <address bus="0x00" domain="0x0000" function="0x3" slot="0x02" type="pci"/> </controller> <controller type="pci" model="pcie-root-port" index="10"> <address bus="0x00" domain="0x0000" function="0x1" slot="0x03" type="pci"/> </controller> <controller type="usb" model="piix3-uhci" index="0"/> <channel type="spicevmc"> <target type="virtio" name="com.redhat.spice.0"/> </channel> <disk type="file" device="cdrom" snapshot="no"> <driver name="qemu" type="raw" error_policy="report"/> <source file="" startupPolicy="optional"> <seclabel model="dac" type="none" relabel="no"/> </source> <target dev="hdc" bus="ide"/> <readonly/> <alias name="ua-ba85d4d1-8143-469f-bb98-61dc020f7c25"/> <address bus="0" controller="0" unit="2" type="drive" target="0"/> </disk> <disk snapshot="no" type="file" device="disk"> <target dev="sda" bus="scsi"/> <source file="/rhev/data-center/61facf2e-82aa-11ea-964e-52540019c104/0c384a7c-ae10-470e-82ae-a1f10728c156/images/5e398581-aa2c-4919-ac9b-bfd14aee5441/3f555307-e7bd-47f3-b9ca-ee1bbdfe88c6"> <seclabel model="dac" type="none" relabel="no"/> </source> <driver name="qemu" io="threads" type="qcow2" error_policy="stop" cache="none"/> <alias name="ua-5e398581-aa2c-4919-ac9b-bfd14aee5441"/> <address bus="0" controller="0" unit="0" type="drive" target="0"/> <boot order="1"/> <serial>5e398581-aa2c-4919-ac9b-bfd14aee5441</serial> </disk> </devices> <pm> <suspend-to-disk enabled="no"/> <suspend-to-mem enabled="no"/> </pm> <os> <type arch="x86_64" machine="pc-i440fx-rhel7.6.0">hvm</type> <smbios mode="sysinfo"/> </os> <metadata> <ovirt-tune:qos/> <ovirt-vm:vm> <ovirt-vm:minGuaranteedMemoryMb type="int">1024</ovirt-vm:minGuaranteedMemoryMb> <ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion> <ovirt-vm:custom/> <ovirt-vm:device devtype="disk" name="sda"> <ovirt-vm:poolID>61facf2e-82aa-11ea-964e-52540019c104</ovirt-vm:poolID> <ovirt-vm:volumeID>3f555307-e7bd-47f3-b9ca-ee1bbdfe88c6</ovirt-vm:volumeID> <ovirt-vm:imageID>5e398581-aa2c-4919-ac9b-bfd14aee5441</ovirt-vm:imageID> <ovirt-vm:domainID>0c384a7c-ae10-470e-82ae-a1f10728c156</ovirt-vm:domainID> </ovirt-vm:device> <ovirt-vm:launchPaused>false</ovirt-vm:launchPaused> <ovirt-vm:resumeBehavior>auto_resume</ovirt-vm:resumeBehavior> </ovirt-vm:vm> </metadata> </domain>
Q35 is a technical preview in 4.3 Even in 4.4, creating a VM from a Q35 template and changing the biostype won't be supportable. However, Nisim, can you check if this is reproducible on 4.4?
(In reply to Ryan Barry from comment #1) > Q35 is a technical preview in 4.3 Yup, customer is aware too :) But it would be nice to have it working. And the workaround is to edit the VM, change to Q35 and then back i440FX. With that it clears the devices and gets new addresses once the VM starts. I think we just need to clear the devices when creating from template and the target chipset is different from the source chipset, like we do on UpdateVm(). > Even in 4.4, creating a VM from a Q35 template and changing the biostype won't be supportable. However, Nisim, can you check if this is reproducible on 4.4? So we need to document this...
(In reply to Ryan Barry from comment #1) > Q35 is a technical preview in 4.3 > > Even in 4.4, creating a VM from a Q35 template and changing the biostype > won't be supportable. However, Nisim, can you check if this is reproducible > on 4.4? Is it reproducible? Yes and no. - The relation between BIOS type and Custom emulated machine should be blocked by the UI if incorrect options are selected (for example: 'pc-i440fx' & 'Q35 chipset with legacy BIOS'). - Following https://bugzilla.redhat.com/show_bug.cgi?id=1829691#c0 steps to reproduce: 1. Issue is reproducible if selecting custom emulated machine pc-i440fx and keeping BIOS type with "Q35 chipset with legacy BIOS" after VM creation from template (see vdsm1.log.xz, 2020-05-03 09:38:19,370+0300 ERROR). 2. Issue is not reproducible if selecting custom emulated machine pc-i440fx and changing BIOS type to legacy after VM creation from template (see vdsm2.log.xz, 2020-05-03 09:34:19,315+0300).
Created attachment 1684375 [details] vdsm1.log.xz, see 2020-05-03 09:38:19,370+0300 ERROR
Created attachment 1684376 [details] vdsm2.log.xz, see 2020-05-03 09:34:19,315+0300
Actually it still doesn't work - we change the devices but for some reason don't clear the device addresses in this flow.
Verified: ovirt-engine-4.4.3.7-0.22.el8ev vdsm-4.40.34-1.el8ev.x86_64 libvirt-daemon-6.6.0-6.module+el8.3.0+8125+aefcf088.x86_64 qemu-kvm-5.1.0-13.module+el8.3.0+8382+afc3bbea.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Low: Red Hat Virtualization security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5179
Due to QE capacity we are not going to cover this issue in our automation