Description of problem: Not sure what is the expected correct behavior here, but the engine and vdsm allow the user to change the vNIC profile of a hostdev (SR-IOV) while the VM and vNIC are up, whereas libvirt seems to forbid it. Because the VM is up, the change triggers a updateDevice, which libvirt rejects because the device is hostdev: 2019-01-14 11:51:53,006+0100 INFO (jsonrpc/3) [api.virt] START updateDevice(params={u'xml': u'<?xml version="1.0" encoding="UTF-8"?><hotplug><devices><interface type="hostdev" managed="no"><driver name="vfio"></driver><vlan><tag id="228"></tag></vlan><source><address bus="65" domain="0" function="0" slot="16" type="pci"></address></source><alias name="ua-eafc1987-7442-4f27-91b4-5eafeb08fb14"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x0b" type="pci"></address><mac address="<removed>"></mac><bandwidth></bandwidth></interface></devices><metadata xmlns:ovirt-vm="http://ovirt.org/vm/1.0"><ovirt-vm:vm><ovirt-vm:device mac_address="<removed>"><ovirt-vm:custom></ovirt-vm:custom></ovirt-vm:device></ovirt-vm:vm></metadata></hotplug>', u'deviceType': u'interface'}) from=<removed>,33654, flow_id=42221ae3-fd99-4661-9850-23117db8e411, vmId=61aba794-6e3f-48ed-b779-aeba6bbbc63b (api:46) 2019-01-14 11:51:53,022+0100 WARN (jsonrpc/3) [virt.vm] (vmId='61aba794-6e3f-48ed-b779-aeba6bbbc63b') Request failed: <?xml version='1.0' encoding='utf-8'?> <interface managed="no" type="hostdev"> <address bus="0x00" domain="0x0000" function="0x0" slot="0x0b" type="pci" /> <mac address="<removed>" /> <source bridge=";vdsmdummy;"> <address bus="0x41" domain="0x0000" function="0x0" slot="0x10" type="pci" /> </source> <vlan> <tag id="228" /> </vlan> <link state="up" /> <driver name="vfio" queues="" /> <alias name="ua-eafc1987-7442-4f27-91b4-5eafeb08fb14" /> </interface> (vm:3266) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3261, in setLinkAndNetwork libvirt.VIR_DOMAIN_AFFECT_LIVE) File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 98, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 130, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 92, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2772, in updateDeviceFlags if ret == -1: raise libvirtError ('virDomainUpdateDeviceFlags() failed', dom=self) libvirtError: Operation not supported: cannot change config of 'hostdev' network type Because libvirt code seems to explicitly forbid this: qemuDomainChangeNet(virQEMUDriverPtr driver, virDomainObjPtr vm, virDomainDeviceDefPtr dev) { oldType = virDomainNetGetActualType(olddev); if (oldType == VIR_DOMAIN_NET_TYPE_HOSTDEV) { /* no changes are possible to a type='hostdev' interface */ virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("cannot change config of '%s' network type"), virDomainNetTypeToString(oldType)); goto cleanup; } Version-Release number of selected component (if applicable): libvirt-4.5.0-10.el7_6.2.x86_64 vdsm-4.20.43-1.el7ev.x86_64 rhvm-4.2.7.5-0.1.el7ev.noarch How reproducible: Always Steps to Reproduce: 1. Change vNIC profile of a SR-IOV vNIC Actual results: Fails with error Expected results: blocked OR work
Germano, can you please explain what you changed and which steps are required to reproduce? Why do you use hostdevices for SRIOV?
Hi Dominik, (In reply to Dominik Holler from comment #1) > can you please explain what you changed and which steps are required to reproduce? Change the profile of a SR-IOV vNIC (Compute -> Virtual Machines -> Network Interfaces -> Edit -> Profile) Both profiles set as Passthrough. It tries to update the XML of the NIC, which is rejected by libvirt. If fact, any change what would require the XML of the vNIC to be updated should trigger the bug, as libvirt rejects it. > Why do you use hostdevices for SRIOV? This is what RHV does. I don't think there is any other way to pass a VF of a NIC to a VM, it uses hostdev to pass the VF to the VM. Maybe the first is to clarify if RHV is supposed to support live modifications of an SR-IOV NIC?
Hi Germano, It is not supported to update/modify the SR-IOV vNIC profile or vise versa while the vNIC is plugged to the VM, you first need to hotunplug the vNIC and then update to the desired profile. "Cannot edit Interface. Updating some of the properties is not supported while the interface is plugged into a running virtual machine. Please un-plug the interface, update the properties, and then plug it back."
Ok so, thanks to Dominik's insight, i was managed to understand and reproduce the bug. The bug here, is that we are not validating the update of SR-IOV vNIC profile to a different SR-IOV vNIC profile, while it is plugged and running. We only validate the update of VirtIo vNIC profile to SR-IOV vNIC profile and vise versa. Steps to reproduce: 1. Enable 2 VFs on the host 2. Create 2 logical networks, net1 and net2 3. Set net1's and net2's vNIC profiles with 'passtghrough' checkbox 4. Add 2 nics to the VM: nic1 with ovirtmgmt vNIC profile(VirtIo) and nic2 with net1 vNIC profile(pci-passtghrough) 5. Start VM 6. Try to update nic1 with net1 or net2 profiles - FAIL as expected, validation is good "Cannot edit Interface. Updating some of the properties is not supported while the interface is plugged into a running virtual machine. Please un-plug the interface, update the properties, and then plug it back." 7. Try to update nic2 with ovirtmgmt profile - FAIL as expected, validation is good "Cannot edit Interface. Updating some of the properties is not supported while the interface is plugged into a running virtual machine. Please un-plug the interface, update the properties, and then plug it back." 8. Try to update nic2 with net2 profile - bad and wrong validation and a bug - "The Network does not exist on the host the VM is running on. Either add the Network to the Host, migrate the VM to a Host that has this Network, or add the vNic as unplugged." 9. Attach net2 to the host via setup networks 10. Now, repeat step 8 and update nic2 with net2 profile - no validation, operation is not supported but allowed and Germano's report is reproduced - if ret == -1: raise libvirtError ('virDomainUpdateDeviceFlags() failed', dom=self) libvirtError: Operation not supported: cannot change config of 'hostdev' network type * Bottom line, the same validation we have for steps 6 and 7, should be when trying to update SR-IOV vNIC profile with a different SR-IOV vNIC profile while the nic is plugged. Such operations must be performed while the vNIC is unplugged only. It's an engine validation bug. Thanks Germano and Dominik
Thanks Michael! It's clear now that this is not supposed to work and the validation was missing.
Verified on - rhvm-4.3.2-0.1.el7.noarch and vdsm-4.30.10-1.el7ev.x86_64 Steps 8 and 10 in comment 4 above ^^ is no longer possible, there is new validation when trying to update a SR-IOV vNIC profile with another SR-IOV vNIC profile while the VM's nic is plugged. Operation Canceled: "Error while executing action: golden_env_mixed_virtio_1: Cannot edit Interface. Updating some of the properties is not supported while the interface is plugged into a running virtual machine. Please un-plug the interface, update the properties, and then plug it back"
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:1085
sync2jira