Description of problem: Hostdev interface can not be plugged back after unplugged with failover setting Version-Release number of selected component (if applicable): libvirt-8.0.0-3.el9.x86_64 qemu-kvm-6.2.0-7.el9.x86_64 How reproducible: 100% Steps to Reproduce: 1. Start vm with failover interface setting: # virsh dumpxml rhel9 | grep /interface -B9 ...... <interface type='network'> <mac address='52:54:00:aa:1c:ef'/> <source network='hostbridge'/> <model type='virtio'/> <teaming type='persistent'/> <alias name='ua-backup0'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <interface type='network'> <mac address='52:54:00:aa:1c:ef'/> <source network='hostdev-net'/> <teaming type='transient' persistent='ua-backup0'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </interface> 2. Login the vm and check there are 3 interfaces and the network works well; 3. Hot unplug the hostdev device, all things are expected, unplug succeed and the network works well; Dump the hostdev interface xml as below: # cat hostdevinterface.xml <interface type='hostdev' managed='yes'> <mac address='52:54:00:aa:1c:ef'/> <driver name='vfio'/> <source> <address type='pci' domain='0x0000' bus='0x82' slot='0x10' function='0x1'/> </source> <teaming type='transient' persistent='ua-backup0'/> <alias name='hostdev0'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </interface> # virsh detach-device rhel9 hostdevinterface.xml Device detached successfully on guest, check network works well: # ping www.baidu.com ... 64 bytes from 182.61.200.7 (182.61.200.7): icmp_seq=151 ttl=48 time=2.97 ms [ 206.393009] pcieport 0000:00:02.3: pciehp: Slot(0-3): Attention button pressed [ 206.394346] pcieport 0000:00:02.3: pciehp: Slot(0-3): Powering off due to button press 64 bytes from 182.61.200.7 (182.61.200.7): icmp_seq=152 ttl=48 time=3.04 ms ... 64 bytes from 182.61.200.7 (182.61.200.7): icmp_seq=156 ttl=48 time=30.4 ms [ 211.519313] virtio_net virtio1 enp1s0: failover primary slave:enp4s0 unregistered 64 bytes from 182.61.200.7 (182.61.200.7): icmp_seq=158 ttl=48 time=3.14 ms ... 64 bytes from 182.61.200.7 (182.61.200.7): icmp_seq=221 ttl=48 time=3.03 ms 64 bytes from 182.61.200.7 (182.61.200.7): icmp_seq=222 ttl=48 time=3.05 ms 4. Tried to hot plug the hostdev interface back, but it failed; # virsh attach-device rhel9 hostdevinterface.xml error: Failed to attach device from hostdevinterface.xml error: internal error: unable to execute QEMU command 'device_add': Duplicate ID 'hostdev0' for device Update the xml to delete the alias, it still fails: # cat hostdevinterface2.xml <interface type='hostdev' managed='yes'> <mac address='52:54:00:aa:1c:ef'/> <driver name='vfio'/> <source> <address type='pci' domain='0x0000' bus='0x82' slot='0x10' function='0x1'/> </source> <teaming type='transient' persistent='ua-backup0'/> </interface> # virsh attach-device rhel9 hostdevinterface.xml error: Failed to attach device from hostdevinterface.xml error: internal error: unable to execute QEMU command 'device_add': Duplicate ID 'hostdev0' for device Actual results: Hostdev interface can not be plugged back after unplugged with failover setting Expected results: It should succeed to hot plug back the interface Additional info: No such issue with qemu-kvm-6.1.0-8.el9.x86_64
qemu-kvm also can reproduce the issue. qemu-kvm-6.2.0-4.el9.x86_64 qemu cmd line: -device virtio-net-pci,mac=36:d8:d5:83:4c:0b,id=idHE7r7C,netdev=idXVstXA,bus=pcie-root-port-3,addr=0x0,failover=on \ -netdev tap,id=idXVstXA,vhost=on,script=/etc/qemu-ifup-private \ -device pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,bus=pcie.0,chassis=5 \ -device vfio-pci,host=0000:06:0a.0,id=idXVst,bus=pcie-root-port-4,failover_pair_id=idHE7r7C \ {"execute": "device_del","arguments":{"id":"idXVst"}} {"return": {}} {"timestamp": {"seconds": 1644474150, "microseconds": 399755}, "event": "DEVICE_DELETED", "data": {"device": "idXVst", "path": "/machine/peripheral/idXVst"}} {"execute":"device_add","arguments":{"driver":"vfio-pci","host":"0000:06:0a.0","id":"idXVst","failover_pair_id":"idHE7r7C","bus":"pcie-root-port-4"}} {"error": {"class": "GenericError", "desc": "Duplicate ID 'idXVst' for device"}}
If hotplugging vf with a different id after unplugging vf, it will print following error: {"execute": "device_del","arguments":{"id":"idXVst"}} {"return": {}} {"timestamp": {"seconds": 1644476217, "microseconds": 527371}, "event": "DEVICE_DELETED", "data": {"device": "idXVst", "path": "/machine/peripheral/idXVst"}} {"execute":"device_add","arguments":{"driver":"vfio-pci","host":"0000:06:0a.0","id":"hostnet0","failover_pair_id":"idHE7r7C","bus":"pcie-root-port-4"}} {"error": {"class": "GenericError", "desc": "Cannot attach more than one primary device to 'idHE7r7C': 'idXVst' and 'hostnet0'"}}
Also reproduces for latest rhel8.7: libvirt-8.0.0-10.module+el8.7.0+16047+746a126c.x86_64 qemu-kvm-6.2.0-18.module+el8.7.0+15999+d24f860e.x86_64 # virsh detach-device avocado-vt-vm1 hostdev.xml Device detached successfully # virsh attach-device avocado-vt-vm1 hostdev.xml error: Failed to attach device from hostdev.xml error: internal error: unable to execute QEMU command 'device_add': Duplicate ID 'hostdev0' for device
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.
Although the priority of the failover bug is low, it is a real bug and can be reproduced by libvirt. So it is valuable for us to continue to track.