Hide Forgot
Description of problem: Error info should be more friendly when plug the vf to guest with <iommu model='intel'/> Version-Release number of selected component (if applicable): qemu-kvm-rhev-2.6.0-20.el7.x86_64 libvirt-2.0.0-5.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1. # lspci | grep Eth 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5764M Gigabit Ethernet PCIe (rev 10) 03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 03:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 03:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 03:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 03:10.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 2.Prepare a guest which is q35 machine type and with <iommu model='intel'/> # virsh dumpxml q35 <domain type='kvm'> <name>q35</name> <uuid>0d308326-beef-4136-9386-0537185439b2</uuid> ... <os> <type arch='x86_64' machine='pc-q35-rhel7.3.0'>hvm</type> <boot dev='hd'/> </os> ... <iommu model='intel'/> ... 3.# virsh start q35 Domain q35 started # virsh dumpxml q35 | grep interface -A 4 # # 4.# cat vf1.xml <interface type='hostdev' managed='yes'> <mac address='52:89:bb:8c:ca:fc'/> <driver name='vfio'/> <source> <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/> </source> <alias name='hostdev0'/> <model type='virtio'/> </interface> # virsh attach-device q35 vf1.xml error: Failed to attach device from vf1.xml error: Unable to read from monitor: Connection reset by peer Actual results: 5. Destroy the guest， Add the xml as below to the guest xml <interface type='hostdev' managed='yes'> <mac address='52:89:bb:8c:ca:fc'/> <driver name='vfio'/> <source> <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/> </source> <alias name='hostdev0'/> <model type='virtio'/> </interface> # virsh edit q35 Domain q35 XML configuration edited. # virsh start q35 error: Failed to start domain q35 error: internal error: qemu unexpectedly closed the monitor: ort=1 -device vfio-pci,host=03:10.0,id=hostdev0,bus=pci.2,addr=0x5 -device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x4 -msg timestamp=on qemu: hardware error: Device at bus pci.2 addr 05.0 requires iommu notifier which is currently not supported by intel-iommu emulation CPU #0: EAX=00000000 EBX=00000000 ECX=00000000 EDX=00010673 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300 CS =f000 ffff0000 0000ffff 00009b00 SS =0000 00000000 0000ffff 00009300 DS =0000 00000000 0000ffff 00009300 FS =0000 00000000 0000ffff 00009300 GS =0000 00000000 0000ffff 00009300 LDT=0000 00000000 0000ffff 00008200 TR =0000 00000000 0000ffff 00008b00 GDT= 00000000 0000ffff IDT= 00000000 0000ffff CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000 Actual results: According to the bug https://bugzilla.redhat.com/show_bug.cgi?id=1346920, it is as expected, But I check the http://libvirt.org/formatdomain.html#elementsIommu, it also has no more related info about this part. Expected results: I think libvirt should give a more friendly error info or add more info in the libvirt.org Additional info:
Could you please post the VM log file (/var/log/libvirt/qemu/q35.log) after qemu crashed/failed to add the device after step4.
Created attachment 1191834 [details] the log of q35
(In reply to Peter Krempa from comment #2) > Could you please post the VM log file (/var/log/libvirt/qemu/q35.log) after > qemu crashed/failed to add the device after step4. OK, I have attached the log in comment3
I would be more worried about the qemu crash, but lets see else can be done.
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.
Test with below packages, the result is as expected: # rpm -q libvirt-libs qemu-kvm libvirt-libs-6.10.0-1.module+el8.4.0+8898+a84e86e1.x86_64 qemu-kvm-5.2.0-2.module+el8.4.0+9186+ec44380f.x86_64 1. start vm with "<iommu model='intel'/>"; 2. hotplug hostdev interface device as below: [root@host ~]# cat net.xml <interface type='hostdev' managed='yes'> <mac address='52:89:bb:8c:ca:fc'/> <driver name='vfio'/> <source> <address type='pci' domain='0x0000' bus='0x82' slot='0x10' function='0x3'/> </source> </interface> [root@host ~]# virsh attach-device rhel net.xml error: Failed to attach device from net.xml error: internal error: unable to execute QEMU command 'device_add': Device assignment is not allowed without enabling caching-mode=on for Intel IOMMU. [root@host ~]# virsh attach-interface rhel hostdev 0000:82:10.3 --managed error: Failed to attach interface error: internal error: unable to execute QEMU command 'device_add': Device assignment is not allowed without enabling caching-mode=on for Intel IOMMU. 3. destroy the vm, and cold plug the interface, then try to start the vm: [root@host ~]# virsh destroy rhel Domain rhel destroyed [root@host ~]# virsh attach-device rhel net.xml --config Device attached successfully [root@host ~]# virsh start rhel error: Failed to start domain rhel error: internal error: qemu unexpectedly closed the monitor: 2020-12-16T01:46:49.200693Z qemu-kvm: We need to set caching-mode=on for intel-iommu to enable device assignment with IOMMU protection. The bug is fixed.