Bug 1627272
| Summary: | boot guest with q35+vIOMMU+ device assignment, qemu crash when return assigned network devices from vfio driver to ixgbe in guest | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Pei Zhang <pezhang> | |
| Component: | qemu-kvm-rhev | Assignee: | Peter Xu <peterx> | |
| Status: | CLOSED ERRATA | QA Contact: | Sitong Liu <siliu> | |
| Severity: | urgent | Docs Contact: | ||
| Priority: | high | |||
| Version: | 7.7 | CC: | ailan, alex.williamson, chayang, hhuang, jinzhao, juzhang, mtessun, peterx, siliu, virt-maint, yfu | |
| Target Milestone: | rc | Keywords: | Regression, ZStream | |
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | qemu-kvm-rhev-2.12.0-19.el7 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1629562 1647719 (view as bug list) | Environment: | ||
| Last Closed: | 2019-08-22 09:18:53 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1629562, 1647719, 1649160, 1651787 | |||
|
Description
Pei Zhang
2018-09-10 14:20:04 UTC
Errno 17 is EEXIST, vIOMMU probably didn't invalidate all the mappings from the DPDK domain before the device was added back to the static identity domain. (In reply to Alex Williamson from comment #3) > Errno 17 is EEXIST, vIOMMU probably didn't invalidate all the mappings from > the DPDK domain before the device was added back to the static identity > domain. Agreed. It's very possible that we didn't handle the case where the context entry switches from valid to invalid. In that case now we might ignore the shadow sync but actually we should do a "unmap all" instead. I'm preparing a package for test. Will update soon. Pei, could you try this package to see whether it can fix the problem? https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=18303398 I made a mistake on building previous package... Please instead try this one: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=18303752 Please make sure that the package is suffixed with bz1627272_2 not bz1627272. (In reply to Peter Xu from comment #6) > I made a mistake on building previous package... Please instead try this one: > > https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=18303752 > > Please make sure that the package is suffixed with bz1627272_2 not bz1627272. Peter, this bug has gone with above build. Thanks. Versions: 3.10.0-945.el7.x86_64 qemu-kvm-rhev-2.12.0-15.el7.bz1627272_2.x86_64 Testings: Testing with both PF and VF, following same steps of Description, everything works well as expected. - Qemu works well - No any error in both host and guest #dmesg - testpmd in guest can receive packets. Thanks for the quick feedback, Pei. Posted a fix upstream: https://lists.gnu.org/archive/html/qemu-devel/2018-09/msg01555.html Fix included in qemu-kvm-rhev-2.12.0-19.el7 == Verification == Versions: qemu-kvm-rhev-2.12.0-21.el7.x86_64 3.10.0-993.el7.x86_64 Steps: 1. Boot vm with q35 + vIOMMU and the assigned NICs. # /usr/libexec/qemu-kvm -M q35,kernel-irqchip=split \ -cpu host -m 8G \ -device intel-iommu,intremap=on,caching-mode=on,device-iotlb=on \ ... -device vfio-pci,host=0000:5e:00.0,bus=root.1 \ -device vfio-pci,host=0000:5e:00.1,bus=root.2 \ ... 2. In vm, load vfio and start testpmd. # modprobe vfio # modprobe vfio-pci # dpdk-devbind --bind=vfio-pci 0000:01:00.0 # dpdk-devbind --bind=vfio-pci 0000:02:00.0 # echo 4 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages # testpmd> start 3. In vm, quit testpmd, return nic from vfio driver to ixgbe. # testpmd> quit # dpdk-devbind --bind=ixgbe 0000:01:00.0 # dpdk-devbind --bind=ixgbe 0000:02:00.0 # ifconfig enp1s0 up 4. In vm, start testpmd and send packets from another host. # ifconfig enp1s0 down # dpdk-devbind --bind=vfio-pci 0000:01:00.0 # dpdk-devbind --bind=vfio-pci 0000:02:00.0 # testpmd> start Result: After step 3, qemu works well, no error in host/guest dmesg. After step 4, testpmd in guest can receive packets well. Move to 'Verified'. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:2553 |