Bug 1300378
| Summary: | ovs-vswitchd exits with IOMMU permissions error when using vfio-pci and 82576 gigabit nic | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Terry Wilson <twilson> |
| Component: | openvswitch-dpdk | Assignee: | Panu Matilainen <pmatilai> |
| Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Red Hat Kernel QE team <kernel-qe> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 7.2 | CC: | ekuris, fleitner, mlopes, mzhan, pmatilai, twilson |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-08-10 09:49:22 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Terry Wilson
2016-01-20 15:36:26 UTC
Terry, Have you tried with UIO? Thanks, fbl Drop "-d /usr/lib64/librte_pmd_e1000.so" from the command line, it is not needed with OVS anyway and in this case its actually harmful: openvswitch-dpdk is linked statically to DPDK 2.0 and you're trying to load a driver from a differently configured DPDK to another one, which is the reason for at least the undefined symbol, and wouldn't be surprised if the rest starts too when that problem is out of the way. fbl: UIO worked. panu: It did work with UIO with the -d. So you are saying to never pass the -d option to OVS? If so, all of our documentation is wrong and will need to be changed. Terry Yes, you never need -d with OVS unless you have a 3rd party PMD. Also openvswitch-dpdk does not need (and cannot use anything from) the dpdk package you apparently have installed since /usr/lib64/librte_pmd_e1000.so exists. And yeah, looking closer at the startup log, it starts despite the -d because e1000 is the wrong driver for your NIC anyway, it uses the igb driver. The IOMMU error is a separate issue related to IOMMU (which is required for VFIO but not UIO), possibly hardware limitation. What kind of system is this? > What kind of system is this?
The machine with the issue belongs to ekuris, so I'll add him as needinfo.
based on rhel OS Eran, the question was about the system hardware, OS is rather obvious in this context. lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 24 On-line CPU(s) list: 0-23 Thread(s) per core: 2 Core(s) per socket: 6 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 45 Model name: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz Stepping: 7 CPU MHz: 1199.953 BogoMIPS: 4609.25 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 15360K NUMA node0 CPU(s): 0-5,12-17 NUMA node1 CPU(s): 6-11,18-23 05:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01) Does the error go away if you bind both NICs to vfio-pci? If not, please add the output of 'find /sys/kernel/iommu_groups/ -type l' from the system. Redirecting the question to ekuris since these are his machines. I have to check it again I can answer that after I deploy the setup Been over half a year now, feel free to reopen if/when you can reproduce the issue. That said, I suspect the system log would contain something along the lines of: "vfio-pci 0000:05:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor." ...at the time of attempted use, which would be a limitation of the server and its configuration and not a bug. But obviously without hard data this is just a guess. |