Description of problem: DPDK CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF and queue_idx out of range issue with X710 (i40e PMD) cards and more than 64 cores per host Version-Release number of selected component (if applicable): OVS 2.6.1 with DPDK 16.11 Additional info: We need to find a fix for this. E.g., on a customer system with 88 cores, the following error message will show up on PMD initialization: ~~~ Jul 28 16:15:53 netqe23.knqe.lab.eng.bos.redhat.com ovs-vswitchd[8125]: PMD: i40e_pf_get_vsi_by_qindex(): queue_idx out of range. VMDQ configured? ~~~ * we are hitting: `i40e_pf_get_vsi_by_qindex(): queue_idx out of range. VMDQ configured?` => according to Intel, need to set in this particular case: ~~~ CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF=88 ~~~ http://openvswitch.org/support/dist-docs-2.5/INSTALL.DPDK.md.html "By default with DPDK 2.2, a maximum of 64 TX queues can be used with an Intel XL710 Network Interface on a platform with more than 64 logical cores. If a user attempts to add an XL710 interface as a DPDK port type to a system as described above, an error will be reported that initialization failed for the 65th queue. OVS will then roll back to the previous successful queue initialization and use that value as the total number of TX queues available with queue locking. If a user wishes to use more than 64 queues and avoid locking, then the CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF config parameter in DPDK must be increased to the desired number of queues. Both DPDK and OVS must be recompiled for this change to take effect."
OVS-DPDK is designed to cater for this case. In OVS 2.6 the num of tx queue's requested = num cores + 1. In the case where there is 72 cores, OVS will attempt to setup 73 txq's on the i40e: - txq's 0-63 will setup correctly - txq 64 will fail and show the reported log message PMD: i40e_pf_get_vsi_by_qindex(): queue_idx out of range. VMDQ configured? Then OVS-DPDK will reconfigure the i40e, setup 64 queues and works with this amount. 2017-08-10T11:19:31Z|00027|dpdk|INFO|Interface dpdk0 txq(64) setup error: Input/output error 2017-08-10T11:19:31Z|00028|dpdk|INFO|Retrying setup with (rxq:1 txq:64)
Closing https://access.redhat.com/solutions/3147281