Bug 2048601
| Summary: | [DPDK][Mellanox][NIC Partitioning] Port creation fails on CX4 VF with higher order of PCI address. | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux Fast Datapath | Reporter: | Haresh Khandelwal <hakhande> | |
| Component: | openvswitch2.15 | Assignee: | Timothy Redaelli <tredaelli> | |
| Status: | CLOSED ERRATA | QA Contact: | liting <tli> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | medium | |||
| Version: | RHEL 8.0 | CC: | apevec, cfields, cfontain, chrisw, ctrautma, dmarchan, ekuris, hakhande, jhsiao, ksundara, ktraynor, ralongi | |
| Target Milestone: | --- | Keywords: | Triaged | |
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | openvswitch2.15-2.15.0-78.el8fdp | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | 2040292 | |||
| : | 2059758 (view as bug list) | Environment: | ||
| Last Closed: | 2022-03-30 16:28:45 UTC | Type: | --- | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 2040292, 2059758, 2077451 | |||
Verify pass on CX4 card.
[root@netqe30 ~]# lspci|grep af
af:00.0 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4]
af:00.1 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4]
Reproduced on openvswitch2.13-2.13.0-72.el8fdp.x86_64.
[root@netqe30 ~]# echo 64 > /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/sriov_numvfs
[root@netqe30 ~]# echo 64 > /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.1/sriov_numvfs
[root@netqe30 ~]# ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
[root@netqe30 ~]# ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem=1024,1024
[root@netqe30 ~]# ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x40000004000000
[root@netqe30 ~]# ovs-vsctl add-br ovsbr0 -- set bridge ovsbr0 datapath_type=netdev
[root@netqe30 ~]# ovs-vsctl add-bond ovsbr0 dpdkbond dpdk0 dpdk1 "bond_mode=active-backup" -- set Interface dpdk0 type=dpdk options:dpdk-devargs=0000:af:07.7 -- set Interface dpdk1 type=dpdk options:dpdk-devargs=0000:af:18.0
[root@netqe30 ~]# tail -f /var/log/openvswitch/ovs-vswitchd.log
2022-03-16T03:06:20.827Z|00490|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 26 created.
2022-03-16T03:06:20.830Z|00491|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 54 created.
2022-03-16T03:06:20.830Z|00492|dpif_netdev|INFO|There are 2 pmd threads on numa node 0
2022-03-16T03:06:20.830Z|00493|dpdk|INFO|Device with port_id=65 already stopped
2022-03-16T03:06:22.258Z|00494|netdev_dpdk|INFO|Port 65: b6:cb:41:18:e0:d3
2022-03-16T03:06:22.259Z|00495|dpif_netdev|WARN|There's no available (non-isolated) pmd thread on numa node 1. Queue 0 on port 'dpdk0' will be assigned to the pmd on core 26 (numa node 0). Expect reduced performance.
2022-03-16T03:06:22.260Z|00496|bridge|INFO|bridge ovsbr0: added interface dpdk0 on port 1
2022-03-16T03:06:22.297Z|00497|dpdk|INFO|EAL: PCI device 0000:af:18.0 on NUMA socket 1
2022-03-16T03:06:22.297Z|00498|dpdk|INFO|EAL: probe driver: 15b3:1014 net_mlx5
2022-03-16T03:06:22.323Z|00499|dpdk|ERR|net_mlx5: unable to allocate switch domain: File exists
2022-03-16T03:06:22.325Z|00500|netdev_dpdk|WARN|Error attaching device '0000:af:18.0' to DPDK
2022-03-16T03:06:22.325Z|00501|netdev|WARN|dpdk1: could not set configuration (Invalid argument)
2022-03-16T03:06:22.325Z|00502|dpdk|ERR|Invalid port_id=128
verify pass on openvswitch2.15-2.15.0-78.el8fdp
[root@netqe30 ~]# ovs-vsctl add-bond ovsbr0 dpdkbond dpdk0 dpdk1 "bond_mode=active-backup" -- set Interface dpdk0 type=dpdk options:dpdk-devargs=0000:af:07.7 -- set Interface dpdk1 type=dpdk options:dpdk-devargs=0000:af:18.0
[root@netqe30 ~]# ovs-vsctl show
3886fc27-762c-4c5a-a32a-a9fe696bb2a2
Bridge ovsbr0
datapath_type: netdev
Port dpdkbond
Interface dpdk0
type: dpdk
options: {dpdk-devargs="0000:af:07.7"}
Interface dpdk1
type: dpdk
options: {dpdk-devargs="0000:af:18.0"}
Port ovsbr0
Interface ovsbr0
type: internal
ovs_version: "2.15.4"
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (openvswitch2.15 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:1145 |
* Tue Mar 01 2022 Timothy Redaelli <tredaelli> - 2.15.0-78 - Change RTE_ETH_MAXPORTS to 1024 [RH git: cbe02d895f] (#2048601) Resolves: #2048601