Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 2048601

Summary: [DPDK][Mellanox][NIC Partitioning] Port creation fails on CX4 VF with higher order of PCI address.
Product: Red Hat Enterprise Linux Fast Datapath Reporter: Haresh Khandelwal <hakhande>
Component: openvswitch2.15Assignee: Timothy Redaelli <tredaelli>
Status: CLOSED ERRATA QA Contact: liting <tli>
Severity: medium Docs Contact:
Priority: medium    
Version: RHEL 8.0CC: apevec, cfields, cfontain, chrisw, ctrautma, dmarchan, ekuris, hakhande, jhsiao, ksundara, ktraynor, ralongi
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openvswitch2.15-2.15.0-78.el8fdp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2040292
: 2059758 (view as bug list) Environment:
Last Closed: 2022-03-30 16:28:45 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2040292, 2059758, 2077451    

Comment 6 OvS team 2022-03-02 00:29:49 UTC
* Tue Mar 01 2022 Timothy Redaelli <tredaelli> - 2.15.0-78
- Change RTE_ETH_MAXPORTS to 1024 [RH git: cbe02d895f] (#2048601)
    Resolves: #2048601

Comment 9 liting 2022-03-16 03:13:42 UTC
Verify pass on CX4 card.
[root@netqe30 ~]# lspci|grep af
af:00.0 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4]
af:00.1 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4]

Reproduced on openvswitch2.13-2.13.0-72.el8fdp.x86_64.
[root@netqe30 ~]# echo 64 > /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/sriov_numvfs
[root@netqe30 ~]# echo 64 > /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.1/sriov_numvfs
[root@netqe30 ~]# ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
[root@netqe30 ~]#  ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem=1024,1024
[root@netqe30 ~]#  ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x40000004000000
[root@netqe30 ~]#  ovs-vsctl add-br ovsbr0 -- set bridge ovsbr0 datapath_type=netdev
[root@netqe30 ~]# ovs-vsctl add-bond ovsbr0 dpdkbond dpdk0 dpdk1 "bond_mode=active-backup" -- set Interface dpdk0 type=dpdk options:dpdk-devargs=0000:af:07.7 -- set Interface dpdk1 type=dpdk options:dpdk-devargs=0000:af:18.0

[root@netqe30 ~]# tail -f /var/log/openvswitch/ovs-vswitchd.log 
2022-03-16T03:06:20.827Z|00490|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 26 created.
2022-03-16T03:06:20.830Z|00491|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 54 created.
2022-03-16T03:06:20.830Z|00492|dpif_netdev|INFO|There are 2 pmd threads on numa node 0
2022-03-16T03:06:20.830Z|00493|dpdk|INFO|Device with port_id=65 already stopped
2022-03-16T03:06:22.258Z|00494|netdev_dpdk|INFO|Port 65: b6:cb:41:18:e0:d3
2022-03-16T03:06:22.259Z|00495|dpif_netdev|WARN|There's no available (non-isolated) pmd thread on numa node 1. Queue 0 on port 'dpdk0' will be assigned to the pmd on core 26 (numa node 0). Expect reduced performance.
2022-03-16T03:06:22.260Z|00496|bridge|INFO|bridge ovsbr0: added interface dpdk0 on port 1
2022-03-16T03:06:22.297Z|00497|dpdk|INFO|EAL: PCI device 0000:af:18.0 on NUMA socket 1
2022-03-16T03:06:22.297Z|00498|dpdk|INFO|EAL:   probe driver: 15b3:1014 net_mlx5
2022-03-16T03:06:22.323Z|00499|dpdk|ERR|net_mlx5: unable to allocate switch domain: File exists
2022-03-16T03:06:22.325Z|00500|netdev_dpdk|WARN|Error attaching device '0000:af:18.0' to DPDK
2022-03-16T03:06:22.325Z|00501|netdev|WARN|dpdk1: could not set configuration (Invalid argument)
2022-03-16T03:06:22.325Z|00502|dpdk|ERR|Invalid port_id=128

verify pass on openvswitch2.15-2.15.0-78.el8fdp
[root@netqe30 ~]# ovs-vsctl add-bond ovsbr0 dpdkbond dpdk0 dpdk1 "bond_mode=active-backup" -- set Interface dpdk0 type=dpdk options:dpdk-devargs=0000:af:07.7 -- set Interface dpdk1 type=dpdk options:dpdk-devargs=0000:af:18.0
[root@netqe30 ~]# ovs-vsctl show
3886fc27-762c-4c5a-a32a-a9fe696bb2a2
    Bridge ovsbr0
        datapath_type: netdev
        Port dpdkbond
            Interface dpdk0
                type: dpdk
                options: {dpdk-devargs="0000:af:07.7"}
            Interface dpdk1
                type: dpdk
                options: {dpdk-devargs="0000:af:18.0"}
        Port ovsbr0
            Interface ovsbr0
                type: internal
    ovs_version: "2.15.4"

Comment 11 errata-xmlrpc 2022-03-30 16:28:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (openvswitch2.15 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1145