Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 2079528

Summary:	[DPDK][Mellanox][NIC Partitioning] Port creation fails on CX4 VF with higher order of PCI address.
Product:	Red Hat Enterprise Linux Fast Datapath	Reporter:	OvS team <ovs-bugzilla>
Component:	openvswitch2.17	Assignee:	Timothy Redaelli <tredaelli>
Status:	CLOSED ERRATA	QA Contact:	liting <tli>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	RHEL 9.0	CC:	ctrautma, jhsiao, ralongi, tredaelli
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openvswitch2.17-2.17.0-12.el9fdp	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-05-27 18:05:34 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description OvS team 2022-04-27 17:03:26 UTC

+++ This bug was initially created as a clone of Bug #2077451 +++

+++ This bug was initially created as a clone of Bug #2059758 +++

+++ This bug was initially created as a clone of Bug #2048601 +++

+++ This bug was initially created as a clone of Bug #2040292 +++

Description of problem:

The OSP deployment involves creation of 64 VFs on 4 CX4 ports (256 VFs), spread across 2 NUMA. A dpdk bond is created between VF0 of port 0 (NUMA0) and VF0 of port 3 (NUMA1). The port creation fails with below error.

    Bridge br-link0
        fail_mode: standalone
        datapath_type: netdev
        Port dpdkbond0
            Interface dpdk1
                type: dpdk
                options: {dpdk-devargs="0000:d8:00.2", n_rxq="1"}
                error: "Error attaching device '0000:d8:00.2' to DPDK"
            Interface dpdk0
                type: dpdk
                options: {dpdk-devargs="0000:12:00.2", n_rxq="1"}
        Port br-link0
            Interface br-link0
                type: internal
    ovs_version: "2.13.4"

How reproducible:

As a quick reproducer with limited VFs, we could build Openvswitch with CONFIG_RTE_MAX_ETHPORTS set to 32 and create a user bridge with dpdk bonds involving a VF with higher order PCI address.


Actual results:
Port creation fails

Expected results:
Port creation shall be successful.

SOS reports are attached in the case.
PS: The port creations were successful when the customer went ahead with 32 VFs per port.

Comment 1 OvS team 2022-04-27 17:03:29 UTC

* Wed Apr 27 2022 Open vSwitch CI <ovs-ci> - 2.17.0-12
- Merging upstream branch-2.17 [RH git: 7a9f21a896]
    Commit list:
    60e7badd6e dpif-netdev-avx512: Fix ubsan shift error in bitmasks.
    9cc329ec5b python: Politely handle misuse of table.condition.
    0631be2b5a ofproto-xlate: Fix crash when forwarding packet between legacy_l3 tunnels.
    df97903099 system-traffic: Fix fragment reassembly with L3 L4 protocol information.
    ba159ee0f9 cirrus: Update FreeBSD versions.


* Thu Apr 21 2022 Timothy Redaelli <tredaelli> - 2.17.0-11
- Set RTE_ETH_MAXPORTS to 1024 [RH git: c02e6bcdc4] (#2077451)
    Resolves: #2077451

Comment 2 Timothy Redaelli 2022-04-29 14:06:45 UTC

*** Bug 2077451 has been marked as a duplicate of this bug. ***

Comment 5 liting 2022-05-16 01:32:21 UTC

verify pass on openvswitch2.17-2.17.0-12.el9fdp.x86_64
[root@netqe30 ~]# rpm -qa|grep openvs
openvswitch-selinux-extra-policy-1.0-31.el9fdp.noarch
openvswitch2.17-2.17.0-12.el9fdp.x86_64
[root@netqe30 ~]# uname -r
5.14.0-70.13.1.el9_0.x86_64

[root@netqe30 ~]# echo 64 > /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/sriov_numvfs
[root@netqe30 ~]# echo 64 > /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.1/sriov_numvfs
[root@netqe30 ~]# ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
[root@netqe30 ~]#  ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem=1024,1024
[root@netqe30 ~]#  ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x40000004000000
[root@netqe30 ~]#  ovs-vsctl add-br ovsbr0 -- set bridge ovsbr0 datapath_type=netdev
[root@netqe30 ~]# ovs-vsctl add-bond ovsbr0 dpdkbond dpdk0 dpdk1 "bond_mode=active-backup" -- set Interface dpdk0 type=dpdk options:dpdk-devargs=0000:af:07.7 -- set Interface dpdk1 type=dpdk options:dpdk-devargs=0000:af:18.0

Comment 7 errata-xmlrpc 2022-05-27 18:05:34 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (openvswitch2.17), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:4792