Bug 1678274

Summary: [Broadcom CCX 7.7 FEAT]libbnxt_re: Add support for Broadcom 57500 adapters in user library
Product: Red Hat Enterprise Linux 7 Reporter: Selvin Xavier (Broadcom) <sxavier>
Component: rdma-coreAssignee: Jarod Wilson <jarod>
Status: CLOSED ERRATA QA Contact: Afom T. Michael <tmichael>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.7CC: anantha.subramanyam, brcm-roce-dev.pdl, ddutile, infiniband-qe, mstowell, rdma-dev-team, selvin.xavier, sxavier, tumeya, vinodkumar.parmar
Target Milestone: rcKeywords: FutureFeature
Target Release: 7.7   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rdma-core-22.1-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-08-06 12:46:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1614007    

Description Selvin Xavier (Broadcom) 2019-02-18 12:06:40 UTC
D

Comment 1 Selvin Xavier (Broadcom) 2019-02-18 12:08:04 UTC
Following patch is under review on the rdma community:

https://marc.info/?l=linux-rdma&m=155003698431927&w=2
https://github.com/linux-rdma/rdma-core/pull/453

Will update the git-tag number once the patch is pulled in.

Comment 2 Jarod Wilson 2019-02-18 15:46:45 UTC
Looks like just the one patch required here, which we should be able to pull once it's merged upstream. Definitely drop a commit hash here once it's merged so we don't overlook it.

Comment 5 Selvin Xavier (Broadcom) 2019-03-06 07:35:32 UTC
Jarod,
 The patch is now merged to rdma-core

https://github.com/linux-rdma/rdma-core/commit/a9f35f4f8e24b294685f30bdc90b14a23c6d403d

Thanks,
Selvin

Comment 6 Selvin Xavier (Broadcom) 2019-03-27 05:30:20 UTC
The two commit relevant for this BZ is given below.a9f35f4f8e24b294685f30bdc90b14a23c6d403d is dependant on  23ce83e8542ccfb1f80422d84c1d21d0fabd8542

a9f35f4f8e24b294685f30bdc90b14a23c6d403d - bnxt_re/lib: Enable Broadcom's 57500 RoCE adapter
23ce83e8542ccfb1f80422d84c1d21d0fabd8542 - Update kernel headers

Comment 12 Afom T. Michael 2019-05-30 20:46:39 UTC
Hello Selvin,

As I commented on https://bugzilla.redhat.com/show_bug.cgi?id=1643182#c19, we don't have two hosts with Broadcom 575xx to run full tests. Can you test this & provide results?

---- below are some outputs ----
[root@rdma-dev-16 ~]$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.7 Beta (Maipo)
[root@rdma-dev-16 ~]$ uname -r
3.10.0-1048.el7.x86_64
[root@rdma-dev-16 ~]$ rpm -qa | grep -E 'rdma|verbs'
libibverbs-22.1-1.el7.x86_64
librdmacm-utils-22.1-1.el7.x86_64
rdma-core-22.1-1.el7.x86_64
libibverbs-utils-22.1-1.el7.x86_64
librdmacm-22.1-1.el7.x86_64
rdma-core-devel-22.1-1.el7.x86_64
[root@rdma-dev-16 ~]$
[root@rdma-dev-16 ~]$ rpm -ql libibverbs | grep libbnxt_re
/usr/lib64/libibverbs/libbnxt_re-rdmav22.so
[root@rdma-dev-16 ~]$
[root@rdma-dev-16 ~]$ lspci | grep -i Broadcom
01:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS-3 3008 [Fury] (rev 02)
02:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
02:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
03:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
03:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
04:00.0 Ethernet controller: Broadcom Inc. and subsidiaries Device 1750
04:00.1 Ethernet controller: Broadcom Inc. and subsidiaries Device 1750
[root@rdma-dev-16 ~]$ ibv_devices 
    device          	   node GUID
    ------          	----------------
    bnxt_re0        	b22628fffe4bcd80
    bnxt_re1        	b22628fffe4bcd81
    hfi1_0          	0011750101671ecf
[root@rdma-dev-16 ~]$ ibv_devinfo -d bnxt_re0
hca_id:	bnxt_re0
	transport:			InfiniBand (0)
	fw_ver:				215.0.170.0
	node_guid:			b226:28ff:fe4b:cd80
	sys_image_guid:			b226:28ff:fe4b:cd80
	vendor_id:			0x14e4
	vendor_part_id:			5968
	hw_ver:				0x2100
	phys_port_cnt:			1
		port:	1
			state:			PORT_ACTIVE (4)
			max_mtu:		4096 (5)
			active_mtu:		1024 (3)
			sm_lid:			0
			port_lid:		0
			port_lmc:		0x00
			link_layer:		Ethernet

[root@rdma-dev-16 ~]$ ibv_devinfo -d bnxt_re1
hca_id:	bnxt_re1
	transport:			InfiniBand (0)
	fw_ver:				215.0.170.0
	node_guid:			b226:28ff:fe4b:cd81
	sys_image_guid:			b226:28ff:fe4b:cd81
	vendor_id:			0x14e4
	vendor_part_id:			5968
	hw_ver:				0x2100
	phys_port_cnt:			1
		port:	1
			state:			PORT_DOWN (1)
			max_mtu:		4096 (5)
			active_mtu:		1024 (3)
			sm_lid:			0
			port_lid:		0
			port_lmc:		0x00
			link_layer:		Ethernet

[root@rdma-dev-16 ~]$

Comment 13 Selvin Xavier (Broadcom) 2019-06-10 05:03:51 UTC
Broadcom QA is testing with alpha kernel on 575xx adapters now. Will post the update after this.

Comment 14 Afom T. Michael 2019-06-25 12:09:06 UTC
(In reply to Selvin Xavier from comment #13)
> Broadcom QA is testing with alpha kernel on 575xx adapters now. Will post
> the update after this.

Hi Selvin,

Any update on testing outcome?

Comment 15 vinodkumar.parmar 2019-06-26 06:26:44 UTC
575xx adapters support has been verified with RH7.7 Beta OS. 

Test coverage: 

iSER, NFSoRDMA, ibperf tests

Comment 16 Afom T. Michael 2019-06-26 17:51:41 UTC
Moving to 'Verified/SanityOnly' as Broadcom test passed (comment #15).

Thanks Vinodkumar/Broadcom. If you have any more test results, please add them here anytime.

Comment 18 errata-xmlrpc 2019-08-06 12:46:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2100

Comment 19 Selvin Xavier (Broadcom) 2019-11-13 04:44:41 UTC
Clearing the needinfo field as all info is provided already by Vinod.