Hide Forgot
Description of problem: SRQ support is not available in libbnxt_re. We are preparing the patches to be pushed upstream. This needs to be pulled in once it is accepted in upstream. Steps to Reproduce: 1. Load bnxt_re drivers and install rdma-core with libbnxt_re 2. Run ibv_srq_ping_pong Actual results: The test fails now since the support is not available in the library. Additional info:
Thanks for the test info in BZ description - adding qa_ack+
Selvin, It cannot be pulled in until it is (minimally) in a for-next, or for-rc branch of rdma-core. Please list the upstream commit-id's once it is pulled in. Then Jarod can assess whether he can pull it into RHEL-7.6 rdma-core, or not.
(In reply to Don Dutile from comment #4) > Selvin, > It cannot be pulled in until it is (minimally) in a for-next, or for-rc > branch of rdma-core. > Please list the upstream commit-id's once it is pulled in. > > Then Jarod can assess whether he can pull it into RHEL-7.6 rdma-core, or not. Don, This is not submitted yet. My colleague would be pushing the patches to upstream this week. Once it is accepted, i will update this BZ with the commit id. Thanks, Selvin
Jarod, This commit is in rdma-core now. commit 602bb7ce1502c2e874eec6f1f677fd0539ceba01 - bnxt_re: Add shared receive queue support to broadcom's roce driver Thanks, Selvin
(In reply to Selvin Xavier from comment #6) > Jarod, > This commit is in rdma-core now. > > commit 602bb7ce1502c2e874eec6f1f677fd0539ceba01 - bnxt_re: Add shared > receive queue support to broadcom's roce driver > > Thanks, > Selvin ... are you sure it isn't in RHEL-7.6's rdma-core already?
(In reply to Don Dutile from comment #7) > (In reply to Selvin Xavier from comment #6) > > Jarod, > > This commit is in rdma-core now. > > > > commit 602bb7ce1502c2e874eec6f1f677fd0539ceba01 - bnxt_re: Add shared > > receive queue support to broadcom's roce driver > > > > Thanks, > > Selvin > > ... are you sure it isn't in RHEL-7.6's rdma-core already? This was posted on last month Sept 17. See the posting message. https://www.spinics.net/lists/linux-rdma/msg69055.html Also, i have confirmed from the source rpms rdma-core-17.2-3.el7.src.rpm and rdma-core-19-3.el8.src.rpm that this is not yet available.
I think this is likely 7.7 material with a z-stream backport now, but I'm not 100% certain on that. We'll also need a RHEL8 bug for it to get backported there as well.
(In reply to Jarod Wilson from comment #9) > I think this is likely 7.7 material with a z-stream backport now, but I'm > not 100% certain on that. We'll also need a RHEL8 bug for it to get > backported there as well. +1, *but*, a (7.6)z-stream backport would require QE to re-test bnxt-re, as that enables new functionality (srpt uses SRQ if available), thus, it's not a 'bug fix' for z-stream, but new functionality.
Oh, right, this is new feature enablement, not a straight bug-fix, so probably no go on the z-stream backport for 7.6.
Created a BZ for 8.x https://bugzilla.redhat.com/show_bug.cgi?id=1639692
Marking verified as our test has passed on 3.10.0-1048.el7.x86_64. Also, the ibv_srq_pingpong test of our libibverbs-utils has passed. [root@rdma-dev-25 ~]$ cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.7 Beta (Maipo) [root@rdma-dev-25 ~]$ uname -r 3.10.0-1048.el7.x86_64 [root@rdma-dev-25 ~]$ ip a s bnxt_roce 6: bnxt_roce: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000 link/ether 00:0a:f7:ea:ce:a0 brd ff:ff:ff:ff:ff:ff inet 172.31.40.125/24 brd 172.31.40.255 scope global noprefixroute dynamic bnxt_roce valid_lft 1679sec preferred_lft 1679sec inet6 fe80::20a:f7ff:feea:cea0/64 scope link noprefixroute valid_lft forever preferred_lft forever [root@rdma-dev-25 ~]$ ibv_devinfo hca_id: bnxt_re0 transport: InfiniBand (0) fw_ver: 212.0.106.0 node_guid: 020a:f7ff:feea:cea0 sys_image_guid: 020a:f7ff:feea:cea0 vendor_id: 0x14e4 vendor_part_id: 5652 hw_ver: 0x4540 phys_port_cnt: 1 port: 1 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 4096 (5) sm_lid: 0 port_lid: 0 port_lmc: 0x00 link_layer: Ethernet [root@rdma-dev-25 ~]$ rpm -qa | grep -E 'rdma|verbs' libibverbs-22.1-1.el7.x86_64 librdmacm-utils-22.1-1.el7.x86_64 rdma-core-22.1-1.el7.x86_64 libibverbs-utils-22.1-1.el7.x86_64 kernel-kernel-infiniband-librdmacm-utils-0.1-53.noarch kernel-kernel-infiniband-libibverbs-utils-0.1-36.noarch librdmacm-22.1-1.el7.x86_64 kernel-kernel-infiniband-nfsordma-1.0-11.noarch rdma-core-devel-22.1-1.el7.x86_64 [root@rdma-dev-25 ~]$ rpm -ql libibverbs | grep libbnxt_re /usr/lib64/libibverbs/libbnxt_re-rdmav22.so [root@rdma-dev-25 ~]$ ibv_srq_pingpong -g 0 -d bnxt_re0 -i 1 local address: LID 0x0000, QPN 0x001022, PSN 0xa4ff12, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x001023, PSN 0x40a3cb, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x001024, PSN 0xcd2d83, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x001025, PSN 0xb43edf, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x001026, PSN 0xb7483d, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x001027, PSN 0x4e7225, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x001028, PSN 0x8325ea, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x001029, PSN 0xee2965, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x00102a, PSN 0x0cc46f, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x00102b, PSN 0x8f678e, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x00102c, PSN 0x885450, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x00102d, PSN 0x3cf9da, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x00102e, PSN 0xfab888, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x00102f, PSN 0xc0385e, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x001030, PSN 0x1a6bfa, GID fe80::20a:f7ff:feea:cea0 local address: LID 0x0000, QPN 0x001031, PSN 0x7ad86c, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001022, PSN 0x414c12, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001023, PSN 0xfb6ccb, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001024, PSN 0xeac283, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001025, PSN 0x2aafdf, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001026, PSN 0xab653d, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001027, PSN 0x4ccb25, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001028, PSN 0xc50aea, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001029, PSN 0xbaaa65, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x00102a, PSN 0x05b16f, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x00102b, PSN 0x7a508e, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x00102c, PSN 0xe38950, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x00102d, PSN 0xb08ada, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x00102e, PSN 0x757588, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x00102f, PSN 0xd0b15e, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001030, PSN 0xd3f0fa, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001031, PSN 0xf6796c, GID fe80::20a:f7ff:feea:cd90 8192000 bytes in 0.00 seconds = 40604.71 Mbit/sec 1000 iters in 0.00 seconds = 1.61 usec/iter [root@rdma-dev-25 ~]$ echo $? 0 [root@rdma-dev-25 ~]$ [root@rdma-dev-26 ~]$ ibv_devinfo hca_id: bnxt_re0 transport: InfiniBand (0) fw_ver: 212.0.106.0 node_guid: 020a:f7ff:feea:cd90 sys_image_guid: 020a:f7ff:feea:cd90 vendor_id: 0x14e4 vendor_part_id: 5652 hw_ver: 0x4540 phys_port_cnt: 1 port: 1 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 4096 (5) sm_lid: 0 port_lid: 0 port_lmc: 0x00 link_layer: Ethernet [root@rdma-dev-26 ~]$ ibv_srq_pingpong -g 0 -d bnxt_re0 -i 1 172.31.40.125 local address: LID 0x0000, QPN 0x001022, PSN 0x414c12, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x001023, PSN 0xfb6ccb, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x001024, PSN 0xeac283, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x001025, PSN 0x2aafdf, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x001026, PSN 0xab653d, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x001027, PSN 0x4ccb25, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x001028, PSN 0xc50aea, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x001029, PSN 0xbaaa65, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x00102a, PSN 0x05b16f, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x00102b, PSN 0x7a508e, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x00102c, PSN 0xe38950, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x00102d, PSN 0xb08ada, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x00102e, PSN 0x757588, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x00102f, PSN 0xd0b15e, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x001030, PSN 0xd3f0fa, GID fe80::20a:f7ff:feea:cd90 local address: LID 0x0000, QPN 0x001031, PSN 0xf6796c, GID fe80::20a:f7ff:feea:cd90 remote address: LID 0x0000, QPN 0x001022, PSN 0xa4ff12, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001023, PSN 0x40a3cb, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001024, PSN 0xcd2d83, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001025, PSN 0xb43edf, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001026, PSN 0xb7483d, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001027, PSN 0x4e7225, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001028, PSN 0x8325ea, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001029, PSN 0xee2965, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x00102a, PSN 0x0cc46f, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x00102b, PSN 0x8f678e, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x00102c, PSN 0x885450, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x00102d, PSN 0x3cf9da, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x00102e, PSN 0xfab888, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x00102f, PSN 0xc0385e, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001030, PSN 0x1a6bfa, GID fe80::20a:f7ff:feea:cea0 remote address: LID 0x0000, QPN 0x001031, PSN 0x7ad86c, GID fe80::20a:f7ff:feea:cea0 8192000 bytes in 0.00 seconds = 76382.28 Mbit/sec 1000 iters in 0.00 seconds = 0.86 usec/iter [root@rdma-dev-26 ~]$ echo $? 0 [root@rdma-dev-26 ~]$ Test results for libibverbs-utils on rdma-dev-26: Result | Status | Test ---------+--------+------------------------------------ PASS | 0 | ibv_devices PASS | 0 | ibv_devinfo PASS | 0 | ibv_srq_pingpong
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2100