Bug 2226768
| Summary: | [RHEL-9.3] libvma test fails consistently on "sockperf pingpong multicast" when tested on MLX4 RoCE devices | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Brian Chae <bchae> |
| Component: | libvma | Assignee: | Kamal Heib <kheib> |
| Status: | NEW --- | QA Contact: | Brian Chae <bchae> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 9.3 | CC: | hwkernel-mgr, rdma-dev-team |
| Target Milestone: | rc | Keywords: | Regression |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | Bug | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Description of problem: "sockperf pingpong multicast" test case consistently fails when libvma is tested on MLX4 RoCE devices. vma test results on rdma-virt-00/rdma-virt-01 & Beaker job J:8112258: 5.14.0-339.el9.x86_64, rdma-core-46.0-1.el9, mlx4, roce.45, ConnectX-3 Pro & mlx4_1 Result | Status | Test ---------+--------+------------------------------------ FAIL | 1 | sockperf pingpong multicast <<<============== PASS | 0 | sockperf throughput multicast PASS | 0 | sockperf throughput unicast PASS | 0 | sockperf pingpong unicast PASS | 0 | sockperf (100 sockets) pingpong multicast PASS | 0 | sockperf (100 sockets) pingpong unicast PASS | 0 | sockperf pingpong multicast pkey/vlan PASS | 0 | sockperf pingpong unicast pkey/vlan ++++++++++++++++++++++++++++++++++ This is a regression from CTC#1 build, RHEL-9.3.0-20230521.45, where the above test case passes. vma test results on rdma-virt-00/rdma-virt-01 & Beaker job J:8112612: 5.14.0-316.el9.x86_64, rdma-core-44.0-2.el9, mlx4, roce.45, ConnectX-3 Pro & mlx4_1 Result | Status | Test ---------+--------+------------------------------------ PASS | 0 | sockperf pingpong multicast <<<========== PASS | 0 | sockperf throughput multicast PASS | 0 | sockperf throughput unicast PASS | 0 | sockperf pingpong unicast PASS | 0 | sockperf (100 sockets) pingpong multicast PASS | 0 | sockperf (100 sockets) pingpong unicast PASS | 0 | sockperf pingpong multicast pkey/vlan PASS | 0 | sockperf pingpong unicast pkey/vlan Checking for failures and known issues: no test failures +++++++++++++++++++++++++++ The same results observed in another MLX4 IB0 host pair : rdma-perf-00/01 vma test results on rdma-perf-00/rdma-perf-01 & Beaker job J:8112242: 5.14.0-339.el9.x86_64, rdma-core-46.0-1.el9, mlx4, roce.45, ConnectX-3 & mlx4_0 Result | Status | Test ---------+--------+------------------------------------ FAIL | 1 | sockperf pingpong multicast PASS | 0 | sockperf throughput multicast PASS | 0 | sockperf throughput unicast PASS | 0 | sockperf pingpong unicast PASS | 0 | sockperf (100 sockets) pingpong multicast PASS | 0 | sockperf (100 sockets) pingpong unicast PASS | 0 | sockperf pingpong multicast pkey/vlan PASS | 0 | sockperf pingpong unicast pkey/vlan Checking for failures and known issues: sockperf pingpong multicast is NOT a known issue on any environment - consider filing a BZ Version-Release number of selected component (if applicable): Clients: rdma-virt-01 Servers: rdma-virt-00 DISTRO=RHEL-9.3.0-20230718.0 + [23-07-26 06:53:45] cat /etc/redhat-release Red Hat Enterprise Linux release 9.3 Beta (Plow) + [23-07-26 06:53:45] uname -a Linux rdma-virt-01.rdma.lab.eng.rdu2.redhat.com 5.14.0-339.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jul 13 07:33:32 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux + [23-07-26 06:53:45] cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.14.0-339.el9.x86_64 root=UUID=63c088f1-90c6-434d-9fad-7d59f30f99c1 ro intel_idle.max_cstate=0 processor.max_cstate=0 intel_iommu=on iommu=on console=tty0 rd_NO_PLYMOUTH crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M resume=UUID=6ec20b3d-43e2-4166-91cd-ae659aee9384 console=ttyS1,115200n81 + [23-07-26 06:53:45] rpm -q rdma-core linux-firmware rdma-core-46.0-1.el9.x86_64 linux-firmware-20230625-137.el9.noarch + [23-07-26 06:53:45] tail /sys/class/infiniband/mlx4_0/fw_ver /sys/class/infiniband/mlx4_1/fw_ver ==> /sys/class/infiniband/mlx4_0/fw_ver <== 2.42.5000 ==> /sys/class/infiniband/mlx4_1/fw_ver <== 2.42.5000 + [23-07-26 06:53:45] lspci + [23-07-26 06:53:45] grep -i -e ethernet -e infiniband -e omni -e ConnectX 02:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe 02:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe 03:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe 03:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe 04:00.0 Network controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] 04:00.1 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.2 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.3 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.4 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.5 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.6 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.7 Network controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 06:00.0 Network controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] + [23-07-26 06:53:46] rpm -q libvma libvma-9.8.20-1.el9.x86_64 How reproducible: 100% Steps to Reproduce: 1. Get both RDMA server & client hosts ready for libvma test with current sockperf, with RHEL-9.3 build as specified above on MLX4 IB0 devices 2. On server host, issue the following commands LD_PRELOAD=libvma.so timeout --preserve-status --kill-after=5m 3m sockperf server -i 172.31.40.200 3. On the client hosts, issue the following command LD_PRELOAD=libvma.so imeout --preserve-status --kill-after=5m 3m sockperf pp -i 172.31.40.200 -t 10 --msg-size=1472 Actual results: Server host: [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.8.20-1 Release built on Jun 1 2023 00:00:00 [0m[0m VMA INFO: Cmd Line: date ++ [%y-%m-%d %H:%M:%S] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m+ [23-07-26 07:01:27] LD_PRELOAD=libvma.so [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.8.20-1 Release built on Jun 1 2023 00:00:00 [0m[0m VMA INFO: Cmd Line: date ++ [%y-%m-%d %H:%M:%S] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m+ [23-07-26 07:01:27] timeout --preserve-status --kill-after=5m 3m sockperf server -i 172.31.40.200 [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.8.20-1 Release built on Jun 1 2023 00:00:00 [0m[0m VMA INFO: Cmd Line: timeout --preserve-status --kill-after=5m 3m sockperf server -i 172.31.40.200 [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.8.20-1 Release built on Jun 1 2023 00:00:00 [0m[0m VMA INFO: Cmd Line: sockperf server -i 172.31.40.200 [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0.8004' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0.8004/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0.8002' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0.8002/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0.8010' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0.8010/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0.8012' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0.8012/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0msockperf: [2;35m== version #3.10-11.gitca16d475d6fa == [0m sockperf: [SERVER] listen on: [ 0] IP = 172.31.40.200 PORT = 11111 # UDP sockperf: Warmup stage (sending a few dummy messages)... sockperf: [tid 61498] using recvfrom() to block on socket(s) Terminated client hosts: + [23-07-26 07:00:16] sleep 10 + [23-07-26 07:00:26] vma_test sockperf pp -i 172.31.40.200 -t 10 --msg-size=1472 + [23-07-26 07:00:26] rm -rf /tmp/vma.txt + [23-07-26 07:00:26] tee /tmp/vma.txt [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.8.20-1 Release built on Jun 1 2023 00:00:00 [0m[0m VMA INFO: Cmd Line: date ++ [%y-%m-%d %H:%M:%S] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m+ [23-07-26 07:00:26] LD_PRELOAD=libvma.so [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.8.20-1 Release built on Jun 1 2023 00:00:00 [0m[0m VMA INFO: Cmd Line: date ++ [%y-%m-%d %H:%M:%S] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m+ [23-07-26 07:00:26] timeout --preserve-status --kill-after=5m 3m sockperf pp -i 172.31.40.200 -t 10 --msg-size=1472 [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.8.20-1 Release built on Jun 1 2023 00:00:00 [0m[0m VMA INFO: Cmd Line: timeout --preserve-status --kill-after=5m 3m sockperf pp -i 172.31.40.200 -t 10 --msg-size=1472 [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.8.20-1 Release built on Jun 1 2023 00:00:00 [0m[0m VMA INFO: Cmd Line: sockperf pp -i 172.31.40.200 -t 10 --msg-size=1472 [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0.8004' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0.8004/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0.8002' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0.8002/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0.8010' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0.8010/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0.8012' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0.8012/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0msockperf: [2;35m== version #3.10-11.gitca16d475d6fa == [0m sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s) [ 0] IP = 172.31.40.200 PORT = 11111 # UDP sockperf: Warmup stage (sending a few dummy messages)... sockperf: Starting test... sockperf: Test end (interrupted by timer) sockperf: Test ended sockperf: No messages were received from the server. Is the server down? + [23-07-26 07:00:40] result=0 + [23-07-26 07:00:40] '[' 0 -ne 0 ']' + [23-07-26 07:00:40] grep -qi -e ' error ' -e 'no messages were received' /tmp/vma.txt + [23-07-26 07:00:40] return 1 + [23-07-26 07:00:40] RQA_check_result -r 1 -t 'sockperf pingpong multicast' Expected results: [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.6.4-1 Release built on Aug 3 2022 00:00:00 [0m[0m VMA INFO: Cmd Line: date ++ [%y-%m-%d %H:%M:%S] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m+ [23-07-26 08:23:41] LD_PRELOAD=libvma.so [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.6.4-1 Release built on Aug 3 2022 00:00:00 [0m[0m VMA INFO: Cmd Line: date ++ [%y-%m-%d %H:%M:%S] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m+ [23-07-26 08:23:41] timeout --preserve-status --kill-after=5m 3m sockperf server -i 172.31.40.200 [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.6.4-1 Release built on Aug 3 2022 00:00:00 [0m[0m VMA INFO: Cmd Line: timeout --preserve-status --kill-after=5m 3m sockperf server -i 172.31.40.200 [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.6.4-1 Release built on Aug 3 2022 00:00:00 [0m[0m VMA INFO: Cmd Line: sockperf server -i 172.31.40.200 [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0msockperf: [2;35m== version #3.10-11.gitca16d475d6fa == [0m sockperf: [SERVER] listen on: [ 0] IP = 172.31.40.200 PORT = 11111 # UDP sockperf: Warmup stage (sending a few dummy messages)... sockperf: [tid 60923] using recvfrom() to block on socket(s) Terminated Client host: + [23-07-26 08:23:57] sleep 10 + [23-07-26 08:24:07] vma_test sockperf pp -i 172.31.40.200 -t 10 --msg-size=1472 + [23-07-26 08:24:07] rm -rf /tmp/vma.txt + [23-07-26 08:24:07] tee /tmp/vma.txt [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.6.4-1 Release built on Aug 3 2022 00:00:00 [0m[0m VMA INFO: Cmd Line: date ++ [%y-%m-%d %H:%M:%S] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m+ [23-07-26 08:24:07] LD_PRELOAD=libvma.so [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.6.4-1 Release built on Aug 3 2022 00:00:00 [0m[0m VMA INFO: Cmd Line: date ++ [%y-%m-%d %H:%M:%S] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m+ [23-07-26 08:24:07] timeout --preserve-status --kill-after=5m 3m sockperf pp -i 172.31.40.200 -t 10 --msg-size=1472 [0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.6.4-1 Release built on Aug 3 2022 00:00:00 [0m[0m VMA INFO: Cmd Line: timeout --preserve-status --kill-after=5m 3m sockperf pp -i 172.31.40.200 -t 10 --msg-size=1472 [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: VMA_VERSION: 9.6.4-1 Release built on Aug 3 2022 00:00:00 [0m[0m VMA INFO: Cmd Line: sockperf pp -i 172.31.40.200 -t 10 --msg-size=1472 [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[0m VMA INFO: Log Level INFO [VMA_TRACELEVEL] [0m[0m VMA INFO: --------------------------------------------------------------------------- [0m[2;35m VMA WARNING: ******************************************************************************************************* [0m[2;35m VMA WARNING: * IPoIB mode of interface 'mlx4_ib0' is "connected" ! [0m[2;35m VMA WARNING: * Please change it to datagram: "echo datagram > /sys/class/net/mlx4_ib0/mode" before loading your application with VMA library [0m[2;35m VMA WARNING: * VMA doesn't support IPoIB in connected mode. [0m[2;35m VMA WARNING: * Please refer to VMA Release Notes for more information [0m[2;35m VMA WARNING: ******************************************************************************************************* [0msockperf: [2;35m== version #3.10-11.gitca16d475d6fa == [0m sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s) [ 0] IP = 172.31.40.200 PORT = 11111 # UDP sockperf: Warmup stage (sending a few dummy messages)... sockperf: Starting test... sockperf: Test end (interrupted by timer) sockperf: Test ended sockperf: [Total Run] RunTime=10.000 sec; Warm up time=400 msec; SentMessages=1574119; ReceivedMessages=1574118 sockperf: ========= Printing statistics for Server No: 0 sockperf: [Valid Duration] RunTime=9.531 sec; SentMessages=1506246; ReceivedMessages=1506246 sockperf: [2;35m====> avg-latency=3.144 (std-dev=0.332, mean-ad=0.090, median-ad=0.033, siqr=0.025, cv=0.106, std-error=0.000, 99.0% ci=[3.143, 3.145])[0m sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0 sockperf: Summary: Latency is 3.144 usec sockperf: [2;35mTotal 1506246 observations[0m; each percentile contains 15062.46 observations sockperf: ---> <MAX> observation = 94.753 sockperf: ---> percentile 99.999 = 57.166 sockperf: ---> percentile 99.990 = 5.478 sockperf: ---> percentile 99.900 = 4.607 sockperf: ---> percentile 99.000 = 4.045 sockperf: ---> percentile 90.000 = 3.226 sockperf: ---> percentile 75.000 = 3.126 sockperf: ---> percentile 50.000 = 3.094 sockperf: ---> percentile 25.000 = 3.075 sockperf: ---> <MIN> observation = 2.986 + [23-07-26 08:24:21] result=0 + [23-07-26 08:24:21] '[' 0 -ne 0 ']' + [23-07-26 08:24:21] grep -qi -e ' error ' -e 'no messages were received' /tmp/vma.txt + [23-07-26 08:24:21] return 0 + [23-07-26 08:24:21] RQA_check_result -r 0 -t 'sockperf pingpong multicast' + [23-07-26 08:24:21] local test_pass=0 + [23-07-26 08:24:21] local test_skip=777 + [23-07-26 08:24:21] test 4 -gt 0 + [23-07-26 08:24:21] case $1 in + [23-07-26 08:24:21] local rc=0 + [23-07-26 08:24:21] shift + [23-07-26 08:24:21] shift + [23-07-26 08:24:21] test 2 -gt 0 + [23-07-26 08:24:21] case $1 in + [23-07-26 08:24:21] local 'msg=sockperf pingpong multicast' + [23-07-26 08:24:21] shift + [23-07-26 08:24:21] shift + [23-07-26 08:24:21] test 0 -gt 0 Additional info: