Bug 2212461

Summary: [RHEL-9.3] Update mvapich2 package
Product: Red Hat Enterprise Linux 9 Reporter: Kamal Heib <kheib>
Component: mvapich2Assignee: Kamal Heib <kheib>
Status: VERIFIED --- QA Contact: Afom T. Michael <tmichael>
Severity: high Docs Contact:
Priority: unspecified    
Version: 9.3CC: hwkernel-mgr, rdma-dev-team, tmichael
Target Milestone: rcKeywords: Triaged
Target Release: 9.3   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: mvapich2-2.3.7.1-1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Component Upgrade
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kamal Heib 2023-06-05 16:20:35 UTC
Description of problem:
Update mvapich2 for rhel-9.3 release

Version-Release number of selected component (if applicable):
2.3.7

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Afom T. Michael 2023-06-11 00:21:16 UTC
Marking as "Verified: Tested" since test with mvapich2-2.3.7.1-1.el9 passed.

[...snip...]
Installed:
  mpitests-mvapich2-5.8-1.el9.x86_64        mvapich2-2.3.7.1-1.el9.x86_64      
  mvapich2-devel-2.3.7.1-1.el9.x86_64       mvapich2-psm2-2.3.7.1-1.el9.x86_64 
  mvapich2-psm2-devel-2.3.7.1-1.el9.x86_64
[...snip...]


mpi/mvapich2 test results on rdma-dev-19/rdma-dev-20 & Beaker job J:7960291:
5.14.0-324.el9.x86_64, rdma-core-46.0-1.el9, mlx5, roce.45, ConnectX-4 Lx & mlx5_bond_0
    Result | Status | Test
  ---------+--------+------------------------------------
      PASS |      0 | mvapich2 IMB-MPI1 PingPong mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 PingPing mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Sendrecv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Exchange mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Bcast mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allgather mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allgatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Gather mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Gatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Scatter mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Scatterv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Alltoall mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Alltoallv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Reduce mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Reduce_scatter mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allreduce mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Barrier mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Write_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Read_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Write_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Read_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_priv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_priv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Window mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Unidir_Put mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Unidir_Get mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Bidir_Get mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Bidir_Put mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Accumulate mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ibcast mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iallgather mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iallgatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Igather mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Igatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iscatter mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iscatterv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ialltoall mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ialltoallv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ireduce mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ireduce_scatter mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iallreduce mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ibarrier mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Unidir_put mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Unidir_get mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Bidir_put mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Bidir_get mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA One_put_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA One_get_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA All_put_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA All_get_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Put_local mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Put_all_local mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Exchange_put mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Exchange_get mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Accumulate mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Get_accumulate mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Fetch_and_op mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Compare_and_swap mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Get_local mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Get_all_local mpirun one_core
      PASS |      0 | mvapich2 OSU acc_latency mpirun one_core
      PASS |      0 | mvapich2 OSU allgather mpirun one_core
      PASS |      0 | mvapich2 OSU allgatherv mpirun one_core
      PASS |      0 | mvapich2 OSU allreduce mpirun one_core
      PASS |      0 | mvapich2 OSU alltoall mpirun one_core
      PASS |      0 | mvapich2 OSU alltoallv mpirun one_core
      PASS |      0 | mvapich2 OSU barrier mpirun one_core
      PASS |      0 | mvapich2 OSU bcast mpirun one_core
      PASS |      0 | mvapich2 OSU bibw mpirun one_core
      PASS |      0 | mvapich2 OSU bw mpirun one_core
      PASS |      0 | mvapich2 OSU cas_latency mpirun one_core
      PASS |      0 | mvapich2 OSU fop_latency mpirun one_core
      PASS |      0 | mvapich2 OSU gather mpirun one_core
      PASS |      0 | mvapich2 OSU gatherv mpirun one_core
      PASS |      0 | mvapich2 OSU get_acc_latency mpirun one_core
      PASS |      0 | mvapich2 OSU get_bw mpirun one_core
      PASS |      0 | mvapich2 OSU get_latency mpirun one_core
      PASS |      0 | mvapich2 OSU hello mpirun one_core
      PASS |      0 | mvapich2 OSU iallgather mpirun one_core
      PASS |      0 | mvapich2 OSU iallgatherv mpirun one_core
      PASS |      0 | mvapich2 OSU iallreduce mpirun one_core
      PASS |      0 | mvapich2 OSU ialltoall mpirun one_core
      PASS |      0 | mvapich2 OSU ialltoallv mpirun one_core
      PASS |      0 | mvapich2 OSU ialltoallw mpirun one_core
      PASS |      0 | mvapich2 OSU ibarrier mpirun one_core
      PASS |      0 | mvapich2 OSU ibcast mpirun one_core
      PASS |      0 | mvapich2 OSU igather mpirun one_core
      PASS |      0 | mvapich2 OSU igatherv mpirun one_core
      PASS |      0 | mvapich2 OSU init mpirun one_core
      PASS |      0 | mvapich2 OSU ireduce mpirun one_core
      PASS |      0 | mvapich2 OSU iscatter mpirun one_core
      PASS |      0 | mvapich2 OSU iscatterv mpirun one_core
      PASS |      0 | mvapich2 OSU latency mpirun one_core
      PASS |      0 | mvapich2 OSU latency_mp mpirun one_core
      PASS |      0 | mvapich2 OSU mbw_mr mpirun one_core
      PASS |      0 | mvapich2 OSU multi_lat mpirun one_core
      PASS |      0 | mvapich2 OSU put_bibw mpirun one_core
      PASS |      0 | mvapich2 OSU put_bw mpirun one_core
      PASS |      0 | mvapich2 OSU put_latency mpirun one_core
      PASS |      0 | mvapich2 OSU reduce mpirun one_core
      PASS |      0 | mvapich2 OSU reduce_scatter mpirun one_core
      PASS |      0 | mvapich2 OSU scatter mpirun one_core
      PASS |      0 | mvapich2 OSU scatterv mpirun one_core
      PASS |      0 | mvapich2 OSU acc_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU allgather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU allgatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU allreduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU alltoall mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU alltoallv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU barrier mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU bcast mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU bibw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU bw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU cas_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU fop_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU gather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU gatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU get_acc_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU get_bw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU get_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU hello mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iallgather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iallgatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iallreduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ialltoall mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ialltoallv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ialltoallw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ibarrier mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ibcast mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU igather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU igatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU init mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ireduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iscatter mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iscatterv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU latency_mp mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU mbw_mr mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU multi_lat mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU put_bibw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU put_bw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU put_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU reduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU reduce_scatter mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU scatter mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU scatterv mpirun_rsh one_core
Checking for failures and known issues:
  no test failures

Comment 4 Afom T. Michael 2023-06-29 05:16:23 UTC
RHEL-9.3 has mvapich2-2.3.7.1-1.el9.x86_64 & tests have passed. Marking this as 'verified'.

Installed:
  mpitests-mvapich2-5.8-1.el9.x86_64        mvapich2-2.3.7.1-1.el9.x86_64

mpi-mvapich2 test results on rdma-perf-02/rdma-perf-03 & Beaker job J:8017687:
5.14.0-332.el9.x86_64, rdma-core-46.0-1.el9, mlx5, ib0, ConnectX-5 & mlx5_0
    Result | Status | Test
  ---------+--------+------------------------------------
      FAIL |    255 | mvapich2 IMB-MPI1 PingPong mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 PingPing mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Sendrecv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Exchange mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Bcast mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allgather mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allgatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Gather mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Gatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Scatter mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Scatterv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Alltoall mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Alltoallv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Reduce mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Reduce_scatter mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allreduce mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Barrier mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Write_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Read_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Write_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Read_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_priv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_priv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Window mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Unidir_Put mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Unidir_Get mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Bidir_Get mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Bidir_Put mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Accumulate mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ibcast mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iallgather mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iallgatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Igather mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Igatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iscatter mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iscatterv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ialltoall mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ialltoallv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ireduce mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ireduce_scatter mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iallreduce mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ibarrier mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Unidir_put mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Unidir_get mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Bidir_put mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Bidir_get mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA One_put_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA One_get_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA All_put_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA All_get_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Put_local mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Put_all_local mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Exchange_put mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Exchange_get mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Accumulate mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Get_accumulate mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Fetch_and_op mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Compare_and_swap mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Get_local mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Get_all_local mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 PingPong mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 PingPing mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Sendrecv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Exchange mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Bcast mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allgather mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allgatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Gather mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Gatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Scatter mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Scatterv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Alltoall mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Alltoallv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Reduce mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Reduce_scatter mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allreduce mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-MPI1 Barrier mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO S_Write_indv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO S_Read_indv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO S_Write_expl mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO S_Read_expl mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_indv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_indv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_expl mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_expl mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_shared mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_shared mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_priv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_priv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_indv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_indv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_expl mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_expl mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_shared mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_shared mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-EXT Window mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-EXT Unidir_Put mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-EXT Unidir_Get mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-EXT Bidir_Get mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-EXT Bidir_Put mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-EXT Accumulate mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Ibcast mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Iallgather mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Iallgatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Igather mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Igatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Iscatter mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Iscatterv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Ialltoall mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Ialltoallv mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Ireduce mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Ireduce_scatter mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Iallreduce mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-NBC Ibarrier mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Unidir_put mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Unidir_get mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Bidir_put mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Bidir_get mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA One_put_all mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA One_get_all mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA All_put_all mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA All_get_all mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Put_local mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Put_all_local mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Exchange_put mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Exchange_get mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Accumulate mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Get_accumulate mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Fetch_and_op mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Compare_and_swap mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Get_local mpirun_rsh one_core
      PASS |      0 | mvapich2 IMB-RMA Get_all_local mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU acc_latency mpirun one_core
      PASS |      0 | mvapich2 OSU allgather mpirun one_core
      PASS |      0 | mvapich2 OSU allgatherv mpirun one_core
      PASS |      0 | mvapich2 OSU allreduce mpirun one_core
      PASS |      0 | mvapich2 OSU alltoall mpirun one_core
      PASS |      0 | mvapich2 OSU alltoallv mpirun one_core
      PASS |      0 | mvapich2 OSU barrier mpirun one_core
      PASS |      0 | mvapich2 OSU bcast mpirun one_core
      PASS |      0 | mvapich2 OSU bibw mpirun one_core
      PASS |      0 | mvapich2 OSU bw mpirun one_core
      PASS |      0 | mvapich2 OSU cas_latency mpirun one_core
      PASS |      0 | mvapich2 OSU fop_latency mpirun one_core
      PASS |      0 | mvapich2 OSU gather mpirun one_core
      PASS |      0 | mvapich2 OSU gatherv mpirun one_core
      PASS |      0 | mvapich2 OSU get_acc_latency mpirun one_core
      PASS |      0 | mvapich2 OSU get_bw mpirun one_core
      PASS |      0 | mvapich2 OSU get_latency mpirun one_core
      PASS |      0 | mvapich2 OSU hello mpirun one_core
      PASS |      0 | mvapich2 OSU iallgather mpirun one_core
      PASS |      0 | mvapich2 OSU iallgatherv mpirun one_core
      PASS |      0 | mvapich2 OSU iallreduce mpirun one_core
      PASS |      0 | mvapich2 OSU ialltoall mpirun one_core
      PASS |      0 | mvapich2 OSU ialltoallv mpirun one_core
      PASS |      0 | mvapich2 OSU ialltoallw mpirun one_core
      PASS |      0 | mvapich2 OSU ibarrier mpirun one_core
      PASS |      0 | mvapich2 OSU ibcast mpirun one_core
      PASS |      0 | mvapich2 OSU igather mpirun one_core
      PASS |      0 | mvapich2 OSU igatherv mpirun one_core
      PASS |      0 | mvapich2 OSU init mpirun one_core
      PASS |      0 | mvapich2 OSU ireduce mpirun one_core
      PASS |      0 | mvapich2 OSU iscatter mpirun one_core
      PASS |      0 | mvapich2 OSU iscatterv mpirun one_core
      PASS |      0 | mvapich2 OSU latency mpirun one_core
      PASS |      0 | mvapich2 OSU latency_mp mpirun one_core
      PASS |      0 | mvapich2 OSU mbw_mr mpirun one_core
      PASS |      0 | mvapich2 OSU multi_lat mpirun one_core
      PASS |      0 | mvapich2 OSU put_bibw mpirun one_core
      PASS |      0 | mvapich2 OSU put_bw mpirun one_core
      PASS |      0 | mvapich2 OSU put_latency mpirun one_core
      PASS |      0 | mvapich2 OSU reduce mpirun one_core
      PASS |      0 | mvapich2 OSU reduce_scatter mpirun one_core
      PASS |      0 | mvapich2 OSU scatter mpirun one_core
      PASS |      0 | mvapich2 OSU scatterv mpirun one_core
      PASS |      0 | mvapich2 OSU acc_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU allgather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU allgatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU allreduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU alltoall mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU alltoallv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU barrier mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU bcast mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU bibw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU bw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU cas_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU fop_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU gather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU gatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU get_acc_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU get_bw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU get_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU hello mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iallgather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iallgatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iallreduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ialltoall mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ialltoallv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ialltoallw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ibarrier mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ibcast mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU igather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU igatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU init mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ireduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iscatter mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iscatterv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU latency_mp mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU mbw_mr mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU multi_lat mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU put_bibw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU put_bw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU put_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU reduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU reduce_scatter mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU scatter mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU scatterv mpirun_rsh one_core
Checking for failures and known issues:
  mvapich2 IMB-MPI1 PingPong mpirun one_core is NOT a known issue on any environment - consider filing a BZ

mpi-mvapich2 test results on rdma-perf-02/rdma-perf-03 & Beaker job J:8017687:
5.14.0-332.el9.x86_64, rdma-core-46.0-1.el9, mlx5, roce.45, ConnectX-5 & mlx5_1
    Result | Status | Test
  ---------+--------+------------------------------------
      PASS |      0 | mvapich2 IMB-MPI1 PingPong mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 PingPing mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Sendrecv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Exchange mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Bcast mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allgather mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allgatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Gather mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Gatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Scatter mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Scatterv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Alltoall mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Alltoallv mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Reduce mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Reduce_scatter mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Allreduce mpirun one_core
      PASS |      0 | mvapich2 IMB-MPI1 Barrier mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Write_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Read_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Write_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO S_Read_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Write_priv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO P_Read_priv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_indv mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_expl mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Write_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-IO C_Read_shared mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Window mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Unidir_Put mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Unidir_Get mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Bidir_Get mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Bidir_Put mpirun one_core
      PASS |      0 | mvapich2 IMB-EXT Accumulate mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ibcast mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iallgather mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iallgatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Igather mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Igatherv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iscatter mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iscatterv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ialltoall mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ialltoallv mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ireduce mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ireduce_scatter mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Iallreduce mpirun one_core
      PASS |      0 | mvapich2 IMB-NBC Ibarrier mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Unidir_put mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Unidir_get mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Bidir_put mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Bidir_get mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA One_put_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA One_get_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA All_put_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA All_get_all mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Put_local mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Put_all_local mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Exchange_put mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Exchange_get mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Accumulate mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Get_accumulate mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Fetch_and_op mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Compare_and_swap mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Get_local mpirun one_core
      PASS |      0 | mvapich2 IMB-RMA Get_all_local mpirun one_core
      PASS |      0 | mvapich2 OSU acc_latency mpirun one_core
      PASS |      0 | mvapich2 OSU allgather mpirun one_core
      PASS |      0 | mvapich2 OSU allgatherv mpirun one_core
      PASS |      0 | mvapich2 OSU allreduce mpirun one_core
      PASS |      0 | mvapich2 OSU alltoall mpirun one_core
      PASS |      0 | mvapich2 OSU alltoallv mpirun one_core
      PASS |      0 | mvapich2 OSU barrier mpirun one_core
      PASS |      0 | mvapich2 OSU bcast mpirun one_core
      PASS |      0 | mvapich2 OSU bibw mpirun one_core
      PASS |      0 | mvapich2 OSU bw mpirun one_core
      PASS |      0 | mvapich2 OSU cas_latency mpirun one_core
      PASS |      0 | mvapich2 OSU fop_latency mpirun one_core
      PASS |      0 | mvapich2 OSU gather mpirun one_core
      PASS |      0 | mvapich2 OSU gatherv mpirun one_core
      PASS |      0 | mvapich2 OSU get_acc_latency mpirun one_core
      PASS |      0 | mvapich2 OSU get_bw mpirun one_core
      PASS |      0 | mvapich2 OSU get_latency mpirun one_core
      PASS |      0 | mvapich2 OSU hello mpirun one_core
      PASS |      0 | mvapich2 OSU iallgather mpirun one_core
      PASS |      0 | mvapich2 OSU iallgatherv mpirun one_core
      PASS |      0 | mvapich2 OSU iallreduce mpirun one_core
      PASS |      0 | mvapich2 OSU ialltoall mpirun one_core
      PASS |      0 | mvapich2 OSU ialltoallv mpirun one_core
      PASS |      0 | mvapich2 OSU ialltoallw mpirun one_core
      PASS |      0 | mvapich2 OSU ibarrier mpirun one_core
      PASS |      0 | mvapich2 OSU ibcast mpirun one_core
      PASS |      0 | mvapich2 OSU igather mpirun one_core
      PASS |      0 | mvapich2 OSU igatherv mpirun one_core
      PASS |      0 | mvapich2 OSU init mpirun one_core
      PASS |      0 | mvapich2 OSU ireduce mpirun one_core
      PASS |      0 | mvapich2 OSU iscatter mpirun one_core
      PASS |      0 | mvapich2 OSU iscatterv mpirun one_core
      PASS |      0 | mvapich2 OSU latency mpirun one_core
      PASS |      0 | mvapich2 OSU latency_mp mpirun one_core
      PASS |      0 | mvapich2 OSU mbw_mr mpirun one_core
      PASS |      0 | mvapich2 OSU multi_lat mpirun one_core
      PASS |      0 | mvapich2 OSU put_bibw mpirun one_core
      PASS |      0 | mvapich2 OSU put_bw mpirun one_core
      PASS |      0 | mvapich2 OSU put_latency mpirun one_core
      PASS |      0 | mvapich2 OSU reduce mpirun one_core
      PASS |      0 | mvapich2 OSU reduce_scatter mpirun one_core
      PASS |      0 | mvapich2 OSU scatter mpirun one_core
      PASS |      0 | mvapich2 OSU scatterv mpirun one_core
      PASS |      0 | mvapich2 OSU acc_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU allgather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU allgatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU allreduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU alltoall mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU alltoallv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU barrier mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU bcast mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU bibw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU bw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU cas_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU fop_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU gather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU gatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU get_acc_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU get_bw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU get_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU hello mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iallgather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iallgatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iallreduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ialltoall mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ialltoallv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ialltoallw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ibarrier mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ibcast mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU igather mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU igatherv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU init mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU ireduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iscatter mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU iscatterv mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU latency_mp mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU mbw_mr mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU multi_lat mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU put_bibw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU put_bw mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU put_latency mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU reduce mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU reduce_scatter mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU scatter mpirun_rsh one_core
      PASS |      0 | mvapich2 OSU scatterv mpirun_rsh one_core
Checking for failures and known issues:
  no test failures