Bug 1817834
| Summary: | [RHEL-8.3/RDMA] update openmpi to latest upstream 4.0.3 | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Honggang LI <honli> |
| Component: | openmpi | Assignee: | Honggang LI <honli> |
| Status: | CLOSED ERRATA | QA Contact: | Afom T. Michael <tmichael> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 8.3 | CC: | rdma-dev-team, tmichael, zhangtao27 |
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
| Target Release: | 8.3 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openmpi-4.0.3-1.el8 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-11-04 01:37:28 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1789391 | ||
| Bug Blocks: | 1789863 | ||
|
Description
Honggang LI
2020-03-27 04:34:29 UTC
*** Bug 1788798 has been marked as a duplicate of this bug. *** Moving to verified as tests on RHEL-8.3 Beta passed.
[root@rdma-perf-03 ~]$ cat /etc/redhat-release
Red Hat Enterprise Linux release 8.3 Beta (Ootpa)
[root@rdma-perf-03 ~]$ uname -r
4.18.0-221.el8.x86_64
[root@rdma-perf-03 ~]$ rpm -qa | grep -E "rdma|ibverbs|ucx|openmpi"
rdma-core-devel-29.0-3.el8.x86_64
ucx-ib-1.8.0-1.el8.x86_64
librdmacm-utils-29.0-3.el8.x86_64
rdma-core-29.0-3.el8.x86_64
libibverbs-29.0-3.el8.x86_64
ucx-cma-1.8.0-1.el8.x86_64
librdmacm-29.0-3.el8.x86_64
ucx-1.8.0-1.el8.x86_64
libibverbs-utils-29.0-3.el8.x86_64
ucx-rdmacm-1.8.0-1.el8.x86_64
[root@rdma-perf-03 ~]$ rpm -qa | grep "openmpi"
mpitests-openmpi-5.6.2-1.el8.x86_64
openmpi-4.0.3-3.el8.x86_64
[root@rdma-perf-03 ~]$
[root@rdma-perf-03 ~]$ ibstatus
Infiniband device 'mlx5_0' port 1 status:
default gid: fe80:0000:0000:0000:248a:0703:00a3:196c
base lid: 0xa
sm lid: 0xd
state: 4: ACTIVE
phys state: 5: LinkUp
rate: 100 Gb/sec (4X EDR)
link_layer: InfiniBand
Infiniband device 'mlx5_1' port 1 status:
default gid: fe80:0000:0000:0000:268a:07ff:fea3:196d
base lid: 0x0
sm lid: 0x0
state: 4: ACTIVE
phys state: 5: LinkUp
rate: 100 Gb/sec (4X EDR)
link_layer: Ethernet
[root@rdma-perf-03 ~]$
Test commands were executed with arguments shown in below.
$ mpirun -hostfile /root/hfile_one_core -np 2 --allow-run-as-root --map-by node -mca btl_openib_warn_nonexistent_if 0 -mca btl_openib_if_include mlx5_0:1 -mca mtl '^psm2,psm,ofi' -mca btl_openib_allow_ib 1 -x UCX_NET_DEVICES=mlx5_ib0 -mca pml ucx -mca btl '^openib' mpitests-IMB-MPI1 PingPong -time 1.5
Test results for mpi/openmpi on rdma-perf-03:
4.18.0-221.el8.x86_64, rdma-core-29.0-3.el8, mlx5, ib0, & mlx5_0
Result | Status | Test
---------+--------+------------------------------------
PASS | 0 | openmpi IMB-MPI1 PingPong mpirun one_core
PASS | 0 | openmpi IMB-MPI1 PingPing mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Sendrecv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Exchange mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Bcast mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Allgather mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Allgatherv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Gather mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Gatherv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Scatter mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Scatterv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Alltoall mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Alltoallv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Reduce mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Reduce_scatter mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Allreduce mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Barrier mpirun one_core
PASS | 0 | openmpi IMB-IO S_Write_indv mpirun one_core
PASS | 0 | openmpi IMB-IO S_Read_indv mpirun one_core
PASS | 0 | openmpi IMB-IO S_Write_expl mpirun one_core
PASS | 0 | openmpi IMB-IO S_Read_expl mpirun one_core
PASS | 0 | openmpi IMB-IO P_Write_indv mpirun one_core
PASS | 0 | openmpi IMB-IO P_Read_indv mpirun one_core
PASS | 0 | openmpi IMB-IO P_Write_expl mpirun one_core
PASS | 0 | openmpi IMB-IO P_Read_expl mpirun one_core
PASS | 0 | openmpi IMB-IO P_Write_shared mpirun one_core
PASS | 0 | openmpi IMB-IO P_Read_shared mpirun one_core
PASS | 0 | openmpi IMB-IO P_Write_priv mpirun one_core
PASS | 0 | openmpi IMB-IO P_Read_priv mpirun one_core
PASS | 0 | openmpi IMB-IO C_Write_indv mpirun one_core
PASS | 0 | openmpi IMB-IO C_Read_indv mpirun one_core
PASS | 0 | openmpi IMB-IO C_Write_expl mpirun one_core
PASS | 0 | openmpi IMB-IO C_Read_expl mpirun one_core
PASS | 0 | openmpi IMB-IO C_Write_shared mpirun one_core
PASS | 0 | openmpi IMB-IO C_Read_shared mpirun one_core
PASS | 0 | openmpi IMB-EXT Window mpirun one_core
PASS | 0 | openmpi IMB-EXT Unidir_Put mpirun one_core
PASS | 0 | openmpi IMB-EXT Unidir_Get mpirun one_core
PASS | 0 | openmpi IMB-EXT Bidir_Get mpirun one_core
PASS | 0 | openmpi IMB-EXT Bidir_Put mpirun one_core
PASS | 0 | openmpi IMB-EXT Accumulate mpirun one_core
PASS | 0 | openmpi IMB-NBC Ibcast mpirun one_core
PASS | 0 | openmpi IMB-NBC Iallgather mpirun one_core
PASS | 0 | openmpi IMB-NBC Iallgatherv mpirun one_core
PASS | 0 | openmpi IMB-NBC Igather mpirun one_core
PASS | 0 | openmpi IMB-NBC Igatherv mpirun one_core
PASS | 0 | openmpi IMB-NBC Iscatter mpirun one_core
PASS | 0 | openmpi IMB-NBC Iscatterv mpirun one_core
PASS | 0 | openmpi IMB-NBC Ialltoall mpirun one_core
PASS | 0 | openmpi IMB-NBC Ialltoallv mpirun one_core
PASS | 0 | openmpi IMB-NBC Ireduce mpirun one_core
PASS | 0 | openmpi IMB-NBC Ireduce_scatter mpirun one_core
PASS | 0 | openmpi IMB-NBC Iallreduce mpirun one_core
PASS | 0 | openmpi IMB-NBC Ibarrier mpirun one_core
PASS | 0 | openmpi IMB-RMA Unidir_put mpirun one_core
PASS | 0 | openmpi IMB-RMA Unidir_get mpirun one_core
PASS | 0 | openmpi IMB-RMA Bidir_put mpirun one_core
PASS | 0 | openmpi IMB-RMA Bidir_get mpirun one_core
PASS | 0 | openmpi IMB-RMA One_put_all mpirun one_core
PASS | 0 | openmpi IMB-RMA One_get_all mpirun one_core
PASS | 0 | openmpi IMB-RMA All_put_all mpirun one_core
PASS | 0 | openmpi IMB-RMA All_get_all mpirun one_core
PASS | 0 | openmpi IMB-RMA Put_local mpirun one_core
PASS | 0 | openmpi IMB-RMA Put_all_local mpirun one_core
PASS | 0 | openmpi IMB-RMA Exchange_put mpirun one_core
PASS | 0 | openmpi IMB-RMA Exchange_get mpirun one_core
PASS | 0 | openmpi IMB-RMA Accumulate mpirun one_core
PASS | 0 | openmpi IMB-RMA Get_accumulate mpirun one_core
PASS | 0 | openmpi IMB-RMA Fetch_and_op mpirun one_core
PASS | 0 | openmpi IMB-RMA Compare_and_swap mpirun one_core
PASS | 0 | openmpi IMB-RMA Get_local mpirun one_core
PASS | 0 | openmpi IMB-RMA Get_all_local mpirun one_core
PASS | 0 | openmpi OSU acc_latency mpirun one_core
PASS | 0 | openmpi OSU allgather mpirun one_core
PASS | 0 | openmpi OSU allgatherv mpirun one_core
PASS | 0 | openmpi OSU allreduce mpirun one_core
PASS | 0 | openmpi OSU alltoall mpirun one_core
PASS | 0 | openmpi OSU alltoallv mpirun one_core
PASS | 0 | openmpi OSU barrier mpirun one_core
PASS | 0 | openmpi OSU bcast mpirun one_core
PASS | 0 | openmpi OSU bibw mpirun one_core
PASS | 0 | openmpi OSU bw mpirun one_core
PASS | 0 | openmpi OSU cas_latency mpirun one_core
PASS | 0 | openmpi OSU fop_latency mpirun one_core
PASS | 0 | openmpi OSU gather mpirun one_core
PASS | 0 | openmpi OSU gatherv mpirun one_core
PASS | 0 | openmpi OSU get_acc_latency mpirun one_core
PASS | 0 | openmpi OSU get_bw mpirun one_core
PASS | 0 | openmpi OSU get_latency mpirun one_core
PASS | 0 | openmpi OSU hello mpirun one_core
PASS | 0 | openmpi OSU iallgather mpirun one_core
PASS | 0 | openmpi OSU iallgatherv mpirun one_core
PASS | 0 | openmpi OSU iallreduce mpirun one_core
PASS | 0 | openmpi OSU ialltoall mpirun one_core
PASS | 0 | openmpi OSU ialltoallv mpirun one_core
PASS | 0 | openmpi OSU ialltoallw mpirun one_core
PASS | 0 | openmpi OSU ibarrier mpirun one_core
PASS | 0 | openmpi OSU ibcast mpirun one_core
PASS | 0 | openmpi OSU igather mpirun one_core
PASS | 0 | openmpi OSU igatherv mpirun one_core
PASS | 0 | openmpi OSU init mpirun one_core
PASS | 0 | openmpi OSU ireduce mpirun one_core
PASS | 0 | openmpi OSU iscatter mpirun one_core
PASS | 0 | openmpi OSU iscatterv mpirun one_core
PASS | 0 | openmpi OSU latency mpirun one_core
PASS | 0 | openmpi OSU mbw_mr mpirun one_core
PASS | 0 | openmpi OSU multi_lat mpirun one_core
PASS | 0 | openmpi OSU put_bibw mpirun one_core
PASS | 0 | openmpi OSU put_bw mpirun one_core
PASS | 0 | openmpi OSU put_latency mpirun one_core
PASS | 0 | openmpi OSU reduce mpirun one_core
PASS | 0 | openmpi OSU reduce_scatter mpirun one_core
PASS | 0 | openmpi OSU scatter mpirun one_core
PASS | 0 | openmpi OSU scatterv mpirun one_core
PASS | 0 | NON-ROOT IMB-MPI1 PingPong
Test results for mpi/openmpi on rdma-perf-03:
4.18.0-232.el8.x86_64, rdma-core-29.0-3.el8, mlx5, roce.45, & mlx5_1
Result | Status | Test
---------+--------+------------------------------------
PASS | 0 | openmpi IMB-MPI1 PingPong mpirun one_core
PASS | 0 | openmpi IMB-MPI1 PingPing mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Sendrecv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Exchange mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Bcast mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Allgather mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Allgatherv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Gather mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Gatherv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Scatter mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Scatterv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Alltoall mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Alltoallv mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Reduce mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Reduce_scatter mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Allreduce mpirun one_core
PASS | 0 | openmpi IMB-MPI1 Barrier mpirun one_core
PASS | 0 | openmpi IMB-IO S_Write_indv mpirun one_core
PASS | 0 | openmpi IMB-IO S_Read_indv mpirun one_core
PASS | 0 | openmpi IMB-IO S_Write_expl mpirun one_core
PASS | 0 | openmpi IMB-IO S_Read_expl mpirun one_core
PASS | 0 | openmpi IMB-IO P_Write_indv mpirun one_core
PASS | 0 | openmpi IMB-IO P_Read_indv mpirun one_core
PASS | 0 | openmpi IMB-IO P_Write_expl mpirun one_core
PASS | 0 | openmpi IMB-IO P_Read_expl mpirun one_core
PASS | 0 | openmpi IMB-IO P_Write_shared mpirun one_core
PASS | 0 | openmpi IMB-IO P_Read_shared mpirun one_core
PASS | 0 | openmpi IMB-IO P_Write_priv mpirun one_core
PASS | 0 | openmpi IMB-IO P_Read_priv mpirun one_core
PASS | 0 | openmpi IMB-IO C_Write_indv mpirun one_core
PASS | 0 | openmpi IMB-IO C_Read_indv mpirun one_core
PASS | 0 | openmpi IMB-IO C_Write_expl mpirun one_core
PASS | 0 | openmpi IMB-IO C_Read_expl mpirun one_core
PASS | 0 | openmpi IMB-IO C_Write_shared mpirun one_core
PASS | 0 | openmpi IMB-IO C_Read_shared mpirun one_core
PASS | 0 | openmpi IMB-EXT Window mpirun one_core
FAIL | 139 | openmpi IMB-EXT Unidir_Put mpirun one_core
PASS | 0 | openmpi IMB-EXT Unidir_Get mpirun one_core
PASS | 0 | openmpi IMB-EXT Bidir_Get mpirun one_core
PASS | 0 | openmpi IMB-EXT Bidir_Put mpirun one_core
PASS | 0 | openmpi IMB-EXT Accumulate mpirun one_core
PASS | 0 | openmpi IMB-NBC Ibcast mpirun one_core
PASS | 0 | openmpi IMB-NBC Iallgather mpirun one_core
PASS | 0 | openmpi IMB-NBC Iallgatherv mpirun one_core
PASS | 0 | openmpi IMB-NBC Igather mpirun one_core
PASS | 0 | openmpi IMB-NBC Igatherv mpirun one_core
PASS | 0 | openmpi IMB-NBC Iscatter mpirun one_core
PASS | 0 | openmpi IMB-NBC Iscatterv mpirun one_core
PASS | 0 | openmpi IMB-NBC Ialltoall mpirun one_core
PASS | 0 | openmpi IMB-NBC Ialltoallv mpirun one_core
PASS | 0 | openmpi IMB-NBC Ireduce mpirun one_core
PASS | 0 | openmpi IMB-NBC Ireduce_scatter mpirun one_core
PASS | 0 | openmpi IMB-NBC Iallreduce mpirun one_core
PASS | 0 | openmpi IMB-NBC Ibarrier mpirun one_core
PASS | 0 | openmpi IMB-RMA Unidir_put mpirun one_core
FAIL | 139 | openmpi IMB-RMA Unidir_get mpirun one_core
PASS | 0 | openmpi IMB-RMA Bidir_put mpirun one_core
PASS | 0 | openmpi IMB-RMA Bidir_get mpirun one_core
PASS | 0 | openmpi IMB-RMA One_put_all mpirun one_core
PASS | 0 | openmpi IMB-RMA One_get_all mpirun one_core
PASS | 0 | openmpi IMB-RMA All_put_all mpirun one_core
PASS | 0 | openmpi IMB-RMA All_get_all mpirun one_core
PASS | 0 | openmpi IMB-RMA Put_local mpirun one_core
PASS | 0 | openmpi IMB-RMA Put_all_local mpirun one_core
PASS | 0 | openmpi IMB-RMA Exchange_put mpirun one_core
PASS | 0 | openmpi IMB-RMA Exchange_get mpirun one_core
FAIL | 139 | openmpi IMB-RMA Accumulate mpirun one_core
PASS | 0 | openmpi IMB-RMA Get_accumulate mpirun one_core
PASS | 0 | openmpi IMB-RMA Fetch_and_op mpirun one_core
PASS | 0 | openmpi IMB-RMA Compare_and_swap mpirun one_core
PASS | 0 | openmpi IMB-RMA Get_local mpirun one_core
PASS | 0 | openmpi IMB-RMA Get_all_local mpirun one_core
PASS | 0 | openmpi OSU acc_latency mpirun one_core
PASS | 0 | openmpi OSU allgather mpirun one_core
PASS | 0 | openmpi OSU allgatherv mpirun one_core
PASS | 0 | openmpi OSU allreduce mpirun one_core
PASS | 0 | openmpi OSU alltoall mpirun one_core
PASS | 0 | openmpi OSU alltoallv mpirun one_core
PASS | 0 | openmpi OSU barrier mpirun one_core
PASS | 0 | openmpi OSU bcast mpirun one_core
PASS | 0 | openmpi OSU bibw mpirun one_core
PASS | 0 | openmpi OSU bw mpirun one_core
PASS | 0 | openmpi OSU cas_latency mpirun one_core
PASS | 0 | openmpi OSU fop_latency mpirun one_core
PASS | 0 | openmpi OSU gather mpirun one_core
PASS | 0 | openmpi OSU gatherv mpirun one_core
PASS | 0 | openmpi OSU get_acc_latency mpirun one_core
PASS | 0 | openmpi OSU get_bw mpirun one_core
PASS | 0 | openmpi OSU get_latency mpirun one_core
PASS | 0 | openmpi OSU hello mpirun one_core
PASS | 0 | openmpi OSU iallgather mpirun one_core
PASS | 0 | openmpi OSU iallgatherv mpirun one_core
PASS | 0 | openmpi OSU iallreduce mpirun one_core
PASS | 0 | openmpi OSU ialltoall mpirun one_core
PASS | 0 | openmpi OSU ialltoallv mpirun one_core
PASS | 0 | openmpi OSU ialltoallw mpirun one_core
PASS | 0 | openmpi OSU ibarrier mpirun one_core
PASS | 0 | openmpi OSU ibcast mpirun one_core
PASS | 0 | openmpi OSU igather mpirun one_core
PASS | 0 | openmpi OSU igatherv mpirun one_core
PASS | 0 | openmpi OSU init mpirun one_core
PASS | 0 | openmpi OSU ireduce mpirun one_core
PASS | 0 | openmpi OSU iscatter mpirun one_core
PASS | 0 | openmpi OSU iscatterv mpirun one_core
PASS | 0 | openmpi OSU latency mpirun one_core
PASS | 0 | openmpi OSU mbw_mr mpirun one_core
PASS | 0 | openmpi OSU multi_lat mpirun one_core
PASS | 0 | openmpi OSU put_bibw mpirun one_core
FAIL | 139 | openmpi OSU put_bw mpirun one_core
PASS | 0 | openmpi OSU put_latency mpirun one_core
PASS | 0 | openmpi OSU reduce mpirun one_core
PASS | 0 | openmpi OSU reduce_scatter mpirun one_core
PASS | 0 | openmpi OSU scatter mpirun one_core
PASS | 0 | openmpi OSU scatterv mpirun one_core
PASS | 0 | NON-ROOT IMB-MPI1 PingPong
Manually running the tests marked as 'FAIL' passed when executed from the command line. For "mpitests-osu put_bw", argument "btl" was adjusted with "-mca btl self" to pass.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (rdma-core bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2020:4456 |