Bug 1828381

Created attachment 1682206 [details]
OC install logs

Description of problem:
Overcloud deploy fails on hybrid environment because nodes are unreachable. See 
logs attached:
fatal: [computesriov-0]: UNREACHABLE! => {
    "changed": false,
    "msg": "Data could not be sent to remote host \\"192.168.25.16\\". Make sure this host can be reached over ssh: OpenSSH_8.0p1, OpenSSL 1.1.1c FIPS  28 May 2019

However, nodes are reachable, both virtualized controllers or BM server computes:
(undercloud) [stack@undercloud-0 ~]$ openstack server list -c Name -c Networks -f value
controller-2 ctlplane=192.168.25.18
controller-1 ctlplane=192.168.25.10
controller-0 ctlplane=192.168.25.17
computesriov-0 ctlplane=192.168.25.16
computesriov-1 ctlplane=192.168.25.13
(undercloud) [stack@undercloud-0 ~]$ ssh heat-admin.25.16
Warning: Permanently added '192.168.25.16' (ECDSA) to the list of known hosts.
This system is not registered to Red Hat Insights. See https://cloud.redhat.com/
To register this system, run: insights-client --register

Last login: Mon Apr 27 11:32:02 2020 from 192.168.25.1
[heat-admin@computesriov-0 ~]$ logout
Connection to 192.168.25.16 closed.
(undercloud) [stack@undercloud-0 ~]$ ssh heat-admin.25.17
Warning: Permanently added '192.168.25.17' (ECDSA) to the list of known hosts.
This system is not registered to Red Hat Insights. See https://cloud.redhat.com/
To register this system, run: insights-client --register

Last login: Mon Apr 27 11:22:57 2020 from 192.168.25.150
[heat-admin@controller-0 ~]$ 



Workaround:
Right after the overcloud_deploy.sh script fails with this error, the script (or the infrared OC install command) is reexecuted and it passes.
But this is not valid for QE automated jobs.


Version-Release number of selected component (if applicable):
RHOS-16.1-RHEL-8-20200424.n.0
openstack-tripleo-common-11.3.3-0.20200423204446.86569f2.el8ost.noarch

How reproducible:
2/2

Steps to Reproduce:
1. Install OSP16.1 with infrared/CI job
2.
3.

Actual results:
OC deploy stage fails

Expected results:
OC deploy stage succeds

Additional info:
See failing job: https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/network/view/networking-ovn/job/DFG-network-networking-ovn-16.1_director-rhel-virthost-3cont_2comp-ipv4-vlan-sriov/2/