Description of problem: Minor update CI job failed on 'overcloud reboot' stage because ovn-controller is not Alive on at least one of the nodes (compute-1), see [0] below Link to the CI minor update job https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/network/view/networking-ovn/job/DFG-network-networking-ovn-update-16_director-rhel-virthost-3cont_2comp_2net-ipv4-geneve-composable/6/ Version-Release number of selected component (if applicable): Minor update from RHOS_TRUNK-16.0-RHEL-8-20191115.n.0 to RHOS_TRUNK-16.0-RHEL-8-20191126.n.2 How reproducible: Tried this scenario once and the issue occurred Note: I also tried update from RHOS_TRUNK-16.0-RHEL-8-20191115.n.0 to the brand new passed_phase1 puddle (RHOS_TRUNK-16.0-RHEL-8-20191202.n.1) but the job failed on undercloud update stage: see https://bugzilla.redhat.com/show_bug.cgi?id=1779165 Steps to Reproduce: 1. run minor update CI job: https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/network/view/networking-ovn/job/DFG-network-networking-ovn-update-16_director-rhel-virthost-3cont_2comp_2net-ipv4-geneve-composable/ specify the following parameter for the build: PRODUCT_BUILD: RHOS_TRUNK-16.0-RHEL-8-20191115.n.0 UPDATE_TO: RHOS_TRUNK-16.0-RHEL-8-20191126.n.2 DIRECTOR_UPDATE_BUILD: RHOS_TRUNK-16.0-RHEL-8-20191126.n.2 Actual results: Not all network agents are alive after overcloud reboot, CI job fails Expected results: Minor update performed successfully, all network agents are Alive after overcloud reboot Additional info: [0] from the job log TASK [Waiting for ovn-controller agent on compute node to come up] ************* task path: /home/rhos-ci/jenkins/workspace/DFG-network-networking-ovn-update-16_director-rhel-virthost-3cont_2comp_2net-ipv4-geneve-composable/infrared/plugins/tripleo-overcloud/overcloud_reboot.yml:430 Sunday 01 December 2019 06:50:38 +0000 (0:00:00.214) 0:12:40.928 ******* FAILED - RETRYING: Waiting for ovn-controller agent on compute node to come up (60 retries left). FAILED - RETRYING: Waiting for ovn-controller agent on compute node to come up (59 retries left). ..... FAILED - RETRYING: Waiting for ovn-controller agent on compute node to come up (2 retries left). FAILED - RETRYING: Waiting for ovn-controller agent on compute node to come up (1 retries left). fatal: [compute-1 -> 172.16.0.98]: FAILED! => { "attempts": 60, "changed": true, "cmd": "source /home/stack/overcloudrc\n openstack network agent list --host compute-1.redhat.local -f json | jq -r -c '.[] | select(.Binary | contains(\"ovn-controller\")) | .Alive'", "delta": "0:00:03.076233", "end": "2019-12-01 06:59:49.934447", "rc": 0, "start": "2019-12-01 06:59:46.858214" } STDOUT: false NO MORE HOSTS LEFT *****