Description of problem:
When deploying the metrics store, deployment hangs while trying to collect VM facts (fqdn).

Version-Release number of selected component (if applicable):
4.3.6, 4.3.7

How reproducible:
100%

Steps to Reproduce:
1. try and deploy metrics store on 4.3.7
2. wait for it to get stuck collecting facts
3.

Actual results:
Deployment fails and unable to deploy metrics store.

Expected results:
Metrics store successfully deploys.

Additional info:
This occurred in both 4.3.6 and also after upgrading to 4.3.7

with rhv-m 4.3.6, it failed with: 
ovirt-engine-metrics-1.3.4.1-1.el7ev.noarch
ansible-2.8.5-2.el7ae.noarch

after updating to 4.3.7, the following packages were installed:
ansible 2.9.4


~~~
TASK [/usr/share/ansible/roles/oVirt.metrics/roles/oVirt.origin-on-ovirt : Login to oVirt] *********************************
task path: /usr/share/ansible/roles/oVirt.metrics/roles/oVirt.origin-on-ovirt/tasks/create_openshift_bastion_vm.yml:9
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1580494688.11-260154395636840 `" && echo ansible-tmp-1580494688.11-260154395636840="` echo /root/.ansible/tmp/ansible-tmp-1580494688.11-260154395636840 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/cloud/ovirt/ovirt_auth.py
<localhost> PUT /root/.ansible/tmp/ansible-local-21049kad7jC/tmpaYZrsQ TO /root/.ansible/tmp/ansible-tmp-1580494688.11-260154395636840/AnsiballZ_ovirt_auth.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1580494688.11-260154395636840/ /root/.ansible/tmp/ansible-tmp-1580494688.11-260154395636840/AnsiballZ_ovirt_auth.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1580494688.11-260154395636840/AnsiballZ_ovirt_auth.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1580494688.11-260154395636840/ > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 
    "changed": false
}

TASK [/usr/share/ansible/roles/oVirt.metrics/roles/oVirt.origin-on-ovirt : Collect oVirt VM facts] *************************
task path: /usr/share/ansible/roles/oVirt.metrics/roles/oVirt.origin-on-ovirt/tasks/create_openshift_bastion_vm.yml:20
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1580494688.85-14579113196712 `" && echo ansible-tmp-1580494688.85-14579113196712="` echo /root/.ansible/tmp/ansible-tmp-1580494688.85-14579113196712 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/cloud/ovirt/_ovirt_vm_facts.py
<localhost> PUT /root/.ansible/tmp/ansible-local-21049kad7jC/tmpXip2zL TO /root/.ansible/tmp/ansible-tmp-1580494688.85-14579113196712/AnsiballZ__ovirt_vm_facts.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1580494688.85-14579113196712/ /root/.ansible/tmp/ansible-tmp-1580494688.85-14579113196712/AnsiballZ__ovirt_vm_facts.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1580494688.85-14579113196712/AnsiballZ__ovirt_vm_facts.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1580494688.85-14579113196712/ > /dev/null 2>&1 && sleep 0'
FAILED - RETRYING: Collect oVirt VM facts (5 retries left).Result was: {
    "attempts": 1, 
    "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 
    "changed": false, 
    "retries": 6
}

...

fatal: [localhost]: FAILED! => {
    "attempts": 5, 

~~~


This is what it's trying to do:


~~~
     20   - name: Collect oVirt VM facts
     21     ovirt_vm_facts:
     22       auth: '{{ ovirt_auth }}'
     23       pattern: 'name={{ openshift_ovirt_bastion_machine_name }} and cluster={{ ovirt_cluster_name }}'
     24       fetch_nested: true
     25       nested_attributes: ips
     26     no_log: true
     27     until: ansible_facts.ovirt_vms[0].fqdn is defined
     28     retries: 5
     29     delay: 60
~~~


When no_log is changed, you can see the he VM facts are listed (metrics-store-installer VM is also there and working fine), but there is no fqdn attribute.

I found something on an ovirt mailing list:
https://www.mail-archive.com/users@ovirt.org/msg59000.html

Which indicates this is an issue with Ansible 2.8, but we see this with both Ansible 2.8 and 2.9. 

We cannot downgrade to Ansible 2.7 because there ovirt-engine-metrics has a dependency for newer ansible.

I am unable to find a bugzilla so creating a new one to track the issue.

There is a reference to an upstream patch here: 
https://gerrit.ovirt.org/#/c/105813/


I tried that and it did not work, still hangs on collecting facts.