Bug 1868571

Unfortunately I couldn't gather engine logs as after failed deployment engine's VM went down:
alma04 ~]# virsh -r list --all
 Id   Name   State
--------------------

*** Bug 1869173 has been marked as a duplicate of this bug. ***

From broker.log:

ainThread::INFO::2020-08-13 10:07:37,570::monitor::63::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Finished loading submonitors
MainThread::ERROR::2020-08-13 10:07:37,570::config_file::60::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker.config.he_local::(load) Configuration file '/etc/ovirt-hosted-engine/hosted-engine.conf' not available [[Errno 13] Permission denied: '/etc/ovirt-hosted-engine/hosted-engine.conf']
MainThread::WARNING::2020-08-13 10:07:37,571::storage_broker::100::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(__init__) Can't connect vdsm storage: 'metadata_image_UUID can't be 'None' 
MainThread::ERROR::2020-08-13 10:07:37,571::broker::69::ovirt_hosted_engine_ha.broker.broker.Broker::(run) Failed initializing the broker: 'metadata_image_UUID can't be 'None'
MainThread::ERROR::2020-08-13 10:07:37,572::broker::71::ovirt_hosted_engine_ha.broker.broker.Broker::(run) Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/broker.py", line 64, in run
    self._storage_broker_instance = self._get_storage_broker()
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/broker.py", line 143, in _get_storage_broker
    return storage_broker.StorageBroker()
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 80, in __init__
    raise_on_none=True),
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/env/config.py", line 87, in get
    "'{0} can't be '{1}'".format(key, val)
ValueError: 'metadata_image_UUID can't be 'None'

QA still have latest bits from 2020-08-18. Waiting for the build to get consumed.

Deployed over NFS with these components:
vdsm-4.40.26-1.el8ev.x86_64
ovirt-hosted-engine-setup-2.4.6-1.el8ev.noarch
ovirt-hosted-engine-ha-2.4.4-1.el8ev.noarch
rhvm-appliance-4.4-20200722.0.el8ev.x86_64
Linux 4.18.0-193.14.3.el8_2.x86_64 #1 SMP Mon Jul 20 15:02:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux release 8.2 (Ootpa)

Deployment still failed with latest packages received from bob, which included an old ovirt-ansible-hosted-engine-setup-1.1.7-1.el8ev.noarch:

[ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 5, "changed": true, "cmd": ["hosted-engine", "--reinitialize-lockspace", "--force"], "delta": "0:00:00.444397", "end": "2020-08-23 21:05:08.806915", "msg": "non-zero return code", "rc": 1, "start": "2020-08-23 21:05:08.362518", "stderr": "Traceback (most recent call last):\n  File \"/usr/lib64/python3.6/runpy.py\", line 193, in _run_module_as_main\n    \"__main__\", mod_spec)\n  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/reinitialize_lockspace.py\", line 30, in <module>\n    ha_cli.reset_lockspace(force)\n  File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/client/client.py\", line 286, in reset_lockspace\n    stats = broker.get_stats_from_storage()\n  File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py\", line 148, in get_stats_from_storage\n    result = self._proxy.get_stats()\n  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1112, in __call__\n    return self.__send(self.__name, args)\n  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1452, in __request\n    verbose=self.__verbose\n  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1154, in request\n    return self.single_request(host, handler, request_body, verbose)\n  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1166, in single_request\n    http_conn = self.send_request(host, handler, request_body, verbose)\n  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1279, in send_request\n    self.send_content(connection, request_body)\n  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1309, in send_content\n    connection.endheaders(request_body)\n  File \"/usr/lib64/python3.6/http/client.py\", line 1249, in endheaders\n    self._send_output(message_body, encode_chunked=encode_chunked)\n  File \"/usr/lib64/python3.6/http/client.py\", line 1036, in _send_output\n    self.send(msg)\n  File \"/usr/lib64/python3.6/http/client.py\", line 974, in send\n    self.connect()\n  File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py\", line 74, in connect\n    self.sock.connect(base64.b16decode(self.host))\nFileNotFoundError: [Errno 2] No such file or directory", "stderr_lines": ["Traceback (most recent call last):", "  File \"/usr/lib64/python3.6/runpy.py\", line 193, in _run_module_as_main", "    \"__main__\", mod_spec)", "  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code", "    exec(code, run_globals)", "  File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/reinitialize_lockspace.py\", line 30, in <module>", "    ha_cli.reset_lockspace(force)", "  File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/client/client.py\", line 286, in reset_lockspace", "    stats = broker.get_stats_from_storage()", "  File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py\", line 148, in get_stats_from_storage", "    result = self._proxy.get_stats()", "  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1112, in __call__", "    return self.__send(self.__name, args)", "  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1452, in __request", "    verbose=self.__verbose", "  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1154, in request", "    return self.single_request(host, handler, request_body, verbose)", "  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1166, in single_request", "    http_conn = self.send_request(host, handler, request_body, verbose)", "  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1279, in send_request", "    self.send_content(connection, request_body)", "  File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1309, in send_content", "    connection.endheaders(request_body)", "  File \"/usr/lib64/python3.6/http/client.py\", line 1249, in endheaders", "    self._send_output(message_body, encode_chunked=encode_chunked)", "  File \"/usr/lib64/python3.6/http/client.py\", line 1036, in _send_output", "    self.send(msg)", "  File \"/usr/lib64/python3.6/http/client.py\", line 974, in send", "    self.connect()", "  File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py\", line 74, in connect", "    self.sock.connect(base64.b16decode(self.host))", "FileNotFoundError: [Errno 2] No such file or directory"], "stdout": "", "stdout_lines": []}
[ ERROR ] Failed to execute stage 'Closing up': Failed executing ansible-playbook

Hosted Engine successfully deployed
Worked with these components:
ovirt-ansible-hosted-engine-setup-1.1.8-1.el8ev.noarch
ovirt-hosted-engine-setup-2.4.6-1.el8ev.noarch
ovirt-hosted-engine-ha-2.4.4-1.el8ev.noarch
Red Hat Enterprise Linux release 8.2 (Ootpa)
Linux 4.18.0-193.14.3.el8_2.x86_64 #1 SMP Mon Jul 20 15:02:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

This bugzilla is included in oVirt 4.4.2 release, published on September 17th 2020.

Since the problem described in this bug report should be resolved in oVirt 4.4.2 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.