Description of problem: ------------------------ When initiating the hosted-engine deployment from cockpit, there is an error encountered, which abandons the hosted-engine deployment wizard Version-Release number of selected component (if applicable): ------------------------------------------------------------- RHVH-4.4.1 (RHVH-4.4-20200722.0-RHVH-x86_64-dvd1.iso) How reproducible: ------------------ Always Steps to Reproduce: ------------------- 1. Lauch hosted-engine deployment from cockpit Actual results: --------------- Error thrown and no hosted-engine wizard seen Expected results: ----------------- Hosted engine wizard should be launched successfully Additional info: ----------------- From initial look, this problem is because of changes with ansible-2.9.11, that was included with the latest RHVH 4.4.1 ISO
Error seen in the cockpit: No valid network interface has been found If you are using Bonds or VLANs Use the following naming conventions: - VLAN interfaces: physical_device.VLAN_ID (for example, eth0.23, eth1.128, enp3s0.50) - Bond interfaces: bond*number* (for example, bond0, bond1) - VLANs on bond interfaces: bond*number*.VLAN_ID (for example, bond0.50, bond1.128) * Supported bond modes: active-backup, balance-xor, broadcast, 802.3ad * Networking teaming is not supported and will cause errors
2020-07-22 08:06:18,238+0000 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_result": { "changed": true, "msg": "All items completed", "results": [ { "_ansible_item_label": { "ansible_facts": { "host_net": [ "enp3s0f0", "enp129s0f0" ] }, "changed": false, "failed": false }, "_ansible_no_log": false, "ansible_loop_var": "nic", "changed": true, "cmd": "set -euo pipefail && nmcli -g GENERAL.TYPE device show {'changed': False, 'ansible_facts': {'host_net': ['enp3s0f0', 'enp129s0f0']}, 'failed': False}", "delta": "0:00:00.058200", "end": "2020-07-22 08:06:17.781668", "failed": true, "invocation": { "module_args": { "_raw_params": "set -euo pipefail && nmcli -g GENERAL.TYPE device show {'changed': False, 'ansible_facts': {'host_net': ['enp3s0f0', 'enp129s0f0']}, 'failed': False}" , "_uses_shell": true, "argv": null, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "stdin_add_newline": true, "strip_empty_ends": true, "warn": true } }, "msg": "non-zero return code", "nic": { "ansible_facts": { "host_net": [ "enp3s0f0", "enp129s0f0" ] }, "changed": false, "failed": false }, "rc": 10, "start": "2020-07-22 08:06:17.723468", "stderr": "Error: Device '{changed:' not found.", "stderr_lines": [ "Error: Device '{changed:' not found." ], "stdout": "", "stdout_lines": [] } ] }, "ansible_task": "Collect interface types", "ansible_type": "task", "status": "FAILED", "task_duration": 1 }
When running the same command, I see the problem: [root@ ~]# set -euo pipefail && nmcli -g GENERAL.TYPE device show {'changed': False, 'ansible_facts': {'host_net': ['enp3s0f0', 'enp129s0f0']}, 'failed': False} Error: Device '{changed:' not found.
Created attachment 1702040 [details] get_network_interfaces.log
Assaf, do you know if this is more likly to be a bug in ansible or in virt-ansible-hosted-engine-setup ?
I think this is because of latest changes in ansible-2.9.11 fix. nmcli - Add compatibility for new networkmanager library (https://github.com/ansible/ansible/pull/65726). Failed ansible task is: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/tasks/filter_team_devices.yml#L3
Ansible fixed the behavior of set_fact module according to the documentation, but issue is our code: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/tasks/pre_checks/001_validate_network_interfaces.yml#L71 It doesn't make sense to register result of set_fact execution to the variable with the same name as the fact. This worked in previous releases only by error. So just removing line with register should fix the issue, here is sample playbook with confirmation: $ cat test_facts.yml --- - name: test facts hosts: localhost connection: local tasks: - name: Set fact aaa set_fact: aaa: 'value1 value2' - name: Display fact aaa debug: var: aaa - name: Set fact bbb and register it set_fact: bbb: 'value1 value2' register: bbb - name: Display fact bbb debug: var: bbb Here is the execution in Ansible 2.9.10: $ ansible-playbook test_facts.yml [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' PLAY [oVirt infra] ********************************************************************************************************************************************************************** TASK [Gathering Facts] ****************************************************************************************************************************************************************** ok: [localhost] TASK [Set fact aaa] ********************************************************************************************************************************************************************* ok: [localhost] TASK [Display fact aaa] ***************************************************************************************************************************************************************** ok: [localhost] => { "aaa": "value1 value2" } TASK [Set fact bbb and register it] ***************************************************************************************************************************************************** ok: [localhost] TASK [Display fact bbb] ***************************************************************************************************************************************************************** ok: [localhost] => { "bbb": "value1 value2" } PLAY RECAP ****************************************************************************************************************************************************************************** localhost : ok=5 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 Here is the execution under Ansible 2.9.11: $ ansible-playbook test_facts.yml [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' PLAY [oVirt infra] ********************************************************************************************************************************************************************** TASK [Gathering Facts] ****************************************************************************************************************************************************************** ok: [localhost] TASK [Set fact aaa] ********************************************************************************************************************************************************************* ok: [localhost] TASK [Display fact aaa] ***************************************************************************************************************************************************************** ok: [localhost] => { "aaa": "value1 value2" } TASK [Set fact bbb and register it] ***************************************************************************************************************************************************** ok: [localhost] TASK [Display fact bbb] ***************************************************************************************************************************************************************** ok: [localhost] => { "bbb": { "ansible_facts": { "bbb": "value1 value2" }, "changed": false, "failed": false } } PLAY RECAP ****************************************************************************************************************************************************************************** localhost : ok=5 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 But this is a clear blocker for RHV 4.4 GA, which need to work with latest Ansible 2.9.z
*** Bug 1859603 has been marked as a duplicate of this bug. ***
This happens also without bonds or VLANs, issue related to interface detection failure.
Works fine on RHEL CLI deployment. ovirt-hosted-engine-setup-2.4.5-1.el8ev.noarch ovirt-hosted-engine-ha-2.4.4-1.el8ev.noarch Please retest on latest rhv-release-engine-4.4.1-12-001.noarch.rpm from 2020-07-23 on RHVH.
This error appears also on cli installation of HE on 4.3 [ ERROR ] {u'stderr_lines': [u"Error: Device '{ufailed:' not found."], u'ansible_loop_var': u'nic', u'end': u'2020-07-23 13:14:48.160525', u'_ansible_item_label': {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net': [u'eth0']}}, u'stdout': u'', u'nic': {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net': [u'eth0']}}, u'changed': True, u'msg': u'non-zero return code', u'rc': 10, u'failed': True, u'start': u'2020-07-23 13:14:47.976689', u'cmd': u"set -euo pipefail && nmcli -g GENERAL.TYPE device show {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net': [u'eth0']}}", u'stderr': u"Error: Device '{ufailed:' not found.", u'delta': u'0:00:00.183836', u'invocation': {u'module_args': {u'creates': None, u'executable': None, u'_uses_shell': True, u'strip_empty_ends': True, u'_raw_params': u"set -euo pipefail && nmcli -g GENERAL.TYPE device show {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net': [u'eth0']}}", u'removes': None, u'argv': None, u'warn': True, u'chdir': None, u'stdin_add_newline': True, u'stdin': None}}, u'stdout_lines': [], u'_ansible_no_log': False} ovirt-hosted-engine-setup-2.3.13-1.el7ev.noarch ansible-2.9.11-1.el7ae.noarch
(In reply to Petr Matyáš from comment #11) > This error appears also on cli installation of HE on 4.3 > > [ ERROR ] {u'stderr_lines': [u"Error: Device '{ufailed:' not found."], > u'ansible_loop_var': u'nic', u'end': u'2020-07-23 13:14:48.160525', > u'_ansible_item_label': {u'failed': False, u'changed': False, > u'ansible_facts': {u'host_net': [u'eth0']}}, u'stdout': u'', u'nic': > {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net': > [u'eth0']}}, u'changed': True, u'msg': u'non-zero return code', u'rc': 10, > u'failed': True, u'start': u'2020-07-23 13:14:47.976689', u'cmd': u"set -euo > pipefail && nmcli -g GENERAL.TYPE device show {u'failed': False, u'changed': > False, u'ansible_facts': {u'host_net': [u'eth0']}}", u'stderr': u"Error: > Device '{ufailed:' not found.", u'delta': u'0:00:00.183836', u'invocation': > {u'module_args': {u'creates': None, u'executable': None, u'_uses_shell': > True, u'strip_empty_ends': True, u'_raw_params': u"set -euo pipefail && > nmcli -g GENERAL.TYPE device show {u'failed': False, u'changed': False, > u'ansible_facts': {u'host_net': [u'eth0']}}", u'removes': None, u'argv': > None, u'warn': True, u'chdir': None, u'stdin_add_newline': True, u'stdin': > None}}, u'stdout_lines': [], u'_ansible_no_log': False} > > ovirt-hosted-engine-setup-2.3.13-1.el7ev.noarch > ansible-2.9.11-1.el7ae.noarch For 4.3, please use ovirt-ansible-hosted-engine-setup-1.0.38
Workaround for customers: 1. yum downgrade ansible 2. Rerun hosted engine setup
Here is the KCS: https://access.redhat.com/solutions/5245051. Big thanks to Steffen Froemer for testing and confirming the workaround.
(In reply to Nikolai Sednev from comment #10) > Works fine on RHEL CLI deployment. > ovirt-hosted-engine-setup-2.4.5-1.el8ev.noarch > ovirt-hosted-engine-ha-2.4.4-1.el8ev.noarch > > Please retest on latest rhv-release-engine-4.4.1-12-001.noarch.rpm from > 2020-07-23 on RHVH. Thanks Nikolai. I have tested with ovirt-ansible-hosted-engine-setup-1.1.7. It worked good. But I hear the possibility of ovirt-ansible-hosted-engine-setup-1.1.8 and respin of RHVH 4.4.1 I am waiting to test this with the last and final RHVH 4.4.1 to verify this bug
Tested with RHVH 4.4.1 (RHVH-4.4-20200722.1-RHVH-x86_64-dvd1.iso) which contains the package - ovirt-ansible-hosted-engine-setup-1.1.7 [root@rhsqa-grafton7-nic2 ~]# rpm -qa | grep ovirt-ansibl ovirt-ansible-engine-setup-1.2.4-1.el8ev.noarch ovirt-ansible-hosted-engine-setup-1.1.7-1.el8ev.noarch 1. Started the RHHI-V/Hyperconverged deployment from web console (cockpit) Gluster configuration is completed successfully 2. Continued to Self-Hosted-Engine/Hyperconverged deployment via web console (cockpit) Hosted Engine deployment is successfully completed 3. On the separate setup attempted for ansible based RHHI-V/Hyperconverged deployment using ovirt-ansible roles Hosted Engine deployment was successful. Based on the above observations, moving this bug to verified
This bugzilla is included in oVirt 4.4.1.1 Async release, published on July 13th 2020. Since the problem described in this bug report should be resolved in oVirt 4.4.1.1 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.