Bug 1860077

Error seen in the cockpit:

No valid network interface has been found
If you are using Bonds or VLANs Use the following naming conventions:
- VLAN interfaces: physical_device.VLAN_ID (for example, eth0.23, eth1.128, enp3s0.50)
- Bond interfaces: bond*number* (for example, bond0, bond1)
- VLANs on bond interfaces: bond*number*.VLAN_ID (for example, bond0.50, bond1.128)
* Supported bond modes: active-backup, balance-xor, broadcast, 802.3ad
* Networking teaming is not supported and will cause errors

(Originally by Satheesaran Sundaramoorthi)

2020-07-22 08:06:18,238+0000 ERROR ansible failed {
    "ansible_host": "localhost",
    "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml",
    "ansible_result": {
        "changed": true,
        "msg": "All items completed",
        "results": [
            {
                "_ansible_item_label": {
                    "ansible_facts": {
                        "host_net": [
                            "enp3s0f0",
                            "enp129s0f0"
                        ]
                    },
                    "changed": false,
                    "failed": false
                },
                "_ansible_no_log": false,
                "ansible_loop_var": "nic",
                "changed": true,
                "cmd": "set -euo pipefail && nmcli -g GENERAL.TYPE device show {'changed': False, 'ansible_facts': {'host_net': ['enp3s0f0', 'enp129s0f0']}, 'failed': False}",
                "delta": "0:00:00.058200",
                "end": "2020-07-22 08:06:17.781668",
                "failed": true,
                "invocation": {
                    "module_args": {
                        "_raw_params": "set -euo pipefail && nmcli -g GENERAL.TYPE device show {'changed': False, 'ansible_facts': {'host_net': ['enp3s0f0', 'enp129s0f0']}, 'failed': False}"
,
                        "_uses_shell": true,
                        "argv": null,
                        "chdir": null,
                        "creates": null,
                        "executable": null,
                        "removes": null,
                        "stdin": null,
                        "stdin_add_newline": true,
                        "strip_empty_ends": true,
                        "warn": true
                    }
                },
                "msg": "non-zero return code",
                "nic": {
                    "ansible_facts": {
                        "host_net": [
                            "enp3s0f0",
                            "enp129s0f0"
                        ]
                    },
                    "changed": false,
                    "failed": false
                },
                "rc": 10,
                "start": "2020-07-22 08:06:17.723468",
                "stderr": "Error: Device '{changed:' not found.",
                "stderr_lines": [
                    "Error: Device '{changed:' not found."
                ],
                "stdout": "",
                "stdout_lines": []
            }
        ]
    },
    "ansible_task": "Collect interface types",
    "ansible_type": "task",
    "status": "FAILED",
    "task_duration": 1
}

(Originally by Satheesaran Sundaramoorthi)

When running the same command, I see the problem:

[root@ ~]# set -euo pipefail && nmcli -g GENERAL.TYPE device show {'changed': False, 'ansible_facts': {'host_net': ['enp3s0f0', 'enp129s0f0']}, 'failed': False}
Error: Device '{changed:' not found.

(Originally by Satheesaran Sundaramoorthi)

Created attachment 1702040 [details]
get_network_interfaces.log

(Originally by Satheesaran Sundaramoorthi)

Assaf, do you know if this is more likly to be a bug in ansible or in virt-ansible-hosted-engine-setup ?

(Originally by Dominik Holler)

I think this is because of latest changes in ansible-2.9.11 fix.
nmcli - Add compatibility for new networkmanager library (https://github.com/ansible/ansible/pull/65726).

Failed ansible task is: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/tasks/filter_team_devices.yml#L3

(Originally by Gobinda Das)

Ansible fixed the behavior of set_fact module according to the documentation, but issue is our code:

https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/blob/master/tasks/pre_checks/001_validate_network_interfaces.yml#L71

It doesn't make sense to register result of set_fact execution to the variable with the same name as the fact. This worked in previous releases only by error.

So just removing line with register should fix the issue, here is sample playbook with confirmation:


$ cat test_facts.yml 
---
- name: test facts
  hosts: localhost
  connection: local

  tasks:
  - name: Set fact aaa
    set_fact:
      aaa: 'value1 value2'

  - name: Display fact aaa
    debug:
      var: aaa

  - name: Set fact bbb and register it
    set_fact:
      bbb: 'value1 value2'
    register: bbb

  - name: Display fact bbb
    debug:
      var: bbb


Here is the execution in Ansible 2.9.10:

$ ansible-playbook test_facts.yml 
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [oVirt infra] **********************************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************************
ok: [localhost]

TASK [Set fact aaa] *********************************************************************************************************************************************************************
ok: [localhost]

TASK [Display fact aaa] *****************************************************************************************************************************************************************
ok: [localhost] => {
    "aaa": "value1 value2"
}

TASK [Set fact bbb and register it] *****************************************************************************************************************************************************
ok: [localhost]

TASK [Display fact bbb] *****************************************************************************************************************************************************************
ok: [localhost] => {
    "bbb": "value1 value2"
}

PLAY RECAP ******************************************************************************************************************************************************************************
localhost                  : ok=5    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   



Here is the execution under Ansible 2.9.11:

$ ansible-playbook test_facts.yml 
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [oVirt infra] **********************************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************************
ok: [localhost]

TASK [Set fact aaa] *********************************************************************************************************************************************************************
ok: [localhost]

TASK [Display fact aaa] *****************************************************************************************************************************************************************
ok: [localhost] => {
    "aaa": "value1 value2"
}

TASK [Set fact bbb and register it] *****************************************************************************************************************************************************
ok: [localhost]

TASK [Display fact bbb] *****************************************************************************************************************************************************************
ok: [localhost] => {
    "bbb": {
        "ansible_facts": {
            "bbb": "value1 value2"
        },
        "changed": false,
        "failed": false
    }
}

PLAY RECAP ******************************************************************************************************************************************************************************
localhost                  : ok=5    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   


But this is a clear blocker for RHV 4.4 GA, which need to work with latest Ansible 2.9.z

(Originally by Martin Perina)

*** Bug 1859603 has been marked as a duplicate of this bug. ***

(Originally by Martin Perina)

This happens also without bonds or VLANs, issue related to interface detection failure.

(Originally by Nikolai Sednev)

Works fine on RHEL CLI deployment.
ovirt-hosted-engine-setup-2.4.5-1.el8ev.noarch
ovirt-hosted-engine-ha-2.4.4-1.el8ev.noarch

Please retest on latest rhv-release-engine-4.4.1-12-001.noarch.rpm from 2020-07-23 on RHVH.

(Originally by Nikolai Sednev)

This error appears also on cli installation of HE on 4.3

[ ERROR ] {u'stderr_lines': [u"Error: Device '{ufailed:' not found."], u'ansible_loop_var': u'nic', u'end': u'2020-07-23 13:14:48.160525', u'_ansible_item_label': {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net': [u'eth0']}}, u'stdout': u'', u'nic': {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net': [u'eth0']}}, u'changed': True, u'msg': u'non-zero return code', u'rc': 10, u'failed': True, u'start': u'2020-07-23 13:14:47.976689', u'cmd': u"set -euo pipefail && nmcli -g GENERAL.TYPE device show {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net': [u'eth0']}}", u'stderr': u"Error: Device '{ufailed:' not found.", u'delta': u'0:00:00.183836', u'invocation': {u'module_args': {u'creates': None, u'executable': None, u'_uses_shell': True, u'strip_empty_ends': True, u'_raw_params': u"set -euo pipefail && nmcli -g GENERAL.TYPE device show {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net': [u'eth0']}}", u'removes': None, u'argv': None, u'warn': True, u'chdir': None, u'stdin_add_newline': True, u'stdin': None}}, u'stdout_lines': [], u'_ansible_no_log': False}

ovirt-hosted-engine-setup-2.3.13-1.el7ev.noarch
ansible-2.9.11-1.el7ae.noarch

(Originally by Petr Matyas)

(In reply to Petr Matyáš from comment #11)
> This error appears also on cli installation of HE on 4.3
> 
> [ ERROR ] {u'stderr_lines': [u"Error: Device '{ufailed:' not found."],
> u'ansible_loop_var': u'nic', u'end': u'2020-07-23 13:14:48.160525',
> u'_ansible_item_label': {u'failed': False, u'changed': False,
> u'ansible_facts': {u'host_net': [u'eth0']}}, u'stdout': u'', u'nic':
> {u'failed': False, u'changed': False, u'ansible_facts': {u'host_net':
> [u'eth0']}}, u'changed': True, u'msg': u'non-zero return code', u'rc': 10,
> u'failed': True, u'start': u'2020-07-23 13:14:47.976689', u'cmd': u"set -euo
> pipefail && nmcli -g GENERAL.TYPE device show {u'failed': False, u'changed':
> False, u'ansible_facts': {u'host_net': [u'eth0']}}", u'stderr': u"Error:
> Device '{ufailed:' not found.", u'delta': u'0:00:00.183836', u'invocation':
> {u'module_args': {u'creates': None, u'executable': None, u'_uses_shell':
> True, u'strip_empty_ends': True, u'_raw_params': u"set -euo pipefail &&
> nmcli -g GENERAL.TYPE device show {u'failed': False, u'changed': False,
> u'ansible_facts': {u'host_net': [u'eth0']}}", u'removes': None, u'argv':
> None, u'warn': True, u'chdir': None, u'stdin_add_newline': True, u'stdin':
> None}}, u'stdout_lines': [], u'_ansible_no_log': False}
> 
> ovirt-hosted-engine-setup-2.3.13-1.el7ev.noarch
> ansible-2.9.11-1.el7ae.noarch

For 4.3, please use ovirt-ansible-hosted-engine-setup-1.0.38

(Originally by Asaf Rachmani)

Workaround for customers:

1. yum downgrade ansible
2. Rerun hosted engine setup

(Originally by Martin Perina)

Here is the KCS: https://access.redhat.com/solutions/5245051.
Big thanks to Steffen Froemer for testing and confirming the workaround.

(Originally by Marina Kalinin)

These changes doesn't impact RHHI-V as there is no corresponding RHVH image available,
as I see that this changes are only for RHEL 7 server based RHEL hosts & Manager, will
get the changes.

@Nikolai, I am changing the QA_contact to your name. I think you have verified the changes
in RHV 4.4.1 too, and also validate the changes with RHV 4.3.10 async with these changes
or assign the QA_contact to someone who will be validating this fix from RHV QE side

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3152