Bug 1860439

Description of problem:
Installing ReaR on undercloud using TripleO ansible 'backup-and-restore' role failed

command:
ansible-playbook -v -i /home/stack/tripleo-inventory.yaml --extra=\"ansible_ssh_common_args='-o StrictHostKeyChecking=no'\" --become --become-user root --tags bar_setup_rear /home/stack/bar_rear_setup_undercloud.yaml

TASK [backup-and-restore : Generate ReaR config file] **************************
fatal: [undercloud]: FAILED! => {"changed": false, "msg": "AnsibleError: Unexpected templating type error occurred on (# This configuration file is generated automatically\n# by the backup-and-restore role part of TripleO\n# Ansible. No not edit this file, all changes\n# will be lost. Refer to the following URL for\n# more information and implementation details:\n# https://opendev.org/openstack/tripleo-ansible\n\nOUTPUT_URL={{ tripleo_backup_and_restore_output_url }}\nISO_PREFIX={{ tripleo_backup_and_restore_hostname.stdout }}\nBACKUP_PROG_OPTIONS+=( --anchored --xattrs-include='*.*' --xattrs )\nBACKUP_PROG_EXCLUDE=( {{ tripleo_backup_and_restore_exclude_paths | sum(start=[]) | map('quote') | join(' ') }} )\nBACKUP_URL={{ tripleo_backup_and_restore_backup_url }}\n{% for item in (tripleo_backup_and_restore_local_config | dict2items) %}\n{{ item.key }}={{ item.value }}\n{% endfor %}\n): can only concatenate list (not \"str\") to list"}

PLAY RECAP *********************************************************************
undercloud                 : ok=9    changed=4    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   



Version-Release number of selected component (if applicable):
core_puddle: RHOS-16.1-RHEL-8-20200723.n.0

How reproducible:
always

Steps to Reproduce:
1) Deploy OSP-16.1 and Perform backup and restore using Tripleo backup-and-restore role located in /usr/share/ansible/roles/backup-and-restore located on the undercloud
2) Upstream documentation can be found here
https://github.com/openstack/tripleo-ansible/blob/master/doc/source/roles/role-backup_and_restore.rst

Actual results:

TASK [Install ReaR on undercloud] **********************************************
task path: /home/rhos-ci/jenkins/workspace/DFG-enterprise-backup_restore-16.1_director-rhel-virthost-3cont_2comp_3ceph-ipv4-geneve-rear_for_undercloud_controllers/infrared/02-install-rear.yaml:22
Friday 24 July 2020  07:57:54 +0000 (0:00:00.828)       0:00:04.651 *********** 
fatal: [undercloud-0]: FAILED! => {
    "changed": true,
    "cmd": "ansible-playbook -v -i /home/stack/tripleo-inventory.yaml --extra=\"ansible_ssh_common_args='-o StrictHostKeyChecking=no'\" --become --become-user root --tags bar_setup_rear /home/stack/bar_rear_setup_undercloud.yaml",
    "delta": "0:00:14.085970",
    "end": "2020-07-24 03:58:09.197093",
    "rc": 2,
    "start": "2020-07-24 03:57:55.111123"
}

STDOUT:

Using /etc/ansible/ansible.cfg as config file

PLAY [Install ReaR] ************************************************************

TASK [Gathering Facts] *********************************************************
ok: [undercloud]

TASK [backup-and-restore : Gather variables for each operating system] *********
ok: [undercloud] => (item=/usr/share/ansible/roles/backup-and-restore/vars/redhat.yml) => {"ansible_facts": {"tripleo_backup_and_restore_nfs_packages": ["nfs-utils"], "tripleo_backup_and_restore_rear_packages": ["rear", "syslinux", "genisoimage", "nfs-utils"]}, "ansible_included_var_files": ["/usr/share/ansible/roles/backup-and-restore/vars/redhat.yml"], "ansible_loop_var": "item", "changed": false, "item": "/usr/share/ansible/roles/backup-and-restore/vars/redhat.yml"}

TASK [backup-and-restore : Gather variables for each operating system] *********
ok: [undercloud] => (item=/usr/share/ansible/roles/backup-and-restore/vars/redhat.yml) => {"ansible_facts": {"tripleo_backup_and_restore_nfs_packages": ["nfs-utils"], "tripleo_backup_and_restore_rear_packages": ["rear", "syslinux", "genisoimage", "nfs-utils"]}, "ansible_included_var_files": ["/usr/share/ansible/roles/backup-and-restore/vars/redhat.yml"], "ansible_loop_var": "item", "changed": false, "item": "/usr/share/ansible/roles/backup-and-restore/vars/redhat.yml"}

TASK [backup-and-restore : Gather variables for each operating system] *********
ok: [undercloud] => (item=/usr/share/ansible/roles/backup-and-restore/vars/redhat.yml) => {"ansible_facts": {"tripleo_backup_and_restore_nfs_packages": ["nfs-utils"], "tripleo_backup_and_restore_rear_packages": ["rear", "syslinux", "genisoimage", "nfs-utils"]}, "ansible_included_var_files": ["/usr/share/ansible/roles/backup-and-restore/vars/redhat.yml"], "ansible_loop_var": "item", "changed": false, "item": "/usr/share/ansible/roles/backup-and-restore/vars/redhat.yml"}

TASK [backup-and-restore : Install required packages for ReaR] *****************
changed: [undercloud] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: syslinux-6.04-4.el8.x86_64", "Installed: xorriso-1.4.8-4.el8.x86_64", "Installed: rear-2.4-12.el8.x86_64", "Installed: bc-1.07.1-5.el8.x86_64", "Installed: libisofs-1.4.8-3.el8.x86_64", "Installed: attr-2.4.48-3.el8.x86_64", "Installed: syslinux-nonlinux-6.04-4.el8.noarch", "Installed: libburn-1.4.8-3.el8.x86_64", "Installed: libisoburn-1.4.8-4.el8.x86_64", "Installed: mtools-4.0.18-14.el8.x86_64"]}

TASK [backup-and-restore : Get local hostname] *********************************
changed: [undercloud] => {"changed": true, "cmd": ["hostname"], "delta": "0:00:00.005496", "end": "2020-07-24 03:58:07.771771", "rc": 0, "start": "2020-07-24 03:58:07.766275", "stderr": "", "stderr_lines": [], "stdout": "undercloud-0.redhat.local", "stdout_lines": ["undercloud-0.redhat.local"]}

TASK [backup-and-restore : Set bootstrap nodeid] *******************************
changed: [undercloud] => {"changed": true, "cmd": ["/bin/hiera", "-c", "/etc/puppet/hiera.yaml", "mysql_short_bootstrap_node_name"], "delta": "0:00:00.125331", "end": "2020-07-24 03:58:08.290515", "rc": 0, "start": "2020-07-24 03:58:08.165184", "stderr": "", "stderr_lines": [], "stdout": "undercloud-0", "stdout_lines": ["undercloud-0"]}

TASK [backup-and-restore : List enabled services by node] **********************
changed: [undercloud] => {"changed": true, "cmd": ["/bin/hiera", "-c", "/etc/puppet/hiera.yaml", "service_names"], "delta": "0:00:00.132733", "end": "2020-07-24 03:58:08.801156", "rc": 0, "start": "2020-07-24 03:58:08.668423", "stderr": "", "stderr_lines": [], "stdout": "[\"keystone_admin_api\",\n \"keystone_public_api\",\n \"apache\",\n \"ca_certs\",\n \"certmonger_user\",\n \"container_image_prepare\",\n \"logrotate_crond\",\n \"docker_registry\",\n \"glance_api\",\n \"haproxy\",\n \"heat_api\",\n \"heat_engine\",\n \"ironic_api\",\n \"ironic_conductor\",\n \"ironic_inspector\",\n \"ironic_neutron_agent\",\n \"ironic_pxe\",\n \"iscsid\",\n \"keepalived\",\n \"kernel\",\n \"keystone\",\n \"masquerade_networks\",\n \"memcached\",\n \"mistral_api\",\n \"mistral_engine\",\n \"mistral_event_engine\",\n \"mistral_executor\",\n \"mysql\",\n \"mysql_client\",\n \"neutron_api\",\n \"neutron_plugin_ml2\",\n \"neutron_dhcp\",\n \"neutron_l3\",\n \"neutron_ovs_agent\",\n \"nova_api\",\n \"nova_conductor\",\n \"nova_ironic\",\n \"nova_scheduler\",\n \"openstack_clients\",\n \"oslo_messaging_notify\",\n \"oslo_messaging_rpc\",\n \"placement\",\n \"podman\",\n \"rsyslog_baremetal_ansible\",\n \"snmp\",\n \"sshd\",\n \"swift_proxy\",\n \"swift_ringbuilder\",\n \"swift_storage\",\n \"tempest\",\n \"chrony\",\n \"timezone\",\n \"logrotate_tmpwatch\",\n \"tripleo_firewall\",\n \"tuned\",\n \"tripleo_validations\",\n \"zaqar_api\"]", "stdout_lines": ["[\"keystone_admin_api\",", " \"keystone_public_api\",", " \"apache\",", " \"ca_certs\",", " \"certmonger_user\",", " \"container_image_prepare\",", " \"logrotate_crond\",", " \"docker_registry\",", " \"glance_api\",", " \"haproxy\",", " \"heat_api\",", " \"heat_engine\",", " \"ironic_api\",", " \"ironic_conductor\",", " \"ironic_inspector\",", " \"ironic_neutron_agent\",", " \"ironic_pxe\",", " \"iscsid\",", " \"keepalived\",", " \"kernel\",", " \"keystone\",", " \"masquerade_networks\",", " \"memcached\",", " \"mistral_api\",", " \"mistral_engine\",", " \"mistral_event_engine\",", " \"mistral_executor\",", " \"mysql\",", " \"mysql_client\",", " \"neutron_api\",", " \"neutron_plugin_ml2\",", " \"neutron_dhcp\",", " \"neutron_l3\",", " \"neutron_ovs_agent\",", " \"nova_api\",", " \"nova_conductor\",", " \"nova_ironic\",", " \"nova_scheduler\",", " \"openstack_clients\",", " \"oslo_messaging_notify\",", " \"oslo_messaging_rpc\",", " \"placement\",", " \"podman\",", " \"rsyslog_baremetal_ansible\",", " \"snmp\",", " \"sshd\",", " \"swift_proxy\",", " \"swift_ringbuilder\",", " \"swift_storage\",", " \"tempest\",", " \"chrony\",", " \"timezone\",", " \"logrotate_tmpwatch\",", " \"tripleo_firewall\",", " \"tuned\",", " \"tripleo_validations\",", " \"zaqar_api\"]"]}

TASK [backup-and-restore : Calculate exclude paths] ****************************
ok: [undercloud] => {"ansible_facts": {"tripleo_backup_and_restore_exclude_paths": [["/data/*", "/tmp/*", "/ctl_plane_backups/*"], [], []]}, "changed": false}

TASK [backup-and-restore : Generate ReaR config file] **************************
fatal: [undercloud]: FAILED! => {"changed": false, "msg": "AnsibleError: Unexpected templating type error occurred on (# This configuration file is generated automatically\n# by the backup-and-restore role part of TripleO\n# Ansible. No not edit this file, all changes\n# will be lost. Refer to the following URL for\n# more information and implementation details:\n# https://opendev.org/openstack/tripleo-ansible\n\nOUTPUT_URL={{ tripleo_backup_and_restore_output_url }}\nISO_PREFIX={{ tripleo_backup_and_restore_hostname.stdout }}\nBACKUP_PROG_OPTIONS+=( --anchored --xattrs-include='*.*' --xattrs )\nBACKUP_PROG_EXCLUDE=( {{ tripleo_backup_and_restore_exclude_paths | sum(start=[]) | map('quote') | join(' ') }} )\nBACKUP_URL={{ tripleo_backup_and_restore_backup_url }}\n{% for item in (tripleo_backup_and_restore_local_config | dict2items) %}\n{{ item.key }}={{ item.value }}\n{% endfor %}\n): can only concatenate list (not \"str\") to list"}

PLAY RECAP *********************************************************************
undercloud                 : ok=9    changed=4    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   


STDERR:

[WARNING]: Found both group and host with same name: hypervisor


MSG:

non-zero return code

	to retry, use: --limit @/home/rhos-ci/jenkins/workspace/DFG-enterprise-backup_restore-16.1_director-rhel-virthost-3cont_2comp_3ceph-ipv4-geneve-rear_for_undercloud_controllers/infrared/02-install-rear.retry

PLAY RECAP *********************************************************************
undercloud-0               : ok=2    changed=1    unreachable=0    failed=1   

Expected results:

Successfully setup ReaR on the Undercloud using the TripleO Ansible backup-and-restore role

Additional info: