Bug 1727064

Summary: Deployment from Bastion Machine fails with Ansible 2.8 on upstream
Product: [oVirt] ovirt-engine-metrics Reporter: Evgeny Slutsky <eslutsky>
Component: GenericAssignee: Evgeny Slutsky <eslutsky>
Status: CLOSED UPSTREAM QA Contact: Ivana Saranova <isaranov>
Severity: high Docs Contact:
Priority: high    
Version: 1.3.2CC: bugs, jzmeskal, lleistne, rmeggins
Target Milestone: ovirt-4.3.5-1Flags: sbonazzo: ovirt-4.3?
Target Release: 1.3.3.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-engine-metrics-1.3.3.2 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-22 14:57:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Evgeny Slutsky 2019-07-04 11:00:29 UTC
when running openshift deployment from bastion VM:
ANSIBLE_ROLES_PATH="/usr/share/ansible/roles/:/usr/share/ansible/openshift-ansible/roles"   ansible-playbook -vvvvv -i integ.ini install_okd.yaml -e @vars.yaml -e @secure_vars.yaml --ask-vault-pass

Fails with error
019-07-03 11:39:06,194 p=35014 u=root |  TASK [openshift_logging : conditional_set_fact] *******************************************************************************************************************************************************************
2019-07-03 11:39:06,194 p=35014 u=root |  task path: /usr/share/ansible/openshift-ansible/roles/openshift_logging/tasks/set_defaults_from_current.yml:187
2019-07-03 11:39:06,194 p=35014 u=root |  Wednesday 03 July 2019  11:39:06 +0000 (0:00:00.084)       0:07:16.269 ******** 
2019-07-03 11:39:06,480 p=35014 u=root |  The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 144, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 648, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/share/ansible/openshift-ansible/roles/lib_openshift/action_plugins/conditional_set_fact.py", line 57, in run
    local_facts[param] = facts[other_var]
TypeError: string indices must be integers

2019-07-03 11:39:06,481 p=35014 u=root |  fatal: [master0.metrics.eng.tlv2.redhat.com]: FAILED! => {
    "msg": "Unexpected failure during module execution.", 
    "stdout": ""
}



Ansible Logs:
https://paste.fedoraproject.org/paste/Plzpb-FfuZLX9sjh1LFjRA/

Note:
during the deployment ansible-2.8 was installed from the epel repository,

ths issue was solved after, after downgrading to ansible-2.7 from ansible official Repos:
root@dhcp-8-148 ~]# cat /etc/yum.repos.d/CentOS-ANSIBLE.repo 
# CentOS-ANSIBLE.repo
#
# Please see https://wiki.centos.org/SpecialInterestGroup/ConfigManagementSIG/Ansible
# for more information

[centos-ansible27]
name=CentOS-7 - ansible27
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=configmanagement-ansible27
#baseurl=http://mirror.centos.org/$contentdir/7/configmanagement/$basearch/ansible27/
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-ConfigManagement

Comment 1 Shirly Radco 2019-07-04 11:07:25 UTC
The bution machine is registered to the OpenShift repo.
AFAIK ansible should remain 2.7 on bastion machine.
Where did epel repo came from?

Comment 2 Evgeny Slutsky 2019-07-04 11:34:23 UTC
epel  repo is installed by cloud-init,
https://github.com/oVirt/ovirt-engine-metrics/blob/b81f7ec43d6d47a497a77f14674f9b5145828f3f/roles/oVirt.origin-on-ovirt/templates/vars.yaml.template#L165

during the bastion VM creation.

Comment 3 Evgeny Slutsky 2019-07-04 11:54:24 UTC
ansible26 repo is installed from SIG:
https://github.com/oVirt/ovirt-engine-metrics/blob/7e6f404b87f59a3cce26e8f1692f2f14f43596bd/roles/oVirt.origin-on-ovirt/vars/main.yml#L27

there is no ansible27 in SIG,
i guess thats why epel was added here, and it worked fine until epel changed to ansible28.

Comment 4 Evgeny Slutsky 2019-07-04 12:25:54 UTC
this is the openshift-ansible bug https://github.com/openshift/openshift-ansible/issues/11697

Comment 5 Rich Megginson 2019-07-08 14:26:58 UTC
What info is being requested from me?

Comment 6 Evgeny Slutsky 2019-07-09 09:36:37 UTC
(In reply to Rich Megginson from comment #5)
> What info is being requested from me?

Please review this patch, 
https://gerrit.ovirt.org/101618

Comment 7 Rich Megginson 2019-07-09 14:46:58 UTC
(In reply to Evgeny Slutsky from comment #6)
> (In reply to Rich Megginson from comment #5)
> > What info is being requested from me?
> 
> Please review this patch, 
> https://gerrit.ovirt.org/101618

Done

Comment 8 Lucie Leistnerova 2019-07-16 12:59:28 UTC
We don't have capacity now to verify metrics bugs. Moving to async 4.3.5-1.