Bug 1602166

Created attachment 1459561 [details]
Verbose log of install

Description of problem:

Unable to install a new OCP cluster 3.6 on RHEL 7.4. A scaleup of a master on an existing 3.6 cluster also fails with the error.

Failure summary:


  1. Hosts:    vm254-69.gsslab.pnq2.redhat.com, vm254-77.gsslab.pnq2.redhat.com, vm254-80.gsslab.pnq2.redhat.com
     Play:     Configure nodes
     Task:     Install Node package
     Message:  No package matching 'atomic-openshift-node-3.6.173.0.124' found available, installed or updated


The issue seems to be with the installer doing the following:

<vm254-77.gsslab.pnq2.redhat.com> (0, '\r\n{"changed": true, "end": "2018-07-17 10:48:38.270975", "stdout": "Loaded plugins: product-id, search-disabled-repos, subscription-manager\\nCleaning repos: rhel-7-fast-datapath-rpms rhel-7-server-extras-rpms\\n              : rhel-7-server-ose-3.6-rpms rhel-7-server-rpms\\nCleaning up everything\\nMaybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos", "cmd": ["yum", "clean", "all"], "rc": 0, "start": "2018-07-17 10:48:34.223863", "stderr": "", "delta": "0:00:04.047112", "invocation": {"module_args": {"warn": false, "executable": null, "_uses_shell": false, "_raw_params": "yum clean all", "removes": null, "creates": null, "chdir": null, "stdin": null}}}\r\n', 'Shared connection to vm254-77.gsslab.pnq2.redhat.com closed.\r\n')
changed: [vm254-77.gsslab.pnq2.redhat.com] => {

Cleaning repos: rhel-7-fast-datapath-rpms rhel-7-server-extras-rpms
              : rhel-7-server-ose-3.6-rpms rhel-7-server-rpms
Cleaning up everything


because the repositories are removed, the installer cannot find the rpm on any of the associated node/master. 

One more observation is 
https://access.redhat.com/articles/2176281

The above page says Ansible 2.4 is not a tested integration with 3.6 on RHEL 7.4.

However, the extras repository installs ansible 2.4 when installing atomic-openshift-utils. 

Unsure if Ansible 2.4 is causing this problem.

Version-Release number of the following components:
openshift-ansible-3.6.173.0.124-1.git.0.5f3f028.el7.noarch
ansible-2.4.2.0-2.el7.noarch


# ansible --version
ansible 2.4.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]


How reproducible:

Steps to Reproduce:
1. Install a RHEL 7.4 and perform the prereqs for OCP 3.6
2. Run the installer for 3.6
3.

Actual results:

PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost                  : ok=14   changed=0    unreachable=0    failed=0   
vm254-68.gsslab.pnq2.redhat.com : ok=261  changed=43   unreachable=0    failed=0   
vm254-69.gsslab.pnq2.redhat.com : ok=202  changed=32   unreachable=0    failed=1   
vm254-77.gsslab.pnq2.redhat.com : ok=544  changed=122  unreachable=0    failed=1   
vm254-80.gsslab.pnq2.redhat.com : ok=421  changed=95   unreachable=0    failed=1   



Failure summary:


  1. Hosts:    vm254-69.gsslab.pnq2.redhat.com, vm254-77.gsslab.pnq2.redhat.com, vm254-80.gsslab.pnq2.redhat.com
     Play:     Configure nodes
     Task:     Install Node package
     Message:  No package matching 'atomic-openshift-node-3.6.173.0.124' found available, installed or updated


Expected results:
Installation to succeed