Bug 1559225

Summary: OpenShift installer upgrade playbook doesn't apply latest rpm packages when a git version change
Product: OpenShift Container Platform Reporter: Takayoshi Kimura <tkimura>
Component: InstallerAssignee: Russell Teague <rteague>
Status: CLOSED WONTFIX QA Contact: liujia <jiajliu>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.7.0CC: aos-bugs, chlunde, ggore, jokerman, mmccomas, mmckinst, pdwyer, rteague, sdodson
Target Milestone: ---Keywords: Reopened
Target Release: 3.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-23 02:57:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Takayoshi Kimura 2018-03-22 02:31:07 UTC 
Description of problem:

When running upgrade.yaml, only first master receives rpm upgrade but all other masters/nodes.

Version-Release number of the following components:

$ rpm -q openshift-ansible
openshift-ansible-3.7.23-1.git.0.bc406aa.el7.noarch
$ rpm -q ansible
ansible-2.4.2.0-2.el7.noarch
$ ansible --version
ansible 2.4.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/nekop/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]

atomic-openshift-3.7.23-1.git.0.8edc154.el7
to:
atomic-openshift-3.7.23-1.git.5.83efd71.el7

How reproducible:

Always

Steps to Reproduce:
1. Run upgrade.yml playbook
2.
3.

Actual results:

Only first master rpms updated

Expected results:

All hosts rpm updated

Additional info:
Comment 2 Takayoshi Kimura 2018-03-22 02:36:22 UTC 
To workaround issue, we need to upgrade the rpm packages and restart services manually.

# /usr/sbin/atomic-openshift-excluder unexclude
# yum update atomic-openshift* tuned-profiles-atomic-openshift* -y
# /usr/sbin/atomic-openshift-excluder exclude
# systemctl restart atomic-openshift-master-* atomic-openshift-node
Comment 9 Scott Dodson 2018-04-16 13:37:10 UTC 
https://github.com/openshift/openshift-ansible/pull/7844
Comment 12 Scott Dodson 2018-04-19 13:38:41 UTC 
We don't have any method to ensure proper upgrade paths while ensuring that latest package has been installed. As such we need to rely on the version having been upgraded in order to properly trigger an upgrade. We cannot enforce upgrades automatically when only the release portion of the version string has been incremented.
Comment 14 liujia 2018-04-23 02:57:10 UTC 
Update status back.