Bug 1399994

Summary: upgrade nodes failed for wrong openshift version was used in containerized env
Product: OpenShift Container Platform Reporter: liujia <jiajliu>
Component: Cluster Version OperatorAssignee: Scott Dodson <sdodson>
Status: CLOSED WONTFIX QA Contact: liujia <jiajliu>
Severity: low Docs Contact:
Priority: low    
Version: 3.4.0CC: aos-bugs, jokerman, mmccomas
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-01-31 16:00:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
upgrade_node log none

Description liujia 2016-11-30 08:47:04 UTC 
Description of problem:
When there is  /usr/bin/openshift in containerized env, the upgrade_nodes.yaml fail at task "Verify masters are already upgraded" and report message "Master running 3.3.1.5 must be upgraded to 3.4.0.30 before node upgrade can be run." 
The root cause is that openshift_fact get current version wrongly using "/usr/bin/openshift version".  It should use "/usr/local/bin/openshift version" which is updated by openshift_container_binary_sync.py


Version-Release number of selected component (if applicable):
atomic-openshift-utils-3.4.31-1.git.0.4b78e57.el7.noarch

How reproducible:
always

Steps to Reproduce:
1. containerized install ocp 3.3 on aws.
2. install atomic-openshift-3.3.1.5 on first master which installed as a dependency of installing atomic-openshift-tests.
3. upgrade masters to v3.4
# ansible-playbook $upgrade_control_plane.yml
4. upgrade nodes  to v3.4
# ansible-playbook $upgrade_nodes.yml

Actual results:
Upgrade failed due to the current version come from /usr/bin/openshift version.

Expected results:
Upgrade node successfully.

Additional info:
upgrade log in attachment
Comment 1 liujia 2016-11-30 08:48:59 UTC 
Created attachment 1226194 [details]
upgrade_node log
Comment 2 Scott Dodson 2016-11-30 19:58:15 UTC 
Only happens when atomic-openshift-tests is installed, marking UpcomingRelease
Comment 3 Scott Dodson 2019-01-31 16:00:16 UTC 
There appear to be no active cases related to this bug. As such we're closing this bug in order to focus on bugs that are still tied to active customer cases. Please re-open this bug if you feel it was closed in error or a new active case is attached.