Bug 1345870

Summary: Failed to upgrade OSE to previous version
Product: OpenShift Container Platform Reporter: Anping Li <anli>
Component: Cluster Version OperatorAssignee: Devan Goodwin <dgoodwin>
Status: CLOSED CANTFIX QA Contact: Anping Li <anli>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.2.1CC: anli, aos-bugs, bleanhar, dgoodwin, jokerman, mmccomas
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-20 11:42:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Anping Li 2016-06-13 10:56:39 UTC
Description of problem:
Failed to upgrade ose to previous version, for example: Running upgrade playbook , it failed with msg: Docker 1.8.2 must be upgraded to Docker 1.10 or greater. After that, the openshift-master service could't be started due to some upgrade task had been done before the docker version is checked.

Version-Release number of selected component (if applicable):
atomic-openshift-utils-3.2.1-1.git.0.b8d1b5e.el7.noarch

How reproducible:
always

Steps to Reproduce:
1) install openshift 3.0
2) upgrade to atomic-openshift 3.1
    ansible-playbook -i config/hosts  /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_0_to_v3_1/upgrade.yml

3) check the master status
  systemctl status openshift-master
  systemctl status atomic-openshift-master


2)
TASK: [docker | Fail if Docker upgrade is required] *************************** 
failed: [192.168.0.36] => {"failed": true}
msg: Docker 1.8.2 must be upgraded to Docker 1.10 or greater

FATAL: all hosts have already failed -- aborting

PLAY RECAP ******************************************************************** 
           to retry, use: --limit @/root/upgrade.retry

192.168.0.36               : ok=80   changed=12   unreachable=0    failed=1   
192.168.0.39               : ok=22   changed=2    unreachable=0    failed=0   
localhost                  : ok=17   changed=0    unreachable=0    failed=0   


3)

[root@host4master ~]# systemctl start openshift-master
Failed to start openshift-master.service: Unit openshift-master.service failed to load: No such file or directory.

[root@host4master ~]# systemctl status atomic-openshift-master
● atomic-openshift-master.service - Atomic OpenShift Master
   Loaded: loaded (/usr/lib/systemd/system/atomic-openshift-master.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: https://github.com/openshift/origin
[root@host4master ~]# systemctl status openshift-master
● openshift-master.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)

Jun 13 04:36:05 host4master.example.com openshift-master[1032]: I0613 04:36:05.560181    1032 controller.go:85] Ignoring Deploy...ql-1
Jun 13 04:36:05 host4master.example.com openshift-master[1032]: I0613 04:36:05.586736    1032 controller.go:85] Ignoring Deploy...ry-2
Jun 13 04:36:05 host4master.example.com openshift-master[1032]: I0613 04:36:05.599374    1032 controller.go:85] Ignoring Deploy...er-1
Jun 13 04:38:07 host4master.example.com openshift-master[1032]: I0613 04:38:07.534893    1032 controller.go:85] Ignoring Deploy...le-2
Jun 13 04:38:07 host4master.example.com openshift-master[1032]: I0613 04:38:07.943703    1032 controller.go:85] Ignoring Deploy...ql-1
Jun 13 04:38:08 host4master.example.com opensh
ift-master[1032]: I0613 04:38:08.002881    1032 controller.go:85] Ignoring Deploy...ry-2
Jun 13 04:38:08 host4master.example.com openshift-master[1032]: I0613 04:38:08.048775    1032 controller.go:85] Ignoring Deploy...er-1
Jun 13 04:38:49 host4master.example.com systemd[1]: Stopping OpenShift Master...
Jun 13 04:38:49 host4master.example.com openshift[1032]: received terminated signal, shutting down...
Jun 13 04:38:49 host4master.example.com systemd[1]: Stopped OpenShift Master.
Hint: Some lines were ellipsized, use -l to show in full.

Expected Result:

The docker version checking should be called before any upgrade task.


Additional info:

Comment 1 Brenton Leanhardt 2016-06-14 15:08:23 UTC
Just a heads up, we are now going to require that the version of openshift-ansible match either the version you are installing or the version you wish to upgrade to.

Comment 2 Scott Dodson 2016-06-28 15:29:43 UTC
As Brenton mentioned, need to use the latest version of atomic-openshift-utils in the 3.1 channels. The 3.2 channels now container 3.2 specific code.

Comment 3 Anping Li 2016-06-29 01:49:10 UTC
@Scott, Yes. To avoid using these unsupported playbooks. I think we need to remove them from v3.2 openshift-ansible.

Comment 4 Devan Goodwin 2016-07-20 11:42:34 UTC
Going to close this as there's nothing to verify here, you can no longer do this upgrade in master or 3.2 version of the playbooks.