Bug 1430231

Summary: Failed to upgrade ocp3.4 to 3.5 when enabled excluders in the cluster
Product: OpenShift Container Platform Reporter: liujia <jiajliu>
Component: Cluster Version OperatorAssignee: Jan Chaloupka <jchaloup>
Status: CLOSED CURRENTRELEASE QA Contact: Anping Li <anli>
Severity: high Docs Contact:
Priority: high    
Version: 3.5.0CC: aos-bugs, jokerman, mmccomas, xtian
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openshift-ansible-3.5.27-1.git.0.08cda99.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-03-13 12:03:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description liujia 2017-03-08 06:07:33 UTC
Description of problem:
Failed to upgrade OCP3.4 to 3.5 while excluders installed and enabled in the cluster due to "No OpenShift version available, please ensure your systems are fully registered and have access to appropriate yum repositories."

TASK [openshift_version : fail] ************************************************
task path: /usr/share/ansible/openshift-ansible/roles/openshift_version/tasks/main.yml:76
fatal: [openshift-119.x.x.x]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_args": {
            "msg": "No OpenShift version available, please ensure your systems are fully registered and have access to appropriate yum repositories."
        }, 
        "module_name": "fail"
    }
}

MSG:

No OpenShift version available, please ensure your systems are fully registered and have access to appropriate yum repositories.


Version-Release number of selected component (if applicable):
atomic-openshift-utils-3.5.26-1.git.0.d88f597.el7.noarch

How reproducible:
always

Steps to Reproduce:
1. Install OCP3.4
2. Install atomic-openshift-excluder and atomic-openshift-docker-excluder(enabled by default)
3. Prepare 3.5 repo and upgrade 3.4 to 3.5

Actual results:
Upgrade failed.

Expected results:
Upgrade successfully.

Additional info:

It seems openshift-excluder is not disabled when upgrade failed.

openshift-118.x.x.x | SUCCESS | rc=0 >>
exclude= docker*1.20*  docker*1.19*  docker*1.18*  docker*1.17*  docker*1.16*  docker*1.15*  docker*1.14*  docker*1.13*  tuned-profiles-atomic-openshift-node  atomic-openshift-tests  atomic-openshift-sdn-ovs  atomic-openshift-recycle  atomic-openshift-pod  atomic-openshift-node  atomic-openshift-master  atomic-openshift-dockerregistry  atomic-openshift-clients-redistributable  atomic-openshift-clients  atomic-openshift 

openshift-119.x.x.x | SUCCESS | rc=0 >>
exclude= docker*1.20*  docker*1.19*  docker*1.18*  docker*1.17*  docker*1.16*  docker*1.15*  docker*1.14*  docker*1.13*  tuned-profiles-atomic-openshift-node  atomic-openshift-tests  atomic-openshift-sdn-ovs  atomic-openshift-recycle  atomic-openshift-pod  atomic-openshift-node  atomic-openshift-master  atomic-openshift-dockerregistry  atomic-openshift-clients-redistributable  atomic-openshift-clients  atomic-openshift

Comment 1 Jan Chaloupka 2017-03-08 14:43:17 UTC
Based on https://docs.openshift.com/container-platform/3.4/install_config/upgrading/automated_upgrades.html#upgrading-using-the-installation-utility-to-upgrade, the openshift excluder needs to be unexcluded if installed by running:

# atomic-openshift-excluder unexclude

before the upgrade.

Comment 2 Jan Chaloupka 2017-03-08 14:47:50 UTC
What I could do is to test if the enable_openshift_excluder is not set and if the atomic-openshift-excluder is installed. If it is I could call the unexclude command so the OCP version is retrieved. But then without running the exclude command at the end of the upgrade.

Or state that if the openshift excluder is installed and it is excluding, user need to unexclude it (as a fail message in the ansible). But running the unexclude command is exactly what is described in the docs.

Comment 3 Scott Dodson 2017-03-08 17:33:01 UTC
fixed in  openshift-ansible-3.5.27-1.git.0.08cda99.el7 by defaulting to managing the excluders

Comment 4 liujia 2017-03-09 08:08:53 UTC
Version:
atomic-openshift-utils-3.5.28-1.git.0.103513e.el7.noarch

Steps:
1. Install OCP3.4
2. Install atomic-openshift-excluder and atomic-openshift-docker-excluder(enabled by default)
3. Prepare 3.5 repo and upgrade 3.4 to 3.5

Result:
Upgrade successfully.