Bug 1482978

Summary: Upgrade from Openshift 3.5 to 3.6 fails when trying to storage migrate some oauthclientauthorizations
Product: OpenShift Container Platform Reporter: Nicolas Nosenzo <nnosenzo>
Component: SecurityAssignee: Mo <mkhan>
Status: CLOSED WONTFIX QA Contact: Xiaoli Tian <xtian>
Severity: low Docs Contact:
Priority: unspecified    
Version: 3.6.0CC: anli, aos-bugs, bleanhar, erich, faltahe, jialiu, jokerman, mmccomas, sdodson
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-30 21:47:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nicolas Nosenzo 2017-08-18 13:51:07 UTC
Description of problem:

I'm opening this bugzilla in order to track Issue described here https://github.com/openshift/origin/issues/15007 and the release where the fix will be included.


Version-Release number of selected component (if applicable):

$ ansible --version
ansible 2.3.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]
$ rpm -qa | grep -e openshift-ansible -e atomic-openshift-utils
openshift-ansible-callback-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-lookup-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-playbooks-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-filter-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch
atomic-openshift-utils-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-docs-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-roles-3.6.173.0.5-3.git.0.522a92a.el7.noarch

How reproducible:

100% in some minor releases of 3.6

Steps to Reproduce:
1. Try to upgrade a 3.5 cluster to 3.6
# ansible-playbook -i </path/to/inventory/file> \
    /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_6/upgrade.yml


Actual results:

TASK [Upgrade all storage] ******************************************************************************************************************************************************************************************************
task path: /usr/share/ansible/openshift-ansible/playbooks/common/openshift-cluster/upgrades/upgrade_control_plane.yml:11
Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py

fatal: [egsoslosm501.linux.lan]: FAILED! => {
    "changed": true, 
    "cmd": [
        "oc", 
        "adm", 
        "--config=/etc/origin/master/admin.kubeconfig", 
        "migrate", 
        "storage", 
        "--include=*", 
        "--confirm"
    ], 
    "delta": "0:00:56.843544", 
    "end": "2017-08-17 12:46:36.417413", 
    "failed": true, 
    "failed_when_result": true, 
    "invocation": {
        "module_args": {
            "_raw_params": "oc adm --config=/etc/origin/master/admin.kubeconfig migrate storage --include=* --confirm", 
            "_uses_shell": false, 
            "chdir": null, 
            "creates": null, 
            "executable": null, 
            "removes": null, 
            "warn": true
        }
    }, 
    "rc": 1, 
    "start": "2017-08-17 12:45:39.573869"
}

STDOUT:

error:     oauthclientauthorizations/H803517:system:serviceaccount:infrastruktur-test:jenkins : OAuthClientAuthorization "H803517:system:serviceaccount:infrastruktur-test:jenkins" is invalid: clientName: Internal error: system:serviceaccount:infrastruktur-test:jenkins has no redirectURIs; set serviceaccounts.openshift.io/oauth-redirecturi.<some-value>=<redirect> or create a dynamic URI using serviceaccounts.openshift.io/oauth-redirectreference.<some-value>=<reference>
summary: total=1652 errors=1 ignored=0 unchanged=1651 migrated=0
info: to rerun only failing resources, add --include=oauthclientauthorizations
error: 1 resources failed to migrate


Expected results:

Upgrade successful. 


Additional info:


Description of problem:

Version-Release number of the following components:
rpm -q openshift-ansible
rpm -q ansible
ansible --version

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Nicolas Nosenzo 2017-08-23 09:46:21 UTC
Is there a temporary workaround for this while we wait for the fix to be included in any errata ?

Comment 3 Nicolas Nosenzo 2017-08-30 07:51:40 UTC
Workaround added to the documentation works properly:

https://docs.openshift.com/container-platform/3.6/install_config/upgrading/upgrading_known_issues.html#upgrading-known-issue-BZ1482978

Comment 6 Mo 2017-08-30 21:47:44 UTC
Workarounds are documented in the 3.6 upgrade docs (3.5 to 3.6).  The cause of these issues has been fixed in the 3.6 release (the same workarounds will not be needed when upgrading 3.6 to 3.7).