Bug 1503995
| Summary: | Fail to pull ose image during upgrade due to docker auth token was not updated even if oreg_auth_credentials_replace=true is set | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | liujia <jiajliu> |
| Component: | Cluster Version Operator | Assignee: | Michael Gugino <mgugino> |
| Status: | CLOSED ERRATA | QA Contact: | liujia <jiajliu> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 3.7.0 | CC: | anli, aos-bugs, jokerman, mmccomas |
| Target Milestone: | --- | ||
| Target Release: | 3.7.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-11-28 22:18:08 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Verified on openshift-ansible-3.7.0-0.189.0. once set oreg_auth_credentials_replace=true, the upgrade finished. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:3188 |
Description of problem: Run upgrade from 3.6 to 3.7 against containerzied ha cluster, when docker token is experired, set oreg_auth_credentials_replace=true,upgrade failed at task [Verify containers are available for upgrade] when authenticated registry updated its key. fatal: []: FAILED! => {"changed": false, "cmd": ["docker", "pull", "openshift3/ose:v3.7.0-0.158.0"], "delta": "0:00:02.563704", "end": "2017-10-19 04:00:47.702619", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2017-10-19 04:00:45.138915", "stderr": "unauthorized: authentication required", "stderr_lines": ["unauthorized: authentication required"], "stdout": "Trying to pull repository registry.reg-aws.openshift.com:443/openshift3/ose ... \nTrying to pull repository registry.access.redhat.com/openshift3/ose ... \nTrying to pull repository registry.reg-aws.openshift.com:443/openshift3/ose ... \nTrying to pull repository registry.access.redhat.com/openshift3/ose ... \nTrying to pull repository docker.io/openshift3/ose ... ", "stdout_lines": ["Trying to pull repository registry.reg-aws.openshift.com:443/openshift3/ose ... ", "Trying to pull repository registry.access.redhat.com/openshift3/ose ... ", "Trying to pull repository registry.reg-aws.openshift.com:443/openshift3/ose ... ", "Trying to pull repository registry.access.redhat.com/openshift3/ose ... ", "Trying to pull repository docker.io/openshift3/ose ... "]} Failure summary: 1. Hosts: x.x.x.x Play: Verify upgrade targets Task: Verify containers are available for upgrade Message: non-zero return code Task [Create credentials for docker cli registry auth] was skipped from upgrade log. Docker login with new password to update .docker/config.json on all hosts can workaround. Version-Release number of the following components: openshift-ansible-3.7.0-0.158.0.git.0.8941dd5.el7.noarch openshift-ansible-roles-3.7.0-0.158.0.git.0.8941dd5.el7.noarch How reproducible: always Steps to Reproduce: 1.Containerized install 3.6 ha cluster 2.Edit hosts file to update registry password when authenticated registry updated its key(A->B). <--snip--> oreg_auth_credentials_replace=true oreg_auth_password=B <--snip--> 3.Upgrade ocp v3.6 to v3.7. #ansible-playbook -i hosts /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_7/upgrade.yml Actual results: Upgrade failed due to can not pull images. Expected results: Upgrade succeed. Additional info: Please attach logs from ansible-playbook with the -vvv flag