Bug 1559248 - Fail to upgrade ocp without docker registry deployed
Summary: Fail to upgrade ocp without docker registry deployed
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cluster Version Operator
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 3.10.0
Assignee: Vadim Rutkovsky
QA Contact: liujia
Depends On:
Blocks: 1577960 1581243
TreeView+ depends on / blocked
Reported: 2018-03-22 04:43 UTC by liujia
Modified: 2018-07-30 19:11 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1577960 1581243 (view as bug list)
Last Closed: 2018-07-30 19:10:48 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:1816 None None None 2018-07-30 19:11:27 UTC
Github openshift openshift-ansible pull 8354 'None' 'closed' 'Don''t validate registry certificates when registry is disabled' 2019-11-11 12:34:46 UTC
Red Hat Knowledge Base (Solution) 3441211 None None None 2018-05-11 23:11:44 UTC

Description liujia 2018-03-22 04:43:06 UTC
Description of problem:
Fail to upgrade ocp without docker registry deployed. Need to add a check if docker registry deployed on current ocp when redeploy it.

TASK [Redeploy docker registry] ************************************************
task path: /usr/share/ansible/openshift-ansible/playbooks/openshift-hosted/private/redeploy-registry-certificates.yml:89
fatal: [x.x.x.x]: FAILED! => {"changed": true, "cmd": ["oc", "rollout", "latest", "dc/docker-registry", "--config=/tmp/openshift-ansible-YjNnLv/admin.kubeconfig", "-n", "default"], "delta": "0:00:00.320098", "end": "2018-03-21 23:22:38.128920", "msg": "non-zero return code", "rc": 1, "start": "2018-03-21 23:22:37.808822", "stderr": "Error from server (NotFound): deploymentconfigs.apps.openshift.io \"docker-registry\" not found", "stderr_lines": ["Error from server (NotFound): deploymentconfigs.apps.openshift.io \"docker-registry\" not found"], "stdout": "", "stdout_lines": []}

  - name: Redeploy docker registry
    command: >
      {{ openshift_client_binary }} rollout latest dc/docker-registry
      --config={{ mktemp.stdout }}/admin.kubeconfig
      -n default

Version-Release number of the following components:

How reproducible:

Steps to Reproduce:
1. Install ocp without docker registry deployed.
2. Upgrade above ocp

Actual results:
Upgrade failed.

Expected results:
Upgrade succeed.

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 5 Scott Dodson 2018-05-11 21:06:33 UTC
https://github.com/openshift/openshift-ansible/pull/8354 proposed fix 

This fix is applicable to 3.9 as well

Comment 10 liujia 2018-05-22 03:05:50 UTC

Still failed to upgrade at the same task.

TASK [Redeploy docker registry] ************************************************
task path: /usr/share/ansible/openshift-ansible/playbooks/openshift-hosted/private/redeploy-registry-certificates.yml:90
Tuesday 22 May 2018  02:42:26 +0000 (0:00:00.071)       0:27:42.027 *********** 
fatal: [x.x.x.x]: FAILED! => {"changed": true, "cmd": ["oc", "rollout", "latest", "dc/docker-registry", "--config=/tmp/openshift-ansible-QAL8jR/admin.kubeconfig", "-n", "default"], "delta": "0:00:00.204963", "end": "2018-05-21 22:42:30.103079", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2018-05-21 22:42:29.898116", "stderr": "Error from server (NotFound): deploymentconfigs.apps.openshift.io \"docker-registry\" not found", "stderr_lines": ["Error from server (NotFound): deploymentconfigs.apps.openshift.io \"docker-registry\" not found"], "stdout": "", "stdout_lines": []}

The playbook was not skipped in pr8354. Need to change to bool for openshift_hosted_manage_registry.

And for pr8354, if the condition should be added for the whole playbook(../../../openshift-hosted/private/upgrade_poll_and_check_certs.yml) or just for docker-registry related tasks. I noticed that there are some tasks about routers in the playbook.

Assign back.

Comment 12 Vadim Rutkovsky 2018-05-22 11:58:00 UTC
Created https://github.com/openshift/openshift-ansible/pull/8472

Comment 13 Vadim Rutkovsky 2018-05-24 07:40:18 UTC
Fix is available in openshift-ansible-3.10.0-0.51.0

Comment 14 liujia 2018-05-28 09:18:17 UTC
Verified on openshift-ansible-3.10.0-0.53.0.git.0.53fe016.el7.noarch

Comment 16 errata-xmlrpc 2018-07-30 19:10:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.