Bug 1446737

Summary: [3.5] Redeploy certificates fails with custom openshift_hosted_router_certificate
Product: OpenShift Container Platform Reporter: Andrew Butcher <abutcher>
Component: InstallerAssignee: Andrew Butcher <abutcher>
Status: CLOSED ERRATA QA Contact: Gaoyun Pei <gpei>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.5.1CC: aos-bugs, gpei, jialiu, jokerman, mmccomas
Target Milestone: ---   
Target Release: 3.5.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Previously, replacement of router certificates through use of the certificate redeployment playbook (playbooks/byo/openshift-cluster/redeploy-certificates.yml) or the router certificate redeployment playbook (playbooks/byo/openshift-cluster/redeploy-router-certificates.yml) would fail when a custom router certificate was provided. Custom router certificates set by openshift_hosted_router_certificate within the inventory may now be redeployed.
Story Points: ---
Clone Of:
: 1446741 1446745 (view as bug list) Environment:
Last Closed: 2017-12-14 21:01:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1446741, 1446745    

Description Andrew Butcher 2017-04-28 17:15:25 UTC
Description of problem:

Certificate redeployment (redeploy-certificates.yml) fails to redeploy router when a custom router certificate has been provided during install via openshift_hosted_router_certificate.


For example:

openshift_hosted_router_certificate={'certfile': '/home/abutcher/wildcard-flibberty-jibbet.com.crt', 'keyfile': '/home/abutcher/wildcard-flibberty-jibbet.com.key', 'cafile'\
: '/home/abutcher/wildcard-ca.crt'}


Redeployment tasks will complete successfully but the router redeploy will not complete.

TASK [Update router environment variables] *************************************
skipping: [ec2-54-146-165-55.compute-1.amazonaws.com] => {"changed": false, "skip_reason": "Conditional check failed", "skipped": true}

TASK [Delete existing router certificate secret] *******************************
changed: [ec2-54-146-165-55.compute-1.amazonaws.com] => {"changed": true, "cmd": ["oc", "delete", "secret/router-certs", "--config=/tmp/openshift-ansible-dqqYTg/admin.kubeconfig", "-n", "default"], "delta": "0:00:00.503799", "end": "2017-04-14 01:49:38.548672", "rc": 0, "start": "2017-04-14 01:49:38.044873", "stderr": "", "stdout": "secret \"router-certs\" deleted", "stdout_lines": ["secret \"router-certs\" deleted"], "warnings": []}

TASK [Remove router service annotations] ***************************************
changed: [ec2-54-146-165-55.compute-1.amazonaws.com] => {"changed": true, "cmd": ["oc", "annotate", "service/router", "service.alpha.openshift.io/serving-cert-secret-name-", "service.alpha.openshift.io/serving-cert-signed-by-", "--config=/tmp/openshift-ansible-dqqYTg/admin.kubeconfig", "-n", "default"], "delta": "0:00:00.497830", "end": "2017-04-14 01:49:40.743203", "rc": 0, "start": "2017-04-14 01:49:40.245373", "stderr": "", "stdout": "service \"router\" annotated", "stdout_lines": ["service \"router\" annotated"], "warnings": []}

TASK [Add serving-cert-secret annotation to router service] ********************
changed: [ec2-54-146-165-55.compute-1.amazonaws.com] => {"changed": true, "cmd": ["oc", "annotate", "service/router", "service.alpha.openshift.io/serving-cert-secret-name=router-certs", "--config=/tmp/openshift-ansible-dqqYTg/admin.kubeconfig", "-n", "default"], "delta": "0:00:00.517662", "end": "2017-04-14 01:49:42.896729", "rc": 0, "start": "2017-04-14 01:49:42.379067", "stderr": "", "stdout": "service \"router\" annotated", "stdout_lines": ["service \"router\" annotated"], "warnings": []}

TASK [Redeploy router] *********************************************************
changed: [ec2-54-146-165-55.compute-1.amazonaws.com] => {"changed": true, "cmd": ["oc", "deploy", "dc/router", "--latest", "--config=/tmp/openshift-ansible-dqqYTg/admin.kubeconfig", "-n", "default"], "delta": "0:00:00.512418", "end": "2017-04-14 01:49:45.097579", "rc": 0, "start": "2017-04-14 01:49:44.585161", "stderr": "", "stdout": "Started deployment #3", "stdout_lines": ["Started deployment #3"], "warnings": []}


After redeployment the router pod will get stuck in ContainerCreating due to secrets "router-certs" not found.

Version-Release number of selected component (if applicable):
openshift-ansible-3.5.60-1.git.0.b6f77a6.el7

How reproducible:
Always.

Steps to Reproduce:
1. Deploy a cluster specifying a custom router certificate via openshift_hosted_router_certificate.
2. Redeploy certificates. The router certificates can be targeted directly by running playbooks/byo/openshift-cluster/redeploy-router-certificates.yml

Actual results:
Router pod is not successfully redeployed.

Expected results:
Router pod is successfully redeployed.

Additional info:

Comment 1 Andrew Butcher 2017-05-01 14:38:16 UTC
Proposed fix: https://github.com/openshift/openshift-ansible/pull/4037

Comment 2 Gaoyun Pei 2017-05-03 08:07:20 UTC
Verify this bug with openshift-ansible-3.5.65-1.git.0.da18a47.el7.noarch

With custom router certificate provided during install via openshift_hosted_router_certificate, then redeploy cert against the cluster, custom router cert would be retained and router pod was running well.

Comment 5 errata-xmlrpc 2017-12-14 21:01:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3438