Hide Forgot
Description of problem: The service catalog does not have a `redeploy-certificates.yml` playbook. If the ca were to be replaced this may cause issues with the service catalog. Version-Release number of the following components: rpm -q openshift-ansible rpm -q ansible ansible --version How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Please include the entire output from the last TASK line through the end of output if an error is generated Expected results: Additional info: Please attach logs from ansible-playbook with the -vvv flag
https://github.com/openshift/openshift-ansible/pull/11554
New PR: https://github.com/openshift/openshift-ansible/pull/11681
Hi, Joseph Yes, you're right. It works well in a fresh cluster. Maybe someone changed something in that old cluster. Thanks! Details: mac:openshift-ansible jianzhang$ ansible-playbook -i qe-inventory-host-file playbooks/redeploy-certificates.yml -vvv ... PLAY RECAP ************************************************************************************************************************************************** localhost : ok=16 changed=0 unreachable=0 failed=0 vm-10-0-77-86.hosted.upshift.rdu2.redhat.com : ok=231 changed=78 unreachable=0 failed=0 INSTALLER STATUS ******************************************************************************************************************************************** Initialization : Complete (0:01:05) Thursday 20 June 2019 10:13:04 +0800 (0:00:35.639) 0:12:29.888 ********* =============================================================================== Gathering Facts ------------------------------------------------------------------------------------------------------------------------------------- 37.53s /Users/jianzhang/project/openshift-ansible/playbooks/init/basic_facts.yml:2 -------------------------------------------------------------------------------- openshift_service_catalog : Verify that the apiserver is running ------------------------------------------------------------------------------------ 37.04s /Users/jianzhang/project/openshift-ansible/roles/openshift_service_catalog/tasks/restart_pods.yml:19 ------------------------------------------------------- openshift_service_catalog : Verify that the controller-manager is running --------------------------------------------------------------------------- 35.98s /Users/jianzhang/project/openshift-ansible/roles/openshift_service_catalog/tasks/restart_pods.yml:40 ------------------------------------------------------- template_service_broker : Verify that the apiserver is running -------------------------------------------------------------------------------------- 35.64s /Users/jianzhang/project/openshift-ansible/roles/template_service_broker/tasks/restart_pods.yml:9 ---------------------------------------------------------- Remove generated certificates ----------------------------------------------------------------------------------------------------------------------- 29.03s /Users/jianzhang/project/openshift-ansible/playbooks/openshift-master/private/certificates-backup.yml:28 --------------------------------------------------- ansible_service_broker : Verify that the ASB is running --------------------------------------------------------------------------------------------- 24.32s /Users/jianzhang/project/openshift-ansible/roles/ansible_service_broker/tasks/restart_pods.yml:20 ---------------------------------------------------------- openshift_control_plane : verify API server --------------------------------------------------------------------------------------------------------- 18.66s /Users/jianzhang/project/openshift-ansible/roles/openshift_control_plane/handlers/main.yml:13 -------------------------------------------------------------- openshift_control_plane : verify API server --------------------------------------------------------------------------------------------------------- 17.14s /Users/jianzhang/project/openshift-ansible/roles/openshift_control_plane/handlers/main.yml:13 -------------------------------------------------------------- openshift_console : Copy console templates to temp directory ---------------------------------------------------------------------------------------- 15.64s /Users/jianzhang/project/openshift-ansible/roles/openshift_console/tasks/install.yml:19 -------------------------------------------------------------------- openshift_console : Waiting for console rollout to complete ----------------------------------------------------------------------------------------- 14.76s /Users/jianzhang/project/openshift-ansible/roles/openshift_console/tasks/start.yml:2 ----------------------------------------------------------------------- template_service_broker : Remove apiserver pods ----------------------------------------------------------------------------------------------------- 14.32s /Users/jianzhang/project/openshift-ansible/roles/template_service_broker/tasks/restart_pods.yml:2 ---------------------------------------------------------- etcd : restart etcd --------------------------------------------------------------------------------------------------------------------------------- 12.01s /Users/jianzhang/project/openshift-ansible/roles/etcd/tasks/restart.yml:2 ---------------------------------------------------------------------------------- openshift_service_catalog : Generating API Server keys ---------------------------------------------------------------------------------------------- 11.61s /Users/jianzhang/project/openshift-ansible/roles/openshift_service_catalog/tasks/generate_certs.yml:29 ----------------------------------------------------- Wait for master API to come back online ------------------------------------------------------------------------------------------------------------- 11.51s /Users/jianzhang/project/openshift-ansible/playbooks/openshift-node/private/restart.yml:54 ----------------------------------------------------------------- Remove web console pods ----------------------------------------------------------------------------------------------------------------------------- 10.87s /Users/jianzhang/project/openshift-ansible/playbooks/openshift-web-console/private/redeploy-certificates.yml:16 -------------------------------------------- etcd : Retrieve etcd ca cert tarball ---------------------------------------------------------------------------------------------------------------- 10.34s /Users/jianzhang/project/openshift-ansible/roles/etcd/tasks/certificates/fetch_server_certificates_from_ca.yml:165 ----------------------------------------- etcd : template -------------------------------------------------------------------------------------------------------------------------------------- 9.45s /Users/jianzhang/project/openshift-ansible/roles/etcd/tasks/certificates/deploy_ca.yml:32 ------------------------------------------------------------------ etcd : Unarchive cert tarball ------------------------------------------------------------------------------------------------------------------------ 9.27s /Users/jianzhang/project/openshift-ansible/roles/etcd/tasks/certificates/fetch_server_certificates_from_ca.yml:149 ----------------------------------------- etcd : copy ------------------------------------------------------------------------------------------------------------------------------------------ 8.95s /Users/jianzhang/project/openshift-ansible/roles/etcd/tasks/certificates/deploy_ca.yml:63 ------------------------------------------------------------------ openshift_hosted : Create OpenShift router ----------------------------------------------------------------------------------------------------------- 8.46s /Users/jianzhang/project/openshift-ansible/roles/openshift_hosted/tasks/router.yml:85 ---------------------------------------------------------------------- These secrets(asb-client, asb-tls, apiserver-serving-cert, templateservicebroker-client) have been updated as expected. As below: [root@qe-phunt-preserve-merrn-1 ~]# oc get secret -n openshift-ansible-service-broker NAME TYPE DATA AGE asb-client kubernetes.io/service-account-token 4 5m asb-client-dockercfg-rrplx kubernetes.io/dockercfg 1 6h asb-client-token-8t55n kubernetes.io/service-account-token 4 6h asb-client-token-hdp2p kubernetes.io/service-account-token 4 6h asb-dockercfg-lzknv kubernetes.io/dockercfg 1 6h asb-registry-auth Opaque 2 6h asb-tls kubernetes.io/tls 2 5m asb-token-gg4g7 kubernetes.io/service-account-token 4 6h ... [root@qe-phunt-preserve-merrn-1 ~]# oc get secret -n openshift-template-service-broker NAME TYPE DATA AGE apiserver-dockercfg-qrprr kubernetes.io/dockercfg 1 6h apiserver-serving-cert kubernetes.io/tls 2 5m apiserver-token-67drq kubernetes.io/service-account-token 4 6h apiserver-token-dpdmr kubernetes.io/service-account-token 4 6h builder-dockercfg-5288k kubernetes.io/dockercfg 1 6h builder-token-jxlbd kubernetes.io/service-account-token 4 6h builder-token-mfw2z kubernetes.io/service-account-token 4 6h default-dockercfg-gfzpm kubernetes.io/dockercfg 1 6h default-token-7pz2t kubernetes.io/service-account-token 4 6h default-token-bvjv8 kubernetes.io/service-account-token 4 6h deployer-dockercfg-7v548 kubernetes.io/dockercfg 1 6h deployer-token-bmghk kubernetes.io/service-account-token 4 6h deployer-token-dcdfk kubernetes.io/service-account-token 4 6h templateservicebroker-client kubernetes.io/service-account-token 4 5m ... And, the ASB/TSB service works well! [root@qe-phunt-preserve-merrn-1 ~]# curl -vvv --cacert /etc/origin/master/service-signer.crt https://asb.openshift-ansible-service-broker.svc:1338 * About to connect() to asb.openshift-ansible-service-broker.svc port 1338 (#0) * Trying 172.30.7.188... * Connected to asb.openshift-ansible-service-broker.svc (172.30.7.188) port 1338 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/origin/master/service-signer.crt CApath: none * SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 * Server certificate: * subject: CN=asb.openshift-ansible-service-broker.svc * start date: Jun 20 02:11:19 2019 GMT * expire date: Jun 19 02:11:20 2021 GMT * common name: asb.openshift-ansible-service-broker.svc * issuer: CN=openshift-service-serving-signer@1560996300 > GET / HTTP/1.1 > User-Agent: curl/7.29.0 > Host: asb.openshift-ansible-service-broker.svc:1338 > Accept: */* > < HTTP/1.1 200 OK < Content-Type: application/json < Date: Thu, 20 Jun 2019 02:17:57 GMT < Content-Length: 162 < { "paths": [ "/apis", "/healthz", "/healthz/ping", "/healthz/poststarthook/generic-apiserver-start-informers", "/metrics", "/osb/" ] * Connection #0 to host asb.openshift-ansible-service-broker.svc left intact [root@qe-phunt-preserve-merrn-1 ~]# curl -vvv --cacert /etc/origin/master/service-signer.crt https://apiserver.openshift-template-service-broker.svc:443 * About to connect() to apiserver.openshift-template-service-broker.svc port 443 (#0) * Trying 172.30.241.218... * Connected to apiserver.openshift-template-service-broker.svc (172.30.241.218) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/origin/master/service-signer.crt CApath: none * NSS: client certificate not found (nickname not specified) * SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 * Server certificate: * subject: CN=apiserver.openshift-template-service-broker.svc * start date: Jun 20 02:12:05 2019 GMT * expire date: Jun 19 02:12:06 2021 GMT * common name: apiserver.openshift-template-service-broker.svc * issuer: CN=openshift-service-serving-signer@1560996300 > GET / HTTP/1.1 > User-Agent: curl/7.29.0 > Host: apiserver.openshift-template-service-broker.svc > Accept: */* > < HTTP/1.1 200 OK < Content-Type: application/json < Date: Thu, 20 Jun 2019 02:31:57 GMT < Content-Length: 214 < { "paths": [ "/apis", "/brokers/template.openshift.io/v2", "/healthz", "/healthz/log", "/healthz/ping", "/healthz/poststarthook/template-service-broker-synctemplates", "/metrics" ] * Connection #0 to host apiserver.openshift-template-service-broker.svc left intact Verify it, thanks! mac:openshift-ansible jianzhang$ ansible --version ansible 2.6.17.post0 config file = /Users/jianzhang/project/openshift-ansible/ansible.cfg configured module search path = [u'/Users/jianzhang/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/local/lib/python2.7/site-packages/ansible-2.6.17.post0-py2.7.egg/ansible executable location = /usr/local/bin/ansible python version = 2.7.15 (default, Nov 27 2018, 21:40:55) [GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)]
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1753