Description of problem: `apb relist --ensure --ca-path` failure. This cause `apb remove --secure --ca-path` and `apb bootstrap --secure --ca-path` all retun failure. Version-Release number of selected component (if applicable): openshift v3.10.0-0.50.0 Version: apb-1.2.8 How reproducible: Always Steps to Reproduce: 1. Deploy servicecatalog and ASB in OCP cluster. 2. #apb relist --secure -ca-path /etc/origin/master/ca.crt Actual results: `apb relist` work, `apb relist --secure -ca-path /etc/origin/master/ca.crt` return failure. # apb relist Successfully relisted the Service Catalog # apb relist --ca-path /etc/origin/master/ca.crt Successfully relisted the Service Catalog # apb relist --secure Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) # apb remove --secure --ca-path /etc/origin/master/ca.crt --id 02461192cf28f69c1fb792f32cb58a74 Contacting the ansible-service-broker at: https://asb-1338-openshift-ansible-service-broker.apps.0523-lsx.qe.rhcloud.com/ansible-service-broker/v2/apb/02461192cf28f69c1fb792f32cb58a74 Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) # apb bootstrap --secure --ca-path /etc/origin/master/ca.crt Contacting the ansible-service-broker at: https://asb-1338-openshift-ansible-service-broker.apps.0523-lsx.qe.rhcloud.com/ansible-service-broker/v2/bootstrap Successfully bootstrapped Ansible Service Broker Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) # apb list --secure --ca-path /etc/origin/master/ca.crt Contacting the ansible-service-broker at: https://asb-1338-openshift-ansible-service-broker.apps.0523-lsx.qe.rhcloud.com/ansible-service-broker/v2/catalog ID NAME DESCRIPTION 0cd794ef27f565cc0e755585dbbcdcdc local-my-01-apb This is a sample application generated by apb init 02461192cf28f69c1fb792f32cb58a74 local-my-02-apb This is a sample application generated by apb init Expected results: `apb relist --secure --ca-path` could work normally. Additional info:
https://github.com/ansibleplaybookbundle/ansible-playbook-bundle/pull/294
https://errata.devel.redhat.com/advisory/33505 moved to QE openshift-enterprise-asb-container-v3.10.0-0.51.0.1 openshift-enterprise-mediawiki-apb-v3.10.0-0.51.0.1 openshift-enterprise-postgresql-apb-v3.10.0-0.51.0.1 openshift-enterprise-mysql-apb-v3.10.0-0.51.0.1 openshift-enterprise-mariadb-apb-v3.10.0-0.51.0.1 openshift-enterprise-apb-tools-v3.10.0-0.32.0.2
verified failed. # apb version Version: apb-1.2.10 [root@ip-172-18-11-188 ~]# apb relist Successfully relisted the Service Catalog [root@ip-172-18-11-188 ~]# apb relist --secure --ca-path /etc/origin/master/ca.crt Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) [root@ip-172-18-11-188 ~]# apb bootstrap --secure --ca-path /etc/origin/master/ca.crt Contacting the ansible-service-broker at: https://asb-1338-openshift-ansible-service-broker.apps.0528-l3l.qe.rhcloud.com/ansible-service-broker/v2/bootstrap Successfully bootstrapped Ansible Service Broker Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) [root@ip-172-18-11-188 ~]# apb remove --id 0cd794ef27f565cc0e755585dbbcdcdc --secure --ca-path /etc/origin/master/ca.crt Contacting the ansible-service-broker at: https://asb-1338-openshift-ansible-service-broker.apps.0528-l3l.qe.rhcloud.com/ansible-service-broker/v2/apb/0cd794ef27f565cc0e755585dbbcdcdc Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) Successfully deleted APB
Okay, I have come to the same conclusion that /etc/origin/master/ca.crt does not work for `apb relist`. I did confirm that I can get `apb relist --ca-path` to succeed if I do the following: $ oc get secret apiserver-ssl -n kube-service-catalog -o json | jq .data | jq '.["tls.crt"]' | sed -e 's/^"//' -e 's/"$//' >> ca.crt $ apb relist --ca-path ./ca.crt Successfully relisted the Service Catalog This means we have to pass in two CAs for relist to work, which is automatically called on `apb bootstrap` and `apb push`.
Fixed the above command: $ oc get secret apiserver-ssl -n kube-service-catalog -o json | jq .data | jq '.["tls.crt"]' | sed -e 's/^"//' -e 's/"$//' | base64 -d >> ca.crt
sunzhaohua, Can you please confirm for me if the above command works if you grab the TLS secret? You may also be able to get this cert from /etc/origin/kube-apiserver (? unsure if this exists, on my machine using oc cluster up its a folder). I would append the CAs to one file and use that as the argument to --ca-path.
Dylan, 1) If I use "/etc/origin/master/ca.crt", "apb relist --ca-path master.crt " return successful. "apb relist --secure --ca-path ca.crt " return failure. Both local and remote the results are same. 2)If I use "/etc/origin/service-catalog/ca.crt", "apb relist --ca-path master.crt " and "apb relist --secure --ca-path ca.crt " all return failure. 1)copy "/etc/origin/master/ca.crt" to master.crt [szh@localhost ~]$ apb relist --ca-path master.crt Successfully relisted the Service Catalog [szh@localhost ~]$ apb relist --secure --ca-path ca.crt Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) 2) [szh@localhost ~]$ oc get secret apiserver-ssl -n kube-service-catalog -o json | jq .data | jq '.["tls.crt"]' | sed -e 's/^"//' -e 's/"$//' | base64 -d >> ca.crt [szh@localhost ~]$ apb relist --ca-path ca.crt Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) copy "/etc/origin/service-catalog/ca.crt" to api.crt. Run remotely [szh@localhost ~]$ apb relist --secure --ca-path api.crt Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) [szh@localhost ~]$ apb relist --ca-path api.crt Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) run locally. [root@qe-zhsun-gceemaster-etcd-1 service-catalog]# apb relist --ca-path ca.crt Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) [root@qe-zhsun-gceemaster-etcd-1 service-catalog]# apb relist --secure --ca-path apiserver.crt Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)
sunzhaohua, I'm confused... you said in the previous comment that running: $ apb relist --secure --ca-path /etc/origin/master/ca.crt Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) resulted in failure. Yet in the last comment you said copying it over and running it succeeded? 1)copy "/etc/origin/master/ca.crt" to master.crt [szh@localhost ~]$ apb relist --ca-path master.crt Successfully relisted the Service Catalog These should be the exact CAs so I'm confused why that would fail in one and not the other. You can ignore my attempt to grab it from the secret since that did not work in your environment. If using /etc/origin/master/ca.crt succeeds then it would mean that the same CA works for all of the apb commands. Can you test all of the commands with the CA that gives you a success for relist? (/etc/origin/master/ca.crt or master.crt as you renamed it).
Moving to ON_QA to please retest using the proper CA.
Dylan, Sorry for the confusing. What I mean is if "apb relist" command with option "--secure --ca-path" it return failue, while with option "--ca-path" return successful. Refer to apb list with option "--ensure --ca-path" and "--ca-path" all return successful. Expected results is apb relist with option "--secure --ca-path" could work normally. 1) apb list with "--ensure --ca-path" and "--ca-path" option all run noramlly. # apb list --ca-path /etc/origin/master/ca.crt No APBs found # apb list --secure --ca-path /etc/origin/master/ca.crt No APBs found 2) apb relist with option "--ca-path" reture successful. With option "--secure --ca-path" return failure. # apb relist --ca-path /etc/origin/master/ca.crt Successfully relisted the Service Catalog # apb relist --secure --ca-path /etc/origin/master/ca.crt Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) 3) apb bootstrap with option "--ca-path" reture successful. With option "--secure --ca-path" return failure. # apb bootstrap --ca-path /etc/origin/master/ca.crt Successfully bootstrapped Ansible Service Broker Successfully relisted the Service Catalog # apb bootstrap --secure --ca-path /etc/origin/master/ca.crt Successfully bootstrapped Ansible Service Broker Relist failure: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)
Thanks for clarifying! I have found the issue, that helped clear things up.
https://github.com/ansibleplaybookbundle/ansible-playbook-bundle/pull/299
verified successful. $ apb relist --secure --ca-path /etc/origin/master/master.crt Successfully relisted the Service Catalog $ apb bootstrap --secure --ca-path /etc/origin/master/master.crt Contacting the ansible-service-broker at: https://asb-1338-openshift-ansible-service-broker.apps.0605-va3.qe.rhcloud.com/ansible-service-broker/v2/bootstrap Successfully bootstrapped Ansible Service Broker Successfully relisted the Service Catalog