in bug 1791970 / https://review.openstack.org/#/c/601677/ the path to the CA cert used by zaqar is hardcoded to /etc/pki/ca-trust/source/anchors/cm-local-ca.pem. this conflicts with the use case when one is using the "undercloud_service_certificate" parameter in undercloud.conf which seemingly should set up a different path for this. I'm observing the use of this parameter when using infrared to generate an RDO undercloud. Per https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ssl.html, "However, it is possible to not use certmonger’s local CA.". When I use infrared to make an undercloud, it creates a new self-signed CA defaulting to /etc/pki/ca-trust/source/anchors/undercloud-cacert.pem; this code is at https://github.com/redhat-openstack/infrared/blob/master/plugins/tripleo-undercloud/tasks/ssl.yml#L32 . Then it sets "undercloud_service_certificate = /etc/pki/instack-certs/undercloud.pem" in undercloud.conf and deploys the undercloud, following the instructions in https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ssl.html. However, when I go to introspect nodes, I get the stack trace described in bug 1791970: Could not establish a connection to the Zaqar websocket. The command was sent but the answer could not be read. # ... File "/usr/lib64/python2.7/ssl.py", line 831, in do_handshake self._sslobj.do_handshake() SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618) to repair this, I just copy the undercloud-cacert.pem on top of the cm-local-ca.pem file: [root@undercloud-0 ~]# cd /etc/pki/ca-trust/source/anchors/ [root@undercloud-0 anchors]# ls cm-local-ca.pem undercloud-cacert.pem [root@undercloud-0 anchors]# cp cm-local-ca.pem cm-local-ca.pem.saved [root@undercloud-0 anchors]# cp undercloud-cacert.pem cm-local-ca.pem cp: overwrite ‘cm-local-ca.pem’? y I'm not sure how this should work, if tripleo is doing the wrong thing or if infrared is but it seems one side or the other needs to be changed.
*** Bug 1703031 has been marked as a duplicate of this bug. ***
Verified by CI, phase1/2 jobs are passing with fixed package included.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0878
Still seeing at : https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/pidone/view/updates/job/DFG-pidone-updates-14_director-rhel-virthost-3cont_2comp_3ceph-ipv4-vxlan-sanity/24/console TASK [register hosts to instack and configure boot] **************************** task path: /home/rhos-ci/jenkins/workspace/DFG-pidone-updates-14_director-rhel-virthost-3cont_2comp_3ceph-ipv4-vxlan-sanity/infrared/plugins/tripleo-overcloud/introspect.yml:79 Thursday 06 June 2019 06:35:46 +0000 (0:00:00.142) 0:02:36.776 ********* fatal: [undercloud-0]: FAILED! => { "changed": true, "cmd": "set -eo pipefail\n source ~/stackrc\n\n openstack overcloud node import --instance-boot-option=local /home/stack/instackenv.json\n ", "delta": "0:00:03.073003", "end": "2019-06-06 02:35:50.415534", "rc": 1, "start": "2019-06-06 02:35:47.342531" } STDERR: Could not establish a connection to the Zaqar websocket. The command was sent but the answer could not be read. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)
logs at : https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/pidone/view/updates/job/DFG-pidone-updates-14_director-rhel-virthost-3cont_2comp_3ceph-ipv4-vxlan-sanity/24/artifact/