STF | In the latest OSP13 client side can not connect to the server side. SSL errors seen in the logs. Errors from tne metrics_qdr.log: 2021-03-03 00:02:42.681359 +0000 SERVER (info) [C3788] Connection to default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 failed: amqp:connection:framing-error SSL Failure: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed 2021-03-03 00:02:48.055460 +0000 SERVER (info) [C3789] Connection to default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 failed: amqp:connection:framing-error SSL Failure: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed 2021-03-03 00:02:53.422553 +0000 SERVER (info) [C3790] Connection to default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 failed: amqp:connection:framing-error SSL Failure: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed conf file that is used: [root@undercloud-0 virt]# cat stf-connectors.yaml parameter_defaults: CeilometerQdrPublishEvents: true CollectdAmqpInstances: cloud3-notify: format: JSON notify: true presettle: false cloud3-telemetry: format: JSON presettle: false CollectdAmqpInterval: 5 CollectdConnectionType: amqp1 CollectdDefaultPollingInterval: 5 CollectdDefaultPlugins: - cpu - df - load - connectivity - intel_rdt - ipmi - procevent MetricsQdrSSLProfiles: - name: sslProfile MetricsQdrConnectors: - host: default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com port: 443 role: edge verifyHostname: false sslProfile: sslProfile
*** Bug 1936560 has been marked as a duplicate of this bug. ***
Attached patch enables TripleO to distribute SSL certificates and hence enables STF client side message bus to connect to server side message bus even with latest AMQ Interconnect versions
To extract OCP certificate run the following command on your OCP: 1.oc get secrets - (you will get a list) You should see in the list default-interconnect-selfsigned 2.oc get secret/default-interconnect-selfsigned -o jsonpath='{.data.ca\.crt}' | base64 -d You will get a decode of the certificate that will look like: -----BEGIN CERTIFICATE----- MIIDpjCCAo6gAwIBAgIQFE4Z7BXVg+paMsLqKl4fnzANBgkqhkiG9w0BAQsFADBf MRUwEwYDVQQKEwxjZXJ0LW1hbmFnZXIxQTA/BgNVBAMTOGRlZmF1bHQtaW50ZXJj b25uZWN0LnNlcnZpY2UtdGVsZW1ldHJ5LnN2Yy5jbHVzdGVyLmxvYeFsMB4XDTIx MDQwODEzMTAwMFoXDTIxMDcwNzEzMTAwMFowWjEVMBMGA1UEChMMY2VydC1tYW5h Z2VyMUEwPwYDVQQDEzhkZWZhdWx0LWludGVyY29ubmVjdC5zZXJ2aWNlLXRlbGVt ZXRyeS5zdmMuY2x1c3Rlci5sb2NhbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC AQoCggEBALB8oTZJuLT0jy2wf4tkiLuxY58Uo0KiHh+dMXWscZ1voLEeATLIoyrK eUa2sBmPBsHdLt4nO34nEeBnNzoKQR8XLc5F8x4WM8mQPv0KjpWJPeskPvvAfeWT XcXBxf5af7HvtzeA+zL2onucaBeEcAbNIEm4Elz3d/BT70w1z235J3a2JMACFGcy kZVS74PxXjx65b57rPbtf0Gnlf0cSObHfJ8n3N7tlAMgfErjGByHLDuEQ8nxzh0N EZaC3yhbLE0IdhLM0V+WPvkUQvrflL2x2K0zboKpcLbGdRc99UjbViLTM6E3prx+ GzEqcExDyMWX4kdhd+rBJHpuZ2AVhPMCAwEAAaNoMGYwDgYDVR0PAQH/BAQDAgKk MA8GA1UdEwEB/wQFMAMBAf8wQwYDVR0RBDwwOoI4ZGVmYXVsdC1pbnRlcmNvbm5l Y3Quc2VydmljZS10ZWxlbWV0cnkuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcN AQELBQADggEBAIi1P31PtEk9nCSKBiPtOGl+RBHZlhoIMPZhLAs1BCmPnjUdBjpq wS6IjihePVSX7mfb5o4TJTz3qlx/OfiyfWZ2+jKttJ8hOEjaxdspEnJ9n4ska9BP eEHM5Xu5djIXRJKHihcrzmecFCMTc9R9kkHqQI8cithG4aXeg8I/KsZ1Isfa4w8y lDNS0Zwk7mf40b5kwG83VB+dfiqvsO/ODTw7xu7aAr8TtnnMoHFfnw/wqr3XEJzw pwBOkI9C7cq7unApiPBfomQFEXIWdTaU/7sy1Dt238rt/sWbznwcXu3M673oq0JV TOpbgNlPcz0sIoNK3Be0jUcYHVMPKGMR2kk= -----END CERTIFICATE----- 3 Edit stf custom template e.g. (stf-connectors.yaml) and add the following content: MetricsQdrSSLProfiles: - name: sslProfile caCertFileContent: | -----BEGIN CERTIFICATE----- MIIDpjCCAo6gAwIBAgIQFE4Z7BXVg+paMsLqKl4fnzANBgkqhkiG9w0BAQsFADBa MRUwEwYDVQQKEwxjZXJ0LW1hbmFnZXIxQTA/BgNVBAMTOGRlZmF1bHQtaW50ZXJj b25uZWN0LnNlcnZpY2UtdGVsZW1ldHJ5LnN2Yy5jbHVzdGVyLmxvY2FsMB4XDTIx MDQwODEzMTAwMFoXDTIxMDcwNzEzMTAwMFowWjEVMBMGA1UEChMMY2VydC1tYW5h Z2VyMUEwPwYDVQQDEzhkZWZhdWx0LWludGVyY29ubmVjdC5zZXJ2aWNlLXRlbGVt ZXRyeS5zdmMuY2x1c3Rlci5sb2NhbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC AQoCggEBALB8oTZJuLT0jy2wf4tkiLuxY58Uo0KiHh+dMXWbcZ1voLEeATLIoyrK eUa2sBmPBsHdLt4nO34nEeBnNzoKQR8XLc5F8x4WM8mQPv0KsdfJPeskPvvAfeWT XcXBxf5af7HvtzeA+zL2onucaBeEcAbNIEm4Elz3d/BT70w1z235J3a2JMACFGcy kZVS74PxXjx65b57rPbtf0Gnlf0cSObHfJ8n3N7tlAMgfErjGByHLDuEQ8nxzh0N EZaC3yhbLE0IdhLM0V+WPvkUQvrflL2x2K0zboKpcLbGdRc9ertbViLTM6E3prx+ GzEqcExDyMWX4kdhd+rBJHpuZ2AVhPMCAwEAAaNoMGYwDgYDVR0PAQH/BAQDAgKk MA8GA1UdEwEB/wQFMAMBAf8wQwYDVR0RBDwwOoI4ZGVh7tVsdC1pbnRlcmNvbm5l Y3Quc2VydmljZS10ZWxlbWV0cnkuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcN AQELBQADggEBAIi1P31PtEk9nCSKBiPtOGl+RBHZlhoIMPZhLAs1BCmPnjUdBjpq wS6IjihePVSX7mfb5o4TJTz3qlx/OfiyfWZ2+jKttJ8hOEjaxdspEnJ9n4ska9BP eEHM5Xu5djIXRJKHihcrzmecFCMTc9R9kkHqQI8cithG4aXeg8I/KsZ1Isfa4w8y lDNS0Zwk7mf40b5kwG83VB+dfiqvsO/ODTw7xu7aAr8TtnnMoHFfnw/wqr3XEJzw pwBOkI9C7cq7unApiPBfomQFEXIWdTaU/7sy1Dt238rt/sWbznwcXu3M673oq0JV TOpbgNlPcz0sIoNK3Be0jUcYHVMPKGMR2kk= -----END CERTIFICATE----- 4. Deploy overcloud. ------------- BZ verified: OSP was able to connect to the OCP 2021-05-07 09:40:32.390553 +0000 CONN_MGR (info) Created SSL Profile with name sslProfile 2021-05-07 09:40:32.399273 +0000 CONN_MGR (info) Configured Connector: default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 proto=any, role=edge, sslProfile=sslProfile 2021-05-07 09:40:32.401534 +0000 CONN_MGR (info) Configured Listener: 172.17.1.106:5666 proto=any, role=normal 2021-05-07 09:40:32.403365 +0000 SERVER (notice) Operational, 4 Threads Running (process ID 9) 2021-05-07 09:40:32.404051 +0000 SERVER (notice) Process VmSize 207.77 MiB (31.26 GiB available memory) 2021-05-07 09:40:32.404493 +0000 SERVER (notice) Listening on 172.17.1.106:5666 2021-05-07 09:40:33.297709 +0000 ROUTER (info) [C1] Connection Opened: dir=out host=default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 vhost= encrypted=TLSv1/SSLv3 auth=ANONYMOUS user=(null) container_id=default-interconnect-69d77454b5-wqbp4 props={:product="qpid-dispatch-router", :version="Red Hat Interconnect 1.10.0 (qpid-dispatch 1.14.0)", :"qd.conn-id"=953349} 2021-05-07 09:40:33.297818 +0000 ROUTER_CORE (info) Edge connection (id=1) to interior established 2021-05-07 09:40:33.297997 +0000 ROUTER_CORE (info) [C1][L6] Link attached: dir=out source={<none> expire:link} target={<none> expire:link} 2021-05-07 09:40:33.298085 +0000 ROUTER_CORE (info) [C1][L7] Link attached: dir=in source={Router.controller-0.redhat.local expire:link caps::"qd.router-edge-downlink"} target={<none> expire:link caps::"qd.router-edge-downlink"} 2021-05-07 09:40:33.298124 +0000 ROUTER_CORE (info) [C1][L8] Link attached: dir=in source={_$qd.edge_addr_tracking expire:link} target={<none> expire:link} 2021-05-07 09:40:33.298177 +0000 ROUTER_CORE (info) [C1][L9] Link attached: dir=out source={} target={$management expire:link} 2021-05-07 09:40:33.298190 +0000 ROUTER_CORE (info) [C1][L10] Link attached: dir=in source={<dynamic> expire:link} target={} 2021-05-07 09:40:33.298208 +0000 ROUTER_CORE (info) [C1][L11] Link attached: dir=out source={} target={_$qd.addr_lookup expire:link} 2021-05-07 09:40:33.298219 +0000 ROUTER_CORE (info) [C1][L12] Link attached: dir=in source={<dynamic> expire:link} target={} 2021-05-07 09:40:34.894964 +0000 SERVER (info) [C2] Accepted connection to 172.17.1.106:5666 from 172.17.1.106:46324 2021-05-07 09:40:34.901265 +0000 ROUTER (info) [C2] Connection Opened: dir=in host=172.17.1.106:46324 vhost= encrypted=no auth=no user=anonymous container_id=openstack.org/om/container/controller-0/ceilometer-agent-notification/28/2ad8e44547724d6db23838a290f15a64 props={:process=b"ceilometer-agent-notification", :node=b"controller-0", :pid=28} 2021-05-07 09:40:34.908614 +0000 ROUTER_CORE (info) [C2][L13] Link attached: dir=out source={<dynamic> expire:sess} target={rpc-response expire:sess} 2021-05-07 09:40:34.917686 +0000 ROUTER_CORE (info) [C2][L14] Link attached: dir=in source={/anycast/ceilometer/cloud2-metering.sample expire:sess} target={/anycast/ceilometer/cloud2-metering.sample expire:sess} 2021-05-07 09:40:35.060918 +0000 ROUTER_CORE (info) [C1][L15] Link attached: dir=out source={<none> expire:link} target={anycast/ceilometer/cloud2-metering.sample expire:link} 2021-05-07 09:40:39.314158 +0000 SERVER (info) [C3] Accepted connection to 172.17.1.106:5666 from 172.17.1.106:46590 2021-05-07 09:40:39.315462 +0000 ROUTER (info) [C3] Connection Opened: dir=in host=172.17.1.106:46590 vhost= encrypted=no auth=ANONYMOUS user=anonymous container_id=metrics props= 2021-05-07 09:40:39.315656 +0000 ROUTER_CORE (info) [C3][L16] Link attached: dir=in source={<none> expire:sess} target={<none> expire:sess}
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 13.0 bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:2385