Bug 1934440 - STF | In the latest OSP13 client side can not connect to the server side. SSL errors seen in the logs.
Summary: STF | In the latest OSP13 client side can not connect to the server side. SSL...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: puppet-tripleo
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: z16
: 13.0 (Queens)
Assignee: Martin Magr
QA Contact: Leonid Natapov
URL:
Whiteboard:
: 1936560 (view as bug list)
Depends On:
Blocks: 1949168 1954722 1982764
TreeView+ depends on / blocked
 
Reported: 2021-03-03 09:24 UTC by Leonid Natapov
Modified: 2023-01-03 21:32 UTC (History)
13 users (show)

Fixed In Version: puppet-tripleo-8.5.1-23.el7ost
Doc Type: Bug Fix
Doc Text:
Before this update, the Service Telemetry Framework (STF) client could not connect to the STF server, because the latest version of Red Hat AMQ Interconnect does not allow TLS connections without a CA certificate. + This update corrects this problem by providing a new Orchestration service (heat) parameter, `MetricsQdrSSLProfiles`. + To obtain a Red Hat OpenShift TLS certificate, enter these commands: + ---- $ oc get secrets $ oc get secret/default-interconnect-selfsigned -o jsonpath='{.data.ca\.crt}' | base64 -d ---- + Add the `MetricsQdrSSLProfiles` parameter with the contents of your Red Hat OpenShift TLS certificate to a custom environment file: + ---- MetricsQdrSSLProfiles: - name: sslProfile caCertFileContent: | -----BEGIN CERTIFICATE----- ... TOpbgNlPcz0sIoNK3Be0jUcYHVMPKGMR2kk= -----END CERTIFICATE----- ---- + Then, redeploy your overcloud with the `openstack overcloud deploy` command.
Clone Of:
: 1949168 (view as bug list)
Environment:
Last Closed: 2021-06-16 10:58:58 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 783648 0 None MERGED Create SSL certificates from sslProfiles 2023-04-26 06:20:17 UTC
Red Hat Issue Tracker OSP-468 0 None None None 2022-10-03 14:32:32 UTC
Red Hat Product Errata RHBA-2021:2385 0 None None None 2021-06-16 10:59:29 UTC

Description Leonid Natapov 2021-03-03 09:24:54 UTC
STF | In the latest OSP13 client side can not connect to the server side. SSL errors seen in the logs.

Errors from tne metrics_qdr.log:
2021-03-03 00:02:42.681359 +0000 SERVER (info) [C3788] Connection to default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 failed: amqp:connection:framing-error SSL Failure: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
2021-03-03 00:02:48.055460 +0000 SERVER (info) [C3789] Connection to default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 failed: amqp:connection:framing-error SSL Failure: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
2021-03-03 00:02:53.422553 +0000 SERVER (info) [C3790] Connection to default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 failed: amqp:connection:framing-error SSL Failure: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed


conf file that is used:

[root@undercloud-0 virt]# cat stf-connectors.yaml 
    parameter_defaults:
        CeilometerQdrPublishEvents: true

        CollectdAmqpInstances:
            cloud3-notify:
                format: JSON
                notify: true
                presettle: false
            cloud3-telemetry:
                format: JSON
                presettle: false
        CollectdAmqpInterval: 5
        CollectdConnectionType: amqp1
        CollectdDefaultPollingInterval: 5

        CollectdDefaultPlugins:
            - cpu
            - df
            - load
            - connectivity
            - intel_rdt
            - ipmi
            - procevent

        MetricsQdrSSLProfiles:
            - name: sslProfile

        MetricsQdrConnectors:
            - host: default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com
              port: 443
              role: edge
              verifyHostname: false
              sslProfile: sslProfile

Comment 9 Martin Magr 2021-03-23 11:15:12 UTC
*** Bug 1936560 has been marked as a duplicate of this bug. ***

Comment 18 Martin Magr 2021-04-07 12:55:32 UTC
Attached patch enables TripleO to distribute SSL certificates and hence enables STF client side message bus to connect to server side message bus even with latest AMQ Interconnect versions

Comment 35 Leonid Natapov 2021-05-07 10:23:15 UTC
To extract OCP certificate run the following command on your OCP:

1.oc get secrets  - (you will get a list)

You should see in the list  default-interconnect-selfsigned 


2.oc get secret/default-interconnect-selfsigned -o jsonpath='{.data.ca\.crt}' | base64 -d

You will get a decode of the certificate that will look like:

-----BEGIN CERTIFICATE-----
MIIDpjCCAo6gAwIBAgIQFE4Z7BXVg+paMsLqKl4fnzANBgkqhkiG9w0BAQsFADBf
MRUwEwYDVQQKEwxjZXJ0LW1hbmFnZXIxQTA/BgNVBAMTOGRlZmF1bHQtaW50ZXJj
b25uZWN0LnNlcnZpY2UtdGVsZW1ldHJ5LnN2Yy5jbHVzdGVyLmxvYeFsMB4XDTIx
MDQwODEzMTAwMFoXDTIxMDcwNzEzMTAwMFowWjEVMBMGA1UEChMMY2VydC1tYW5h
Z2VyMUEwPwYDVQQDEzhkZWZhdWx0LWludGVyY29ubmVjdC5zZXJ2aWNlLXRlbGVt
ZXRyeS5zdmMuY2x1c3Rlci5sb2NhbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBALB8oTZJuLT0jy2wf4tkiLuxY58Uo0KiHh+dMXWscZ1voLEeATLIoyrK
eUa2sBmPBsHdLt4nO34nEeBnNzoKQR8XLc5F8x4WM8mQPv0KjpWJPeskPvvAfeWT
XcXBxf5af7HvtzeA+zL2onucaBeEcAbNIEm4Elz3d/BT70w1z235J3a2JMACFGcy
kZVS74PxXjx65b57rPbtf0Gnlf0cSObHfJ8n3N7tlAMgfErjGByHLDuEQ8nxzh0N
EZaC3yhbLE0IdhLM0V+WPvkUQvrflL2x2K0zboKpcLbGdRc99UjbViLTM6E3prx+
GzEqcExDyMWX4kdhd+rBJHpuZ2AVhPMCAwEAAaNoMGYwDgYDVR0PAQH/BAQDAgKk
MA8GA1UdEwEB/wQFMAMBAf8wQwYDVR0RBDwwOoI4ZGVmYXVsdC1pbnRlcmNvbm5l
Y3Quc2VydmljZS10ZWxlbWV0cnkuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcN
AQELBQADggEBAIi1P31PtEk9nCSKBiPtOGl+RBHZlhoIMPZhLAs1BCmPnjUdBjpq
wS6IjihePVSX7mfb5o4TJTz3qlx/OfiyfWZ2+jKttJ8hOEjaxdspEnJ9n4ska9BP
eEHM5Xu5djIXRJKHihcrzmecFCMTc9R9kkHqQI8cithG4aXeg8I/KsZ1Isfa4w8y
lDNS0Zwk7mf40b5kwG83VB+dfiqvsO/ODTw7xu7aAr8TtnnMoHFfnw/wqr3XEJzw
pwBOkI9C7cq7unApiPBfomQFEXIWdTaU/7sy1Dt238rt/sWbznwcXu3M673oq0JV
TOpbgNlPcz0sIoNK3Be0jUcYHVMPKGMR2kk=
-----END CERTIFICATE-----


3 Edit stf custom template e.g. (stf-connectors.yaml) and add the following content:

MetricsQdrSSLProfiles:
    -   name: sslProfile
        caCertFileContent: |
           -----BEGIN CERTIFICATE-----
           MIIDpjCCAo6gAwIBAgIQFE4Z7BXVg+paMsLqKl4fnzANBgkqhkiG9w0BAQsFADBa
           MRUwEwYDVQQKEwxjZXJ0LW1hbmFnZXIxQTA/BgNVBAMTOGRlZmF1bHQtaW50ZXJj
           b25uZWN0LnNlcnZpY2UtdGVsZW1ldHJ5LnN2Yy5jbHVzdGVyLmxvY2FsMB4XDTIx
           MDQwODEzMTAwMFoXDTIxMDcwNzEzMTAwMFowWjEVMBMGA1UEChMMY2VydC1tYW5h
           Z2VyMUEwPwYDVQQDEzhkZWZhdWx0LWludGVyY29ubmVjdC5zZXJ2aWNlLXRlbGVt
           ZXRyeS5zdmMuY2x1c3Rlci5sb2NhbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
           AQoCggEBALB8oTZJuLT0jy2wf4tkiLuxY58Uo0KiHh+dMXWbcZ1voLEeATLIoyrK
           eUa2sBmPBsHdLt4nO34nEeBnNzoKQR8XLc5F8x4WM8mQPv0KsdfJPeskPvvAfeWT
           XcXBxf5af7HvtzeA+zL2onucaBeEcAbNIEm4Elz3d/BT70w1z235J3a2JMACFGcy
           kZVS74PxXjx65b57rPbtf0Gnlf0cSObHfJ8n3N7tlAMgfErjGByHLDuEQ8nxzh0N
           EZaC3yhbLE0IdhLM0V+WPvkUQvrflL2x2K0zboKpcLbGdRc9ertbViLTM6E3prx+
           GzEqcExDyMWX4kdhd+rBJHpuZ2AVhPMCAwEAAaNoMGYwDgYDVR0PAQH/BAQDAgKk
           MA8GA1UdEwEB/wQFMAMBAf8wQwYDVR0RBDwwOoI4ZGVh7tVsdC1pbnRlcmNvbm5l
           Y3Quc2VydmljZS10ZWxlbWV0cnkuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcN
           AQELBQADggEBAIi1P31PtEk9nCSKBiPtOGl+RBHZlhoIMPZhLAs1BCmPnjUdBjpq
           wS6IjihePVSX7mfb5o4TJTz3qlx/OfiyfWZ2+jKttJ8hOEjaxdspEnJ9n4ska9BP
           eEHM5Xu5djIXRJKHihcrzmecFCMTc9R9kkHqQI8cithG4aXeg8I/KsZ1Isfa4w8y
           lDNS0Zwk7mf40b5kwG83VB+dfiqvsO/ODTw7xu7aAr8TtnnMoHFfnw/wqr3XEJzw
           pwBOkI9C7cq7unApiPBfomQFEXIWdTaU/7sy1Dt238rt/sWbznwcXu3M673oq0JV
           TOpbgNlPcz0sIoNK3Be0jUcYHVMPKGMR2kk=
           -----END CERTIFICATE-----


4. Deploy overcloud.



-------------

BZ verified:

OSP was able to connect to the OCP

2021-05-07 09:40:32.390553 +0000 CONN_MGR (info) Created SSL Profile with name sslProfile
2021-05-07 09:40:32.399273 +0000 CONN_MGR (info) Configured Connector: default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 proto=any, role=edge, sslProfile=sslProfile
2021-05-07 09:40:32.401534 +0000 CONN_MGR (info) Configured Listener: 172.17.1.106:5666 proto=any, role=normal
2021-05-07 09:40:32.403365 +0000 SERVER (notice) Operational, 4 Threads Running (process ID 9)
2021-05-07 09:40:32.404051 +0000 SERVER (notice) Process VmSize 207.77 MiB (31.26 GiB available memory)
2021-05-07 09:40:32.404493 +0000 SERVER (notice) Listening on 172.17.1.106:5666
2021-05-07 09:40:33.297709 +0000 ROUTER (info) [C1] Connection Opened: dir=out host=default-interconnect-5671-service-telemetry.apps.leonidcluster.lab.upshift.rdu2.redhat.com:443 vhost= encrypted=TLSv1/SSLv3 auth=ANONYMOUS user=(null) container_id=default-interconnect-69d77454b5-wqbp4 props={:product="qpid-dispatch-router", :version="Red Hat Interconnect 1.10.0 (qpid-dispatch 1.14.0)", :"qd.conn-id"=953349}
2021-05-07 09:40:33.297818 +0000 ROUTER_CORE (info) Edge connection (id=1) to interior established
2021-05-07 09:40:33.297997 +0000 ROUTER_CORE (info) [C1][L6] Link attached: dir=out source={<none> expire:link} target={<none> expire:link}
2021-05-07 09:40:33.298085 +0000 ROUTER_CORE (info) [C1][L7] Link attached: dir=in source={Router.controller-0.redhat.local expire:link caps::"qd.router-edge-downlink"} target={<none> expire:link caps::"qd.router-edge-downlink"}
2021-05-07 09:40:33.298124 +0000 ROUTER_CORE (info) [C1][L8] Link attached: dir=in source={_$qd.edge_addr_tracking expire:link} target={<none> expire:link}
2021-05-07 09:40:33.298177 +0000 ROUTER_CORE (info) [C1][L9] Link attached: dir=out source={} target={$management expire:link}
2021-05-07 09:40:33.298190 +0000 ROUTER_CORE (info) [C1][L10] Link attached: dir=in source={<dynamic> expire:link} target={}
2021-05-07 09:40:33.298208 +0000 ROUTER_CORE (info) [C1][L11] Link attached: dir=out source={} target={_$qd.addr_lookup expire:link}
2021-05-07 09:40:33.298219 +0000 ROUTER_CORE (info) [C1][L12] Link attached: dir=in source={<dynamic> expire:link} target={}
2021-05-07 09:40:34.894964 +0000 SERVER (info) [C2] Accepted connection to 172.17.1.106:5666 from 172.17.1.106:46324
2021-05-07 09:40:34.901265 +0000 ROUTER (info) [C2] Connection Opened: dir=in host=172.17.1.106:46324 vhost= encrypted=no auth=no user=anonymous container_id=openstack.org/om/container/controller-0/ceilometer-agent-notification/28/2ad8e44547724d6db23838a290f15a64 props={:process=b"ceilometer-agent-notification", :node=b"controller-0", :pid=28}
2021-05-07 09:40:34.908614 +0000 ROUTER_CORE (info) [C2][L13] Link attached: dir=out source={<dynamic> expire:sess} target={rpc-response expire:sess}
2021-05-07 09:40:34.917686 +0000 ROUTER_CORE (info) [C2][L14] Link attached: dir=in source={/anycast/ceilometer/cloud2-metering.sample expire:sess} target={/anycast/ceilometer/cloud2-metering.sample expire:sess}
2021-05-07 09:40:35.060918 +0000 ROUTER_CORE (info) [C1][L15] Link attached: dir=out source={<none> expire:link} target={anycast/ceilometer/cloud2-metering.sample expire:link}
2021-05-07 09:40:39.314158 +0000 SERVER (info) [C3] Accepted connection to 172.17.1.106:5666 from 172.17.1.106:46590
2021-05-07 09:40:39.315462 +0000 ROUTER (info) [C3] Connection Opened: dir=in host=172.17.1.106:46590 vhost= encrypted=no auth=ANONYMOUS user=anonymous container_id=metrics props=
2021-05-07 09:40:39.315656 +0000 ROUTER_CORE (info) [C3][L16] Link attached: dir=in source={<none> expire:sess} target={<none> expire:sess}

Comment 41 errata-xmlrpc 2021-06-16 10:58:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 13.0 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2385


Note You need to log in before you can comment on or make changes to this bug.