Bug 1769868 - OSP 16 | Service Assurance | qdr mesh patch broke qdr connector configuration. Seems like only puppet side of the change went to OSP16
Summary: OSP 16 | Service Assurance | qdr mesh patch broke qdr connector configuration...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 16.0 (Train)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: z1
: 16.0 (Train on RHEL 8.1)
Assignee: Martin Magr
QA Contact: Leonid Natapov
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-07 16:11 UTC by Leonid Natapov
Modified: 2023-01-03 21:35 UTC (History)
10 users (show)

Fixed In Version: puppet-tripleo-11.4.1-0.20200131.09fa984.el8ost openstack-tripleo-heat-templates-11.3.2-0.20200119215823.f90eb2c.el8ost
Doc Type: Bug Fix
Doc Text:
Previously, the mesh network infrastructure was configured incorrectly for the message router, QDR, causing the AMQP-1.0 message bus on the Service Telemetry Framework (STF) client to malfunction. This fix corrects the configuration for the qdrouterd daemon on all overcloud nodes, and the STF client now functions properly.
Clone Of:
Environment:
Last Closed: 2020-03-03 09:45:05 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 693805 0 'None' MERGED Split QDR environment from collectd 2021-02-17 14:44:42 UTC
OpenStack gerrit 694307 0 'None' MERGED Synchronize connection configuration for metrics 2021-02-17 14:44:42 UTC
OpenStack gerrit 694308 0 'None' MERGED Use public network for external interface 2021-02-17 14:44:42 UTC
OpenStack gerrit 697100 0 'None' MERGED Split QDR environment from collectd 2021-02-17 14:44:42 UTC
OpenStack gerrit 697101 0 'None' MERGED Synchronize connection configuration for metrics 2021-02-17 14:44:42 UTC
OpenStack gerrit 697102 0 'None' MERGED Fill sslProfile only when it is defined 2021-02-17 14:44:42 UTC
OpenStack gerrit 697106 0 'None' MERGED Use public network for external interface 2021-02-17 14:44:43 UTC
OpenStack gerrit 699491 0 None MERGED Fix typo in sslProfile conditional 2021-02-17 14:44:43 UTC
OpenStack gerrit 699493 0 None MERGED Fix typos in hiera values 2021-02-17 14:44:42 UTC
OpenStack gerrit 700417 0 None MERGED Fix typo in sslProfile conditional 2021-02-17 14:44:43 UTC
OpenStack gerrit 700418 0 None MERGED Fix typos in hiera values 2021-02-17 14:44:43 UTC
OpenStack gerrit 702656 0 None MERGED Open ports for Metrics QDRs 2021-02-17 14:44:43 UTC
OpenStack gerrit 702754 0 None MERGED Use ctlplane for internal QDR communication 2021-02-17 14:44:44 UTC
OpenStack gerrit 702892 0 None MERGED Open ports for Metrics QDRs 2021-02-17 14:44:44 UTC
OpenStack gerrit 703678 0 None MERGED split() function expects string not array 2021-02-17 14:44:44 UTC
Red Hat Issue Tracker OSP-21138 0 None None None 2023-01-03 21:35:42 UTC
Red Hat Product Errata RHBA-2020:0655 0 None None None 2020-03-03 09:45:47 UTC

Description Leonid Natapov 2019-11-07 16:11:27 UTC 
qdr mesh patch broke qdr connector configuration. Seems like  only puppet side of the change went to OSP16

Currently controller node  thinks that it should operate as edge node and tries to connect to itself.

019-11-06 16:22:26.440406 +0000 SERVER (info) [2]: Connection to controller-0:5668 failed: proton:io Connection refused - disconnected controller-0:5668
2019-11-06 16:22:31.442049 +0000 SERVER (info) [3]: Connection to controller-0:5668 failed: proton:io Connection refused - disconnected controller-0:5668
2019-11-06 16:22:36.443500 +0000 SERVER (info) [4]: Connection to controller-0:5668 failed: proton:io Connection refused - disconnected controller-0:5668


W/A" Add  ControllerExtraConfig option in the env. file.

ControllerExtraConfig:
        tripleo::profile::base::metrics::qdr::router_mode: interior
Comment 7 Leonid Natapov 2019-12-17 10:52:41 UTC 
Failed QA. Moving back to Assign.We have to open firewall for the internal ports
Comment 17 Leonid Natapov 2020-02-20 05:22:55 UTC 
overcloud deployment im mesh mode fails,moving back to assign

<13>Feb 19 18:23:38 puppet-user: Error: Evaluation Error: Error while evaluating a Function Call, Value of interior_ip '192.168.24.16' is not member of interior_mesh_nodes ''. (file: /etc/puppet/modules/tripleo/manifests/profile/base/metrics/qdr.pp, line: 176, column: 9) on node controller-2.redhat.local", "+ rc=1", "+ '[' False = false ']'", "+ set -e", "+ '[' 1 -ne 2 -a 1 -ne 0 ']'", "+ exit 1", " attempt(s): 3", "2020-02-19 18:23:41,976 INFO: 52437 -- Removing container: container-puppet-metrics-qdr", "2020-02-19 18:23:42,207 WARNING: 52437 -- Retrying running container: metrics-qdr", "2020-02-19 18:23:42,207 ERROR: 52437 -- Failed running container for metrics-qdr"


puppet-tripleo-11.4.1-0.20200205150840.71ff36d.el8ost.noarch
openstack-tripleo-heat-templates-11.3.2-0.20200211065543.d3d6dc3.el8ost.noarch
Comment 18 Leonid Natapov 2020-02-21 00:14:37 UTC 
verified. the problem was in THT that not purges that hieradata while stack update.

CLean deployment for the mesh mode - works.
Comment 20 errata-xmlrpc 2020-03-03 09:45:05 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0655
Note You need to log in before you can comment on or make changes to this bug.