Bug 1580338 - [RFE] Enable collectd-amq1 configuration via template parameters
Summary: [RFE] Enable collectd-amq1 configuration via template parameters
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 14.0 (Rocky)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: beta
: 14.0 (Rocky)
Assignee: Martin Magr
QA Contact: Leonid Natapov
URL:
Whiteboard:
Depends On: 1580331 1640502 1689278 1691506
Blocks: 1566081
TreeView+ depends on / blocked
 
Reported: 2018-05-21 10:08 UTC by Martin Magr
Modified: 2019-04-03 13:08 UTC (History)
8 users (show)

Fixed In Version: openstack-tripleo-heat-templates-9.0.0-0.20180827161726.1bdefbe.0rc1.el7ost puppet-tripleo-9.3.1-0.20180828211312.dabcc3f.el7ost
Doc Type: Enhancement
Doc Text:
With this enhancement, you can configure collectd to send metrics data to local QPID dispatch router services, to support service assurance framework implementation. New collectd tripleo heat template parameters let you configure the amqp1 write plugin. Set the CollectdConnectionType parameter to 'amqp1'. As a result, all metrics data are sent to the local QDR unless explicitly overridden by the parameters shown below. To deploy the local QDR, use the environment file /usr/share/openstack-tripleo-heat-templates/environments/metrics-collectd-qdr.yaml. The relevant heat template parameters are shown below. CollectdAmqpHost: type: string description: Hostname or IP address of the AMQP 1.0 intermediary. default: nil CollectdAmqpPort: type: string description: > Service name or port number on which the AMQP 1.0 intermediary accepts connections. This argument must be a string, even if the numeric form is used. default: '5666' CollectdAmqpUser: type: string description: > User part of credentials used to authenticate to the AMQP 1.0 intermediary. default: guest CollectdAmqpPassword: type: string description: > Password part of credentials used to authenticate to the AMQP 1.0 intermediary. default: guest hidden: true CollectdAmqpTransportName: type: string description: Name of the AMQP 1.0 transport. default: metrics CollectdAmqpAddress: type: string description: > This option specifies the prefix for the send-to value in the message. default: collectd CollectdAmqpInstances: type: json description: > Hash of hashes. Each inner hash represent Instance block in plugin configuration file. Key of outer hash represents instance name. The 'address' value concatenated with the 'name' given will be used as the send-to address for communications over the messaging link. default: {} CollectdAmqpRetryDelay: type: number description: > When the AMQP 1.0 connection is lost, defines the time in seconds to wait before attempting to reconnect. default: 1 CollectdAmqpInterval: type: number description: > Interval on which metrics should be sent to AMQP intermediary. If not set the default for all collectd plugins is used. default: -666
Clone Of:
Environment:
Last Closed: 2019-01-11 11:49:59 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 571152 0 None MERGED Collectd QDR connection 2020-04-15 17:21:54 UTC
OpenStack gerrit 576057 0 None MERGED Enable collectd to connect to metrics QDR 2020-04-15 17:21:54 UTC
Red Hat Product Errata RHEA-2019:0045 0 None None None 2019-01-11 11:50:16 UTC

Description Martin Magr 2018-05-21 10:08:42 UTC 
We need to be able to configure collectd-amq1 plugin via tripleo-heat-templates to enable sending metric data through QDR.
Comment 7 Martin Magr 2018-07-24 09:51:55 UTC 
Patch wasn't merged upstream yet, moving back to ON_DEV.
Comment 19 Leonid Natapov 2018-11-20 13:53:50 UTC 
Without SA server I see the following log:
2018-11-20 12:37:47.819618 +0000 CONN_MGR (info) Configured Listener: 172.17.1.31:5666 proto=any, role=normal
2018-11-20 12:37:47.820508 +0000 AGENT (debug) Add entity: ListenerEntity(authenticatePeer=False, cost=1, host=172.17.1.31, http=False, identity=listener/172.17.1.31:5666, idleTimeoutSeconds=16, initialHandshakeTimeoutSeconds=0, maxFrameSize=16384, maxSessions=32768,
messageLoggingComponents=none, multiTenant=False, port=5666, requireEncryption=False, requireSsl=False, role=normal, saslMechanisms=ANONYMOUS, stripAnnotations=both, type=org.apache.qpid.dispatch.listener)
2018-11-20 12:37:47.821408 +0000 AGENT (debug) Add entity: AddressEntity(distribution=closest, identity=address/0, name=address/0, prefix=unicast, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-20 12:37:47.821952 +0000 AGENT (debug) Add entity: AddressEntity(distribution=closest, identity=address/1, name=address/1, prefix=exclusive, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-20 12:37:47.822566 +0000 AGENT (debug) Add entity: AddressEntity(distribution=multicast, identity=address/2, name=address/2, prefix=broadcast, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-20 12:37:47.823330 +0000 AGENT (debug) Add entity: AddressEntity(distribution=multicast, identity=address/3, name=address/3, prefix=collectd/notify, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-20 12:37:47.824200 +0000 AGENT (debug) Add entity: AddressEntity(distribution=multicast, identity=address/4, name=address/4, prefix=collectd/telemetry, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-20 12:37:47.824632 +0000 POLICY (info) Policy configured maxConnections: 65535, policyDir: '', access rules enabled: 'false'
2018-11-20 12:37:47.824762 +0000 AGENT (debug) Add entity: PolicyEntity(defaultVhost=$default, enableVhostPolicy=False, maxConnections=65535, policyDir=, type=org.apache.qpid.dispatch.policy)
2018-11-20 12:37:47.825529 +0000 POLICY (info) Policy fallback defaultVhost is defined: '$default'
2018-11-20 12:37:47.825581 +0000 SERVER (notice) Operational, 2 Threads Running (process ID 1)
2018-11-20 12:37:47.825694 +0000 SERVER (notice) Listening on 172.17.1.31:5666
2018-11-20 12:55:09.660760 +0000 SERVER (info) Accepted connection to 172.17.1.31:5666 from 172.17.1.31:48548

----


with configured SA server I see the following log:

pid.dispatch.allocator, typeName=qd_node_t, typeSize=56)
2018-11-19 14:55:02.939625 +0000 AGENT (debug) Add entity: AllocatorEntity(batchesRebalancedToGlobal=0, batchesRebalancedToThreads=0, globalFreeListMax=0, heldByThreads=128, localFreeListMax=128, totalAllocFromHeap=128, totalFreeToHeap=0, transferBatchSize=64, type=org.apache.qpid.dispatch.allocator, typeName=qd_timer_t, typeSize=56)
2018-11-19 14:55:02.940030 +0000 AGENT (debug) Add entity: AllocatorEntity(batchesRebalancedToGlobal=0, batchesRebalancedToThreads=0, globalFreeListMax=0, heldByThreads=64, localFreeListMax=128, totalAllocFromHeap=64, totalFreeToHeap=0, transferBatchSize=64, type=org.apache.qpid.dispatch.allocator, typeName=qdr_action_t, typeSize=160)
2018-11-19 14:55:02.940548 +0000 AGENT (debug) Add entity: AllocatorEntity(batchesRebalancedToGlobal=0, batchesRebalancedToThreads=0, globalFreeListMax=0, heldByThreads=64, localFreeListMax=128, totalAllocFromHeap=64, totalFreeToHeap=0, transferBatchSize=64, type=org.apache.qpid.dispatch.allocator, typeName=qdr_field_t, typeSize=40)
2018-11-19 14:55:02.940964 +0000 AGENT (debug) Add entity: AllocatorEntity(batchesRebalancedToGlobal=0, batchesRebalancedToThreads=0, globalFreeListMax=0, heldByThreads=64, localFreeListMax=128, totalAllocFromHeap=64, totalFreeToHeap=0, transferBatchSize=64, type=org.apache.qpid.dispatch.allocator, typeName=qd_buffer_t, typeSize=536)
2018-11-19 14:55:02.941519 +0000 AGENT (debug) Add entity: AllocatorEntity(batchesRebalancedToGlobal=0, batchesRebalancedToThreads=0, globalFreeListMax=0, heldByThreads=64, localFreeListMax=128, totalAllocFromHeap=64, totalFreeToHeap=0, transferBatchSize=64, type=org.apache.qpid.dispatch.allocator, typeName=qd_parse_node_t, typeSize=104)
2018-11-19 14:55:02.942002 +0000 AGENT (debug) Add entity: AllocatorEntity(batchesRebalancedToGlobal=0, batchesRebalancedToThreads=0, globalFreeListMax=0, heldByThreads=64, localFreeListMax=128, totalAllocFromHeap=64, totalFreeToHeap=0, transferBatchSize=64, type=org.apache.qpid.dispatch.allocator, typeName=qdr_address_t, typeSize=288)
2018-11-19 14:55:02.942563 +0000 AGENT (debug) Add entity: AllocatorEntity(batchesRebalancedToGlobal=0, batchesRebalancedToThreads=0, globalFreeListMax=0, heldByThreads=64, localFreeListMax=128, totalAllocFromHeap=64, totalFreeToHeap=0, transferBatchSize=64, type=org.apache.qpid.dispatch.allocator, typeName=qd_bitmask_t, typeSize=24)
2018-11-19 14:55:02.942997 +0000 AGENT (debug) Add entity: AllocatorEntity(batchesRebalancedToGlobal=0, batchesRebalancedToThreads=0, globalFreeListMax=0, heldByThreads=64, localFreeListMax=128, totalAllocFromHeap=64, totalFreeToHeap=0, transferBatchSize=64, type=org.apache.qpid.dispatch.allocator, typeName=qd_hash_handle_t, typeSize=16)
2018-11-19 14:55:02.943602 +0000 AGENT (info) Activating management agent on $_management_internal
2018-11-19 14:55:02.943675 +0000 ROUTER_CORE (info) In-process subscription L/$_management_internal
2018-11-19 14:55:02.944422 +0000 CONN_MGR (info) Configured Listener: 172.17.1.12:5666 proto=any, role=normal
2018-11-19 14:55:02.945183 +0000 AGENT (debug) Add entity: ListenerEntity(authenticatePeer=False, cost=1, host=172.17.1.12, http=False, identity=listener/172.17.1.12:5666, idleTimeoutSeconds=16, initialHandshakeTimeoutSeconds=0, maxFrameSize=16384, maxSessions=32768, messageLoggingComponents=none, multiTenant=False, port=5666, requireEncryption=False, requireSsl=False, role=normal, saslMechanisms=ANONYMOUS, stripAnnotations=both, type=org.apache.qpid.dispatch.listener)
2018-11-19 14:55:02.945813 +0000 CONN_MGR (info) Configured Connector: 10.35.64.7:5666 proto=any, role=inter-router
2018-11-19 14:55:02.945997 +0000 AGENT (debug) Add entity: ConnectorEntity(allowRedirect=True, cost=1, host=10.35.64.7, identity=connector/10.35.64.7:5666, idleTimeoutSeconds=16, maxFrameSize=16384, maxSessions=32768, messageLoggingComponents=none, port=5666, role=inter-router, stripAnnotations=both, type=org.apache.qpid.dispatch.connector, verifyHostname=True)
2018-11-19 14:55:02.946623 +0000 CONN_MGR (info) Configured Connector: 10.35.64.71:5667 proto=any, role=inter-router
2018-11-19 14:55:02.946800 +0000 AGENT (debug) Add entity: ConnectorEntity(allowRedirect=True, cost=1, host=10.35.64.71, identity=connector/10.35.64.71:5667, idleTimeoutSeconds=16, maxFrameSize=16384, maxSessions=32768, messageLoggingComponents=none, port=5667, role=inter-router, stripAnnotations=both, type=org.apache.qpid.dispatch.connector, verifyHostname=True)
2018-11-19 14:55:02.947575 +0000 AGENT (debug) Add entity: AddressEntity(distribution=closest, identity=address/0, name=address/0, prefix=unicast, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-19 14:55:02.948179 +0000 AGENT (debug) Add entity: AddressEntity(distribution=closest, identity=address/1, name=address/1, prefix=exclusive, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-19 14:55:02.948800 +0000 AGENT (debug) Add entity: AddressEntity(distribution=multicast, identity=address/2, name=address/2, prefix=broadcast, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-19 14:55:02.949389 +0000 AGENT (debug) Add entity: AddressEntity(distribution=multicast, identity=address/3, name=address/3, prefix=collectd/notify, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-19 14:55:02.949953 +0000 AGENT (debug) Add entity: AddressEntity(distribution=multicast, identity=address/4, name=address/4, prefix=collectd/telemetry, type=org.apache.qpid.dispatch.router.config.address, waypoint=False)
2018-11-19 14:55:02.950449 +0000 POLICY (info) Policy configured maxConnections: 65535, policyDir: '', access rules enabled: 'false'
2018-11-19 14:55:02.950590 +0000 AGENT (debug) Add entity: PolicyEntity(defaultVhost=$default, enableVhostPolicy=False, maxConnections=65535, policyDir=, type=org.apache.qpid.dispatch.policy)
2018-11-19 14:55:02.951342 +0000 POLICY (info) Policy fallback defaultVhost is defined: '$default'
2018-11-19 14:55:02.951388 +0000 SERVER (notice) Operational, 2 Threads Running (process ID 1)
2018-11-19 14:55:02.951489 +0000 SERVER (notice) Listening on 172.17.1.12:5666
2018-11-19 14:55:02.952722 +0000 SERVER (info) Connection to 10.35.64.7:5666 failed: proton:io No route to host - disconnected 10.35.64.7:5666
2018-11-19 14:55:05.529605 +0000 SERVER (info) Connection to 10.35.64.71:5667 failed: proton:io No route to host - disconnected 10.35.64.71:5667
2018-11-19 14:55:07.953747 +0000 SERVER (info) Connection to 10.35.64.7:5666 failed: proton:io No route to host - disconnected 10.35.64.7:5666
2018-11-19 14:55:12.954080 +0000 SERVER (info) Connection to 10.35.64.7:5666 failed: proton:io No route to host - disconnected 10.35.64.7:5666
2018-11-19 14:55:13.535622 +0000 SERVER (info) Connection to 10.35.64.71:5667 failed: proton:io No route to host - disconnected 10.35.64.71:5667
2018-11-19 14:55:17.955289 +0000 SERVER (info) Connection to 10.35.64.7:5666 failed: proton:io No route to host - disconnected 10.35.64.7:5666
2018-11-19 14:55:19.547821 +0000 SERVER (info) Connection to 10.35.64.71:5667 failed: proton:io No route to host - disconnected 10.35.64.71:5667
2018-11-19 14:55:22.956114 +0000 SERVER (info) Connection to 10.35.64.7:5666 failed: proton:io No route to host - disconnected 10.35.64.7:5666
Comment 23 errata-xmlrpc 2019-01-11 11:49:59 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045
Note You need to log in before you can comment on or make changes to this bug.