Bug 1258898 - [RFE] Remove pipeline logic from ceilometer API service
Summary: [RFE] Remove pipeline logic from ceilometer API service
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ceilometer
Version: 8.0 (Liberty)
Hardware: Unspecified
OS: Unspecified
low
medium
Target Milestone: beta
: 8.0 (Liberty)
Assignee: Eoghan Glynn
QA Contact: Yurii Prokulevych
URL: https://blueprints.launchpad.net/ceil...
Whiteboard: upstream_milestone_liberty-2 upstream...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-09-01 13:28 UTC by Eoghan Glynn
Modified: 2016-04-07 21:06 UTC (History)
5 users (show)

Fixed In Version: openstack-ceilometer-5.0.0-1.el7ost
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-04-07 21:06:07 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:0603 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 8 Enhancement Advisory 2016-04-08 00:53:53 UTC

Description Eoghan Glynn 2015-09-01 13:28:29 UTC
Currently the ceilometer-api service uses the pipeline logic in a fashion similar to the agents when processing samples POSTed via the REST API. This introduces unnecessary complexity in code and testing/deployment.

Instead the need for the pipeline at the API layer could be avoided with the simple expedient of emitting notifcations containing POSTed samples for the ceilometer notification agent to consume and dispatch to the collector in the normal way.

Comment 3 Chris Dent 2015-09-09 14:34:58 UTC
The primary thing that needs to be confirmed when testing this change is that POST-ing samples still works correctly (the samples are stored) using the new code. A secondary thing to confirm is that POST-ed samples that should be transformed by the pipeline are. We can do this by creating some samples using python-ceilometerclient:

This creates an arbitrary meter:

$ ceilometer sample-create -r $(uuidgen) -m cow --meter-type gauge --meter-unit C --sample-volume 5

we can confirm it was received by looking at:

$ ceilometer sample-list -q meter=cow
+--------------------------------------+--------------------------------------+------+-------+--------+------+----------------------------+
| ID                                   | Resource ID                          | Name | Type  | Volume | Unit | Timestamp                  |
+--------------------------------------+--------------------------------------+------+-------+--------+------+----------------------------+
| 62848c20-56fc-11e5-bd04-3417ebd4f75d | f95be10e-e9f0-46aa-b6ad-2c3ff6c87297 | cow  | gauge | 5.0    | C    | 2015-09-09T14:09:28.926000 |
+--------------------------------------+--------------------------------------+------+-------+--------+------+----------------------------+
 
Then to confirm that the samples are going through the notification agent, shut down the notification agent and create a sample (as above), confirm that it is _not_ there (as above). Then restart the notification agent, wait a few minutes and confirm the sample (which will have been waiting in the message bus) is saved:

$ ceilometer sample-list -q meter=cow
+--------------------------------------+--------------------------------------+------+-------+--------+------+----------------------------+
| ID                                   | Resource ID                          | Name | Type  | Volume | Unit | Timestamp                  |
+--------------------------------------+--------------------------------------+------+-------+--------+------+----------------------------+
| 1ffbd8b6-56fe-11e5-bd04-3417ebd4f75d | 12494bf6-df3e-41f4-9585-ec94b2eae053 | cow  | gauge | 10.0   | C    | 2015-09-09T14:21:56.294000 |
| 62848c20-56fc-11e5-bd04-3417ebd4f75d | f95be10e-e9f0-46aa-b6ad-2c3ff6c87297 | cow  | gauge | 5.0    | C    | 2015-09-09T14:09:28.926000 |
+--------------------------------------+--------------------------------------+------+-------+--------+------+----------------------------+


Finally we can confirm the operation of the pipeline by creating samples using that we know will resort in a transform. The default pipeline.yaml will transform 'cpu' samples into 'cpu_util'. In order for this test to work cleanly, no existing instances should be present:

$ export INSTANCE=$(uuidgen) 
$ for i in 5000 10000 15000 20000 ; do ceilometer sample-create -r $INSTANCE -m cpu --meter-type cumulative --meter-unit ns --sample-volume $i ; done
$ ceilometer sample-list -q meter=cpu_util

In the log file for the notification agent (when debugging in turned on) you should see entries like:

2015-09-09 14:32:22.778 25382 DEBUG ceilometer.transformer.conversions [req-77a78b1c-1314-4eea-858d-cac4f8c072e6 - - - - -] handling sample <name: cpu, volume: 15000.0, resource_id: 87a42ab8-e8ad-4178-9800-4fb740db7ae4, timestamp: 2015-09-09T14:32:22.774625> handle_sample /home/opt/stack/ceilometer/ceilometer/transformer/conversions.py:115

where the resource_id matches the value of $INSTANCE

Comment 5 Yurii Prokulevych 2016-01-14 07:33:58 UTC
Verified according test plan.

Comment 7 errata-xmlrpc 2016-04-07 21:06:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0603.html


Note You need to log in before you can comment on or make changes to this bug.