Bug 1258912

Summary: [RFE] Move transfomers from polling agents to the notification agent
Product: Red Hat OpenStack Reporter: Eoghan Glynn <eglynn>
Component: openstack-ceilometerAssignee: Eoghan Glynn <eglynn>
Status: CLOSED ERRATA QA Contact: Yurii Prokulevych <yprokule>
Severity: medium Docs Contact:
Priority: low    
Version: 8.0 (Liberty)CC: eglynn, jruzicka, jschluet, nbarcet, yeylon
Target Milestone: betaKeywords: FutureFeature, OtherQA
Target Release: 8.0 (Liberty)   
Hardware: Unspecified   
OS: Unspecified   
URL: https://blueprints.launchpad.net/ceilometer/+spec/pollsters-no-transform
Whiteboard: upstream_milestone_liberty-3 upstream_definition_approved upstream_status_implemented
Fixed In Version: openstack-ceilometer-5.0.0-1.el7ost Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-07 21:06:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eoghan Glynn 2015-09-01 14:02:26 UTC
As a foundational enabler for splitting out the polling agents into a separate project, the transformation logic should be moved into the notification agent.

The polling agents should instead emit raw samples, with the transformation logic applied in the notification agent. This will require ensuring that related samples are always consumed by the same notification agent, in order to support multi-sample transformations.

Comment 3 Chris Dent 2015-09-10 10:05:51 UTC
In the ideal case this change will not be visible to users: although the data will travel a different path (through the notification agent before going to the collector) the consumable data that ends up in storage ought to be the same. Therefore the first and easiest way to confirm this change is to make sure that measures are gathered from the polling agent (ceilometer-polling) on all three namespaces: compute, central and ipmi. This presumably happens as part of regular testing. Then if problems are discovered it could potentially be traced back to this change.

To test the specifics of this change we need to confirm two things:

* a measure polled by the polling agent is seen in the logs of the notification agent
* measures polled by the polling agent (which does not do transforms) are transformed in the notification agent (and those transformation which require multiple samples actually work)

The ideas are also tested in https://bugzilla.redhat.com/show_bug.cgi?id=1258898 so that might be a useful reference for this one.

* Run up an installation (single node is okay) running at least two notification agents and at last one polling agent polling on at least the compute namespace:

   ceilometer-polling --polling-namespaces compute

(To speed up this process you may wish to lower the polling interval in pipeline.yaml from the default 600 seconds to something less than a minute.)

* Boot an instance and record its resource_id for later inspection.
* Confirm there are 'cpu' and 'cpu_util' meters for this resource.

brick:~ $ ceilometer meter-list -q resource_id=$INSTANCE |grep cpu
| cpu                      | cumulative | ns        | a97591c0-44a0-4b49-91ae-e68f3c014acb | 48a97aa69f8144e094f6dd86f111d072 | fac4bea00a174fc1b2eb303f24dfff5e |
| cpu_util                 | gauge      | %         | a97591c0-44a0-4b49-91ae-e68f3c014acb | 48a97aa69f8144e094f6dd86f111d072 | fac4bea00a174fc1b2eb303f24dfff5e |
| vcpus                    | gauge      | vcpu      | a97591c0-44a0-4b49-91ae-e68f3c014acb | 48a97aa69f8144e094f6dd86f111d072 | fac4bea00a174fc1b2eb303f24dfff5e |
brick:~ $ ceilometer sample-list -m cpu -q resource_id=$INSTANCE --limit 2
+--------------------------------------+------+------------+--------------+------+----------------------------+
| Resource ID                          | Name | Type       | Volume       | Unit | Timestamp                  |
+--------------------------------------+------+------------+--------------+------+----------------------------+
| a97591c0-44a0-4b49-91ae-e68f3c014acb | cpu  | cumulative | 1820000000.0 | ns   | 2015-09-10T09:57:27.080000 |
| a97591c0-44a0-4b49-91ae-e68f3c014acb | cpu  | cumulative | 1810000000.0 | ns   | 2015-09-10T09:57:17.056000 |
+--------------------------------------+------+------------+--------------+------+----------------------------+
brick:~ $ ceilometer sample-list -m cpu_util -q resource_id=$INSTANCE --limit 2
+--------------------------------------+----------+-------+-----------------+------+----------------------------+
| Resource ID                          | Name     | Type  | Volume          | Unit | Timestamp                  |
+--------------------------------------+----------+-------+-----------------+------+----------------------------+
| a97591c0-44a0-4b49-91ae-e68f3c014acb | cpu_util | gauge | 0.100325506105  | %    | 2015-09-10T09:57:37.047000 |
| a97591c0-44a0-4b49-91ae-e68f3c014acb | cpu_util | gauge | 0.0997586538886 | %    | 2015-09-10T09:57:27.080000 |
+--------------------------------------+----------+-------+-----------------+------+----------------------------+


* Check in the polling agent and notification agent logs for the resource_id. You should see entries like:

compute-agent:
DEBUG ceilometer.compute.pollsters.cpu [-] checking instance a97591c0-44a0-4b49-91ae-e68f3c014acb get_samples /home/opt/stack/ceilometer/ceilometer/compute/pollsters/cpu.py:33

notification-agent:
2015-09-10 09:54:17.054 15942 DEBUG ceilometer.transformer.conversions [req-3b8762b4-14ff-4fe0-a2ad-477860e997c9 - - - - -] handling sample <name: cpu, volume: 1720000000, resource_id: a97591c0-44a0-4b49-91ae-e68f3c014acb, timestamp: 2015-09-10T09:54:17.037038> handle_sample /home/opt/stack/ceilometer/ceilometer/transformer/conversions.py:115
2015-09-10 09:54:17.054 15942 DEBUG ceilometer.transformer.conversions [req-3b8762b4-14ff-4fe0-a2ad-477860e997c9 - - - - -] converted to: <name: cpu_util, volume: 9.50924329123, resource_id: a97591c0-44a0-4b49-91ae-e68f3c014acb, timestamp: 2015-09-10T09:54:17.037038> handle_sample /home/opt/stack/ceilometer/ceilometer/transformer/conversions.py:143


This covers the basics.

Comment 5 Yurii Prokulevych 2016-01-14 12:46:44 UTC
Verified according test plan.

Comment 7 errata-xmlrpc 2016-04-07 21:06:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0603.html