Bug 1042379

Cloned from launchpad blueprint https://blueprints.launchpad.net/python-ceilometerclient/+spec/meter-sql-export.

Description:

The aim is to provide a way in Ceilometer to export meter data for a use
with a Business Intelligence tool. Those tools can collect data from
SQL backends so we need a way in Ceilometer to feed an external SQL storage
with meter data.

An export tool could be written to do that and could work as follow.

The export tool will query the Ceilometer API/v2 to retreive meter data then
push those data in a specified SQL backend.

We need to define some values in a configuration file to properly
configure the export worker. The configuration could have the following
statements :
- Ceilometer API endpoint
- The tenant
- The user
- The user key
- The SQL connection string
- A query loop delay
- A list associating a filter query with a list of meter ids

Some explanations about the above configuration statements:
- Tenant/user/key must be a user that own a keystone role that let it browse
  meters of all resources/projets.
- The SQL connection string define the remote SQL database with credentials
  provided.
- A query loop delay define the frequency in second the worker will query the
  API for all meters for each resource/project.
- The 'list associating a filter query with a list of meter ids' let the user
  define for which resources/projects the worker will retreive meters. And
  for each of those resources/projects which meters the worker will retreive.  

The API/v2 filter query can be used to select resources/project to retreive meters
the worker is looking for.

The remote SQL database could be setup with one table schema described below. This curl
query 'curl -X GET -H "X-Auth-Token: $a"  http://127.0.0.1:8777/v2/meters/image.size'
returns all samples for the image.size meter. Data returned by querying this endpoint
regardless the meter has the same structure. 

Proposed table schema:
- counter_name VARCHAR
- counter_type VARCHAR
- counter_unit VARCHAR
- counter_volume FLOAT
- message_id VARCHAR
- project_id VARCHAR
- resource_id VARCHAR
- source VARCHAR
- timestamp DATETIME

message_id could be used as primary key as this is the message uuid.

As the worker will query the API for retreive last meter samples since the
last query we need to keep a synchronization point. The point can be the timestamp.
This timestamp can be used in the API filter query to only retreive the most
recent records.

Specification URL (additional information):

None