Bug 1042323

Summary:	[RFE][ceilometer]: ElasticSearch Storage Driver Support
Product:	Red Hat OpenStack	Reporter:	RHOS Integration <rhos-integ>
Component:	openstack-ceilometer	Assignee:	Eoghan Glynn <eglynn>
Status:	CLOSED ERRATA	QA Contact:	Yurii Prokulevych <yprokule>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	unspecified	CC:	eglynn, flepied, jruzicka, markmc, pbrady, rrosa, yeylon
Target Milestone:	Upstream M2	Keywords:	FutureFeature, OtherQA, TechPreview, Triaged
Target Release:	7.0 (Kilo)
Hardware:	Unspecified
OS:	Unspecified
URL:	https://blueprints.launchpad.net/ceilometer/+spec/elasticsearch-driver
Whiteboard:	upstream_milestone_kilo-2 upstream_status_implemented upstream_definition_approved
Fixed In Version:	openstack-ceilometer-2015.1.0-2.el7ost	Doc Type:	Technology Preview
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-08-05 13:11:09 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description RHOS Integration 2013-12-12 21:45:12 UTC

Cloned from launchpad blueprint https://blueprints.launchpad.net/ceilometer/+spec/elasticsearch-driver.

Description:

This bp implements ElasticSearch Storage driver.

ElasticSearch: http://www.elasticsearch.org/
ElasticSearch has good query language and UI  : Kibana3 http://three.kibana.org/

ElasticSearch is also good for storing logs, so we can use ceilometer + Elasticsearch as Log as a Service framework.



Specification URL (additional information):

None

Comment 3 Yurii Prokulevych 2015-04-07 15:24:36 UTC

Hi Eoghan,

I do have some questions regarding the RFE:

1. Is this driver applicable also for meters/alarms storing or just for events?

2. Elasticsearh version suppose to be >= 1.3.0 ?

3. Is Kibana needed for testing? If yes what version?

4. Elasticsearch/Kibana configuration - deployer is responsible for setting/configuring them up? Or it would be also part of packstack/staypufft installers?
    Would ES/Kibana put into our repo or should be downloaded from the www.elastic.co site?

5. Connection URL format:
    According to http://www.elastic.co/guide/en/elasticsearch/guide/current/_indexing_employee_documents.html
    'The index name' and 'the type name' have to be provided. 
    So what is the format of {event|alarm|metring}_connection option in ceilometer.conf. Something like:
        event_connection=es://<hostname|ip>:<port>/<index_name>/<type_name> ?
    What are valid names for the 'index/type' names?

6. If events are stored in ES it's still doable to get events list via 'ceilometer event-list' call?

7. How is Elasticsearch inaccessibility handled?(predefined amount of retries or smth?)

Thanks in advance.

Regards,
Yurii

Comment 5 gchung 2015-05-04 13:40:59 UTC

1. Is this driver applicable also for meters/alarms storing or just for events?
 - as discussed, it only stores events

2. Elasticsearh version suppose to be >= 1.3.0 ?
 - i've been testing against >= 1.3.0 but i'm not sure if there's a minimum version required

3. Is Kibana needed for testing? If yes what version?
 - not required but can be used. versioning will probably be based on elasticsearch version

4. Elasticsearch/Kibana configuration - deployer is responsible for setting/configuring them up? Or it would be also part of packstack/staypufft installers?
    Would ES/Kibana put into our repo or should be downloaded from the www.elastic.co site?
 - there is 0 configuration done on ceilometer end. it just connects to a specific url

5. Connection URL format:
    According to http://www.elastic.co/guide/en/elasticsearch/guide/current/_indexing_employee_documents.html
    'The index name' and 'the type name' have to be provided. 
    So what is the format of {event|alarm|metring}_connection option in ceilometer.conf. Something like:
        event_connection=es://<hostname|ip>:<port>/<index_name>/<type_name> ?
    What are valid names for the 'index/type' names?
 - the index is events_YYYY-MM-DD, and the type are each possible event_type of notifications

6. If events are stored in ES it's still doable to get events list via 'ceilometer event-list' call?
 - yes, elasticsearch backend supports the same queries as other backends in addition to kibana. ie. event-list, event-type-list, trait-list, trait-type-list

7. How is Elasticsearch inaccessibility handled?(predefined amount of retries or smth?)
 - i don't really have an answer for this, i didn't consider this.

Comment 13 errata-xmlrpc 2015-08-05 13:11:09 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1548