Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1353519

Summary:	gnocchi_aggregation_by_metrics_threshold throws traceback when overlap is not satisfied
Product:	Red Hat OpenStack	Reporter:	Yurii Prokulevych <yprokule>
Component:	openstack-aodh	Assignee:	Mehdi ABAAKOUK <mabaakou>
Status:	CLOSED ERRATA	QA Contact:	Yurii Prokulevych <yprokule>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	9.0 (Mitaka)	CC:	apevec, jjoyce, jschluet, lhh, mburns, mlopes, pkilambi
Target Milestone:	ga	Keywords:	Triaged
Target Release:	9.0 (Mitaka)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-aodh-2.0.3-2.el7ost	Doc Type:	Bug Fix
Doc Text:	Previously, when creating an alarm of type `gnocchi_aggregation_by_metrics_threshold`, the evaluator would throw an error, causing an exception. This update addresses this issue by setting `needed_overlap` to always apply on aggregation.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-08-15 07:18:54 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1360399
Bug Blocks:

Description Yurii Prokulevych 2016-07-07 11:34:49 UTC

Description of problem:
When one of metrics that composes gnocchi_aggregation_by_metrics_threshold alarm, stops being updated/is not updated, aodh-evaluator throws exception:

2016-07-07 11:23:31.912 18519 DEBUG aodh.evaluator [-] evaluating alarm bb643280-7c3a-4eea-b9bd-f4369faf6dd8 _evaluate_alarm /usr/lib/python2.7/site-packages/aodh/evaluator/__init__.py:220
2016-07-07 11:23:31.912 18519 DEBUG aodh.evaluator.threshold [-] query stats from 2016-07-07 11:19:31.912641 to 2016-07-07 11:23:31.912641 _bound_duration /usr/lib/python2.7/site-packages/aodh/evaluator/threshol
d.py:79
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi [-] alarm stats retrieval failed
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi Traceback (most recent call last):
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi   File "/usr/lib/python2.7/site-packages/aodh/evaluator/gnocchi.py", line 79, in _statistics
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi     aggregation=rule['aggregation_method'])
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi   File "/usr/lib/python2.7/site-packages/gnocchiclient/v1/metric.py", line 229, in aggregation
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi     params=params).json()
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi   File "/usr/lib/python2.7/site-packages/gnocchiclient/v1/base.py", line 37, in _get
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi     return self.client.api.get(*args, **kwargs)
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi   File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 173, in get
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi     return self.request(url, 'GET', **kwargs)
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi   File "/usr/lib/python2.7/site-packages/gnocchiclient/client.py", line 38, in request
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi     raise exceptions.from_response(resp, method)
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi BadRequest: One of the metrics being aggregated doesn't have matching granularity: Metrics 5c069bcc-13b9-429c-aedd-886c64893157, 8bb12f7f-87f2-41d7-ae38
-b053307e3061 can't be aggregated: Less than 100.000000% of datapoints overlap in this timespan (85.71%) (HTTP 400)
2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi 
2016-07-07 11:23:33.376 18519 DEBUG aodh.evaluator.gnocchi [-] sanitize stats [] _sanitize /usr/lib/python2.7/site-packages/aodh/evaluator/gnocchi.py:52
2016-07-07 11:23:33.376 18519 DEBUG aodh.evaluator.gnocchi [-] pruned statistics to 0 _sanitize /usr/lib/python2.7/site-packages/aodh/evaluator/gnocchi.py:56

This has to be handled gracefully. 

Version-Release number of selected component (if applicable):
openstack-aodh-notifier-2.0.1-3.el7ost.noarch
python-aodh-2.0.1-3.el7ost.noarch
openstack-aodh-listener-2.0.1-3.el7ost.noarch
openstack-aodh-common-2.0.1-3.el7ost.noarch
openstack-aodh-evaluator-2.0.1-3.el7ost.noarch
python-aodhclient-0.5.0-1.el7ost.noarch
openstack-aodh-api-2.0.1-3.el7ost.noarch

How reproducible:


Steps to Reproduce:
1. Create alarm:
aodh --debug alarm create \
--type gnocchi_aggregation_by_metrics_threshold \
--name 'AClient-Gn-Aggr-By-Metrics' \
--description 'AClient Gnocchi-Aggregation-By-Metrics-Threshold' \
--severity moderate \
--enabled True \
--alarm-action 'log://' \
--ok-action 'log://' \
--insufficient-data-action 'log://' \
--comparison-operator ge \
--evaluation-periods 3 \
--threshold 4.0 \
--granularity 60 \
--aggregation-method mean \
--metrics 5c069bcc-13b9-429c-aedd-886c64893157 \
--metrics 8bb12f7f-87f2-41d7-ae38-b053307e3061

"""  5c069bcc-13b9-429c-aedd-886c64893157 is "radosgw.api.request" on ceph-alarm-2 resource
""" 8bb12f7f-87f2-41d7-ae38-b053307e3061 is "radosgw.api.request" on ceph-alarm-3 resource

Comment 3 Yurii Prokulevych 2016-07-29 14:33:43 UTC

Alarm transitioned between states after one metric stopped being updated.

Packages:
---------
openstack-aodh-notifier-2.0.3-2.el7ost.noarch
python-aodh-2.0.3-2.el7ost.noarch
openstack-aodh-evaluator-2.0.3-2.el7ost.noarch
openstack-aodh-common-2.0.3-2.el7ost.noarch
openstack-aodh-listener-2.0.3-2.el7ost.noarch
openstack-aodh-api-2.0.3-2.el7ost.noarch

Comment 5 errata-xmlrpc 2016-08-15 07:18:54 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-1597.html