Red Hat Bugzilla – Bug 1353519
gnocchi_aggregation_by_metrics_threshold throws traceback when overlap is not satisfied
Last modified: 2016-08-15 03:18:54 EDT
Description of problem: When one of metrics that composes gnocchi_aggregation_by_metrics_threshold alarm, stops being updated/is not updated, aodh-evaluator throws exception: 2016-07-07 11:23:31.912 18519 DEBUG aodh.evaluator [-] evaluating alarm bb643280-7c3a-4eea-b9bd-f4369faf6dd8 _evaluate_alarm /usr/lib/python2.7/site-packages/aodh/evaluator/__init__.py:220 2016-07-07 11:23:31.912 18519 DEBUG aodh.evaluator.threshold [-] query stats from 2016-07-07 11:19:31.912641 to 2016-07-07 11:23:31.912641 _bound_duration /usr/lib/python2.7/site-packages/aodh/evaluator/threshol d.py:79 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi [-] alarm stats retrieval failed 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi Traceback (most recent call last): 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi File "/usr/lib/python2.7/site-packages/aodh/evaluator/gnocchi.py", line 79, in _statistics 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi aggregation=rule['aggregation_method']) 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi File "/usr/lib/python2.7/site-packages/gnocchiclient/v1/metric.py", line 229, in aggregation 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi params=params).json() 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi File "/usr/lib/python2.7/site-packages/gnocchiclient/v1/base.py", line 37, in _get 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi return self.client.api.get(*args, **kwargs) 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 173, in get 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi return self.request(url, 'GET', **kwargs) 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi File "/usr/lib/python2.7/site-packages/gnocchiclient/client.py", line 38, in request 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi raise exceptions.from_response(resp, method) 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi BadRequest: One of the metrics being aggregated doesn't have matching granularity: Metrics 5c069bcc-13b9-429c-aedd-886c64893157, 8bb12f7f-87f2-41d7-ae38 -b053307e3061 can't be aggregated: Less than 100.000000% of datapoints overlap in this timespan (85.71%) (HTTP 400) 2016-07-07 11:23:33.375 18519 ERROR aodh.evaluator.gnocchi 2016-07-07 11:23:33.376 18519 DEBUG aodh.evaluator.gnocchi [-] sanitize stats [] _sanitize /usr/lib/python2.7/site-packages/aodh/evaluator/gnocchi.py:52 2016-07-07 11:23:33.376 18519 DEBUG aodh.evaluator.gnocchi [-] pruned statistics to 0 _sanitize /usr/lib/python2.7/site-packages/aodh/evaluator/gnocchi.py:56 This has to be handled gracefully. Version-Release number of selected component (if applicable): openstack-aodh-notifier-2.0.1-3.el7ost.noarch python-aodh-2.0.1-3.el7ost.noarch openstack-aodh-listener-2.0.1-3.el7ost.noarch openstack-aodh-common-2.0.1-3.el7ost.noarch openstack-aodh-evaluator-2.0.1-3.el7ost.noarch python-aodhclient-0.5.0-1.el7ost.noarch openstack-aodh-api-2.0.1-3.el7ost.noarch How reproducible: Steps to Reproduce: 1. Create alarm: aodh --debug alarm create \ --type gnocchi_aggregation_by_metrics_threshold \ --name 'AClient-Gn-Aggr-By-Metrics' \ --description 'AClient Gnocchi-Aggregation-By-Metrics-Threshold' \ --severity moderate \ --enabled True \ --alarm-action 'log://' \ --ok-action 'log://' \ --insufficient-data-action 'log://' \ --comparison-operator ge \ --evaluation-periods 3 \ --threshold 4.0 \ --granularity 60 \ --aggregation-method mean \ --metrics 5c069bcc-13b9-429c-aedd-886c64893157 \ --metrics 8bb12f7f-87f2-41d7-ae38-b053307e3061 """ 5c069bcc-13b9-429c-aedd-886c64893157 is "radosgw.api.request" on ceph-alarm-2 resource """ 8bb12f7f-87f2-41d7-ae38-b053307e3061 is "radosgw.api.request" on ceph-alarm-3 resource
Alarm transitioned between states after one metric stopped being updated. Packages: --------- openstack-aodh-notifier-2.0.3-2.el7ost.noarch python-aodh-2.0.3-2.el7ost.noarch openstack-aodh-evaluator-2.0.3-2.el7ost.noarch openstack-aodh-common-2.0.3-2.el7ost.noarch openstack-aodh-listener-2.0.3-2.el7ost.noarch openstack-aodh-api-2.0.3-2.el7ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-1597.html