Bug 1028624 - Measurement data points can be supplied to condition processing in different order than generated on agent
Measurement data points can be supplied to condition processing in different ...
Status: CLOSED CURRENTRELEASE
Product: RHQ Project
Classification: Other
Component: Alerts, Core Server (Show other bugs)
4.10
Unspecified Unspecified
unspecified Severity high (vote)
: GA
: RHQ 4.10
Assigned To: John Sanda
Mike Foley
:
Depends On:
Blocks: 1028623
  Show dependency treegraph
 
Reported: 2013-11-08 16:44 EST by Lukas Krejci
Modified: 2014-04-23 08:30 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1028623
Environment:
Last Closed: 2014-04-23 08:30:00 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Lukas Krejci 2013-11-08 16:44:22 EST
+++ This bug was initially created as a clone of Bug #1028623 +++

Description of problem:
When measurement data comes in to the server, the alert condition processing relies on the order to be strictly the same as generated on the agent side. I.e. no datapoint should be processed before another that was detected prior to it.

The persistence to the Cassandra storage node is asynchronous and done on datapoint-by-datapoint basis. This means that the datapoints can be persisted in different order than the original order.

When supplying the data to alert condition processing, we use the order of datapoints in which they were persisted, not the one they were created on the agent.

This can lead to subtle bugs in alerting like false positive or negative dampening events.

Version-Release number of selected component (if applicable):
JON 3.2.0.ER4

How reproducible:
hardly, can be best seen in debugger

Steps to Reproduce:
1. Have a server, agent and some enabled metric schedules for some inventoried resources
2. Once a measurement report comes in, check the order of elements in the input parameter of the MeasurementDataManagerBean#addNumericData() method.
3. Check the order of elements of the data passed to the condition processing in the onFinish() callback inside addNumericData()

Actual results:
The order sometimes differ, depending on the order cassandra cluster happened to store the events

Expected results:
The order of elements passed to the condition processing should be the same as the one coming from agents.

Additional info:
Comment 1 John Sanda 2013-11-09 18:11:01 EST
I have updated MeasurementDataManagerBean.addNumericData to ensure data is ordered by timestamp.

master commit hash: 1e3cf0f4ea0
Comment 2 Heiko W. Rupp 2014-04-23 08:30:00 EDT
Bulk closing of 4.10 issues.

If an issue is not solved for you, please open a new BZ (or clone the existing one) with a version designator of 4.10.

Note You need to log in before you can comment on or make changes to this bug.