Bug 535576 (RHQ-2257)

Summary:	Alert Templates take hours to be applied to existing resources
Product:	[Other] RHQ Project	Reporter:	Jeff Weiss <jweiss>
Component:	Alerts	Assignee:	Joseph Marques <jmarques>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Chandrasekar Kannan <ckannan>
Severity:	medium	Docs Contact:
Priority:	high
Version:	1.3pre	CC:	benl, bkearney, dajohnso, whayutin
Target Milestone:	---	Keywords:	AutoVerified
Target Release:	---
Hardware:	All
OS:	All
URL:	http://jira.rhq-project.org/browse/RHQ-2257
Whiteboard:
Fixed In Version:	2.4	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:	rev4508
Last Closed:	2010-08-12 16:50:31 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	536569, 547833, 565635, 576714, 584435, 591531

Description Jeff Weiss 2009-07-22 20:11:00 UTC

How to repeat:

Create an alert template (i used the Jopr/RHQ Server resource) that always fires (i used "Active Thread Count > -1").  Now go to the Alert tab of the RHQ server.  Notice there is no child alert definition created, nor does the alert fire.  I have run this test twice, in each case, the alert def eventtually appears and starts firing, but not until 1-3 hours later.

tested on solaris/postgres and linux/postgres.

Comment 1 Corey Welton 2009-08-26 18:45:08 UTC

Pushed to 1.4

Comment 2 Red Hat Bugzilla 2009-11-10 21:00:56 UTC

This bug was previously known as http://jira.rhq-project.org/browse/RHQ-2257

Comment 3 wes hayutin 2010-02-07 14:40:14 UTC

Steve,
This still looks like an issue in rhq master. Can you please work with Heiko and find out why the scenario in the description takes 1-3 hours for alerts to fire.

Can we adjust a setting, or does something need to be fixed.
Thank you!

Comment 4 wes hayutin 2010-02-17 13:27:04 UTC

mass move to rhq_chainsaw tracker bug

Comment 5 wes hayutin 2010-02-18 14:49:31 UTC

This bug has now been triaged by Chainsaw on 2/18. The expectation is the bug to be addressed by the end of sprint06 roughly 3/10/10.

Comment 6 Joseph Marques 2010-05-04 13:46:03 UTC

commit 6d09c27b3dee045a909e820eaaa1ecb7e378906a

fix performance when creating/updating alert definitions en masse
    
1) revamp mechanism from hibernate session manipulation to transactional boundary manipulation
2a) make aggregate (template & group) create/update NOT execute in transaction
2b) make individual/child create/update alert definition execute in their own transactions
2c) no longer continue create/update logic on exception, hault processing and show exception to the UI
2d) purge unmatched AlertDampeningEvents and AlertConditionLogs in its own transaction
3a) change cache loading/reloading so that the main logic occurs outside a transaction
3b) when the cache queries the DB, that will occur in a transaction
4) remove flush/clear methods, which are no longer needed with small transactions
5) implement more robust cache reloading, retrying on exceptions due to hibernate session consistency issues

Comment 7 Joseph Marques 2010-05-12 21:02:51 UTC

commit de743f644555cd0d8857eabb779dca621e5a0cfa

change error-handling for template/group alert definition create/update workflows
    
* instead of failing fast on first error, presume that errors on rare and continue on failure
* after the create/update workflow completes, if any errors, throw the first exception back tot he caller
* exceptions indicate that one or more updates failed and will give the cause
* any updates that didn't experience an exception will be updated, because each individual alert definition update occurred in its own transaction

Comment 8 Jeff Weiss 2010-05-13 12:49:29 UTC

QA Verified (by automated overnight run).  Postgres/Linux build b75aca8.

Comment 10 Corey Welton 2010-08-12 16:50:31 UTC

Mass-closure of verified bugs against JON.