Bug 861979 - when alert notification fails, it appears alert doesn't even get saved in history
when alert notification fails, it appears alert doesn't even get saved in his...
Status: CLOSED WORKSFORME
Product: JBoss Operations Network
Classification: JBoss
Component: Monitoring - Alerts (Show other bugs)
JON 3.1.0
Unspecified Unspecified
unspecified Severity high
: ---
: JON 3.2.0
Assigned To: RHQ Project Maintainer
Mike Foley
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-01 09:47 EDT by John Mazzitelli
Modified: 2013-02-05 06:14 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-02-05 06:14:20 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description John Mazzitelli 2012-10-01 09:47:36 EDT
Set up an alert with an alert notification (I was on the AS7 branch with a CLI script notification, which isn't working yet, so I expected the notification to fail)

Here's what I think is happening, in a general sense:

1)  an alert's conditions go true - RHQ wants to send an alert
2) the alert definition has a notification definition 
3) RHQ tries to send the notification, but "some bad" error occurs (I don't know of a good way to replicate this - in my case, I was in AS7 branch with CLI script notification that is known to fail because that stuff doesn't yet work in the AS7 branch)
4) it appears RHQ, on failure of notification, fails to then even add a row to RHQ_ALERT (so you don't even see the alert in the Alert history subtab)

4 is a bug, if its true that this is what happened
Comment 1 Charles Crouch 2012-10-01 11:25:06 EDT
Targeting this for investigation in JON312
Comment 2 Lukas Krejci 2013-02-05 06:14:20 EST
I could not reproduce this in RHQ 4.6.0-SNAPSHOT. CLI alerts seem to work OK now that the port to AS7 is complete.

Also, looking at the code, the firing of notifications is guarded by a catch of Throwable (which is logged and saved as the notification result). Also a mere failure to save a notification result is just logged and doesn't cause the overall process of alert firing to fail so I am sceptical that the CLI notifications in and of themselves were the sole cause of the perceived failure.

I am closing this as "works for me". If someone ever finds repro steps, I am all ears to hear them as this would be quite serious.

Note You need to log in before you can comment on or make changes to this bug.