Red Hat Bugzilla – Bug 861979
when alert notification fails, it appears alert doesn't even get saved in history
Last modified: 2013-02-05 06:14:20 EST
Set up an alert with an alert notification (I was on the AS7 branch with a CLI script notification, which isn't working yet, so I expected the notification to fail)
Here's what I think is happening, in a general sense:
1) an alert's conditions go true - RHQ wants to send an alert
2) the alert definition has a notification definition
3) RHQ tries to send the notification, but "some bad" error occurs (I don't know of a good way to replicate this - in my case, I was in AS7 branch with CLI script notification that is known to fail because that stuff doesn't yet work in the AS7 branch)
4) it appears RHQ, on failure of notification, fails to then even add a row to RHQ_ALERT (so you don't even see the alert in the Alert history subtab)
4 is a bug, if its true that this is what happened
Targeting this for investigation in JON312
I could not reproduce this in RHQ 4.6.0-SNAPSHOT. CLI alerts seem to work OK now that the port to AS7 is complete.
Also, looking at the code, the firing of notifications is guarded by a catch of Throwable (which is logged and saved as the notification result). Also a mere failure to save a notification result is just logged and doesn't cause the overall process of alert firing to fail so I am sceptical that the CLI notifications in and of themselves were the sole cause of the perceived failure.
I am closing this as "works for me". If someone ever finds repro steps, I am all ears to hear them as this would be quite serious.