Bug 779265 (SOA-1656)

Summary: Individual message's processing time not available as separate metric
Product: [JBoss] JBoss Enterprise SOA Platform 5 Reporter: Jaroslaw Kijanowski <kijanowski>
Component: Monitoring and ManagementAssignee: David Le Sage <dlesage>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: 5.0.0 ER4   
Target Milestone: ---   
Target Release: 5.0.0 GA   
Hardware: Unspecified   
OS: Unspecified   
URL: http://jira.jboss.org/jira/browse/SOA-1656
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-02-19 08:01:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 779266    
Bug Blocks:    

Description Jaroslaw Kijanowski 2009-11-30 15:33:56 UTC
Date of First Response: 2009-11-30 10:53:50
project_key: SOA

From the PRD:

=======
The current response time is an average across all messages within a time window, alerts are based on that average but there is no way to determine if a single message failed to meet the alerting rules.

A JON alert is setup to fire if an individual message's processing time exceeds N seconds.  The currently solution only looks at message averages.
=======

Individual message's processing time isn't available.

Comment 1 Kevin Conner 2009-11-30 15:53:50 UTC
The ERD, which is the definitive document, contains the following

"tracer bullet" type feature (track the message through the ESB, track the changes that have taken place on the message, track metadata with the message, and then view through JON what happened and how long it took) is deferred.

"alerting" mechanism (alerts if a message took over a set amount of time to be processed, or if a message that was bigger than some specified size was being processed) is committed.

Given that this mechanism is in place, and Len has even written a blog about it, what functionality do you feel is missing?

Comment 2 Jaroslaw Kijanowski 2009-11-30 16:03:26 UTC
Link: Added: This issue is a dependency of JBESB-3012


Comment 3 Jaroslaw Kijanowski 2009-11-30 16:07:26 UTC
Link: Added: This issue related JBQA-2726


Comment 4 Kevin Conner 2009-11-30 16:09:15 UTC
Link: Removed: This issue is a dependency of JBESB-3012 


Comment 5 Jaroslaw Kijanowski 2009-11-30 16:12:04 UTC
Ok, I understood it that the alerting mechanism should be also available at the JON level, not just the esb level (Len's blog).
So I'm wrong, right?

Comment 6 Kevin Conner 2009-11-30 16:34:05 UTC
I have no idea whether you are wrong or not as I don't yet understand what it is which you believe to be missing.

As far as I am aware there should be some JON integration with the alerts, I'll find out the details.

In the meantime, please add a comment to this issue explaining what you believe to be missing.

Comment 7 tcunning 2009-11-30 16:45:55 UTC
The JON integration for message alerts collects the alerts and displays them as JON events, or if the administrator wishes to do so, as JON alerts that are fired triggered by the message alert JON event.       Both the message's size and the processing time are available within that context.

Comment 8 Jaroslaw Kijanowski 2009-11-30 22:27:30 UTC
Thanks a lot Tom! This makes sense now.. and it works.
Reading the PRD/ERD I thought this 'message's individual processing time' would be available in JON as a separate metric which could be used to set up alerts.

IMHO it should be clearly documented that first you need to 'instrument' your esb deployment with the alerting feature (alertTimeThreshold,  alertLengthThreshold) and then, in the JON GUI, setup alerts for events filtered from the server log.

Comment 9 Jaroslaw Kijanowski 2009-12-07 13:27:51 UTC
These two strictly JON related issues will impact customers:
https://bugzilla.redhat.com/show_bug.cgi?id=542828
https://bugzilla.redhat.com/show_bug.cgi?id=545062

Need to be documented as well.

Comment 10 Len DiMaggio 2010-01-28 20:44:04 UTC
Link: Added: This issue depends SOA-1657


Comment 11 David Le Sage 2010-02-12 06:52:31 UTC
These issues have been documented for the Release Notes.  Here is the draft text:


https://jira.jboss.org/jira/browse/SOA-1656
    
    The ability to set up a message's individual processing time in JON for alerts does not work by
    default.

    To work around this problem, firstly add the alerting feature (alertTimeThreshold and
    alertLengthThreshold) to the ESB deployment.
    Having done so,. you must then set up alerts for events filtered from the server log. Do this using
    the JON GUI.

    Note that, in some situations, JON cannot filter correctly for log messages containing underscores
    (_). If event logging has been enabled and the you search for a log message with with that
    character via the Details Filter box, they will not be displayed.

    Another issue with JON event logging is that there is a delay in receiving alerts based on WARN
    messages. If event logging has been enabled and alerts have been configured to display on these
    messages, they may not appear immediately in the Alert view. The delay may be between fifteen
    and fifty minutes in length.


Comment 12 Jaroslaw Kijanowski 2010-02-12 19:16:18 UTC
Please remove the dot in "Having done so,. you must then"

====================

This can be removed, it's fixed and going to be in the JON 2.3.1 release:
"Note that, in some situations, JON cannot filter correctly for log messages containing underscores (_). If event logging has been enabled and the you search for a log message with with that character via the Details Filter box, they will not be displayed."

====================

IMHO this needs to be reworded:
"Another issue with JON event logging is that there is a delay in receiving alerts based on WARN messages. If event logging has been enabled and alerts have been configured to display on these messages, they may not appear immediately in the Alert view. The delay may be between fifteen and fifty minutes in length."

The problem here is more critical, since before the whole alerts mechanism kicks in (which may take 15 minutes, 50 minutes, but in general an *unknown* ammount of time) all JON Alerts *get lost* - they will never show up. The WARN messages from the SOA-P server console however will be displayed in the EVENTS view.

Comment 13 David Le Sage 2010-02-15 00:34:39 UTC
Hello Jaroslaw, 

Thanks for your feedback.


Second draft reads:


https://jira.jboss.org/jira/browse/SOA-1656

    The ability to set up a message's individual processing time in JON for alerts does not work by
    default.

    To work around this problem, firstly add the alerting feature (alertTimeThreshold and
    alertLengthThreshold) to the ESB deployment.

    Having done so, you must then set up alerts for events filtered from the server log. Do this using
    the JON GUI.

    Another issue with JON event logging is that the alerts mechanism may be delayed in starting
    for an indefiinite period of time (possibly between fifteen and fifty minutes in some cases.) Whilst
    this delay is occurring, all JON Alerts are lost and will never be seen. (However, WARN messages,
    emanating from the SOA Platform Server Console, will be displayed in the EVENTS view.)


Comment 14 Jaroslaw Kijanowski 2010-02-15 08:01:24 UTC
Just perfect! Thanks.

Comment 16 Jaroslaw Kijanowski 2010-02-19 07:59:22 UTC
reopening to reword the title of this jira