Are you monitoring that server with JON?

yes, it doesn't happen when JON agent is down

The ESB JON plugin is clearing the alert queue once it has collected the alerts from the MBean.

Is this consistent with other information used/collected by JON? It's no longer available after JON collects it? Or - is this configurable? Either way - we will need to document this for users.

Could this be a difference between full JON (enables historical data) and embedded JON?

Hey Tom, since the ESB JON plugin consumes these messages, where does it use this data? Is it exposed somewhere?

This is expected behaviour - should expectations be changed, if so what should be expected

ER7 + admin console:

When I sent some messages the alerts show up in the jmx console. When I then login to the admin console and refresh the MBean view in the jmx console, all alerts disappear and following NPE is thrown every 60 seconds:

server.log:

05:01:10,344 WARN  [EventManager] Failure to report Events to Server.
java.lang.NullPointerException
	at org.rhq.core.pc.event.EventManager.sendEventReport(EventManager.java:133)
	at org.rhq.core.pc.event.EventSenderRunner.call(EventSenderRunner.java:48)
	at org.rhq.core.pc.event.EventSenderRunner.run(EventSenderRunner.java:54)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:619)

Assigning to Charles - this sounds like a JON bug.

Clearing the message alerts is expected behavior for the JON plugin.       Not sure what the NPE here is - do we know it is definitely related?

Question: 

If the JON Server or the Embedded Console are consuming these alerts then the collected alerts cannot be accessed programmatically by a user via JMX.  

They will have to access these via the JON API. 

Where is the best reference to refer users to for this task?

Here's my understanding:

You could access JON's resources via the command line interface (CLI), here is the documentation:
https://www.redhat.com/docs/en-US/JBoss_ON/2.3/html/Installation_Guide/Installation_Guide-Command_Line_Interface_Installation.html
https://www.redhat.com/docs/en-US/JBoss_ON/2.3/html/Installation_Guide/Installation_Guide-JON_Best_Practices-Remote_Clients.html
http://community.jboss.org/wiki/JON23ScriptedGroupDeploymentsUsingTheCLIAPI
http://community.jboss.org/wiki/JON23ControlGroupedServersViaTheCLI

=======================

W.r.t. the embedded admin console: once these message alerts are consumed, they are no longer available (at least not through the web interface, no idea if there is another way to get those message alerts).
JON however allows to you to setup *JON Alerts* triggered by events which are based up on the *server log* and not the MessageAlerts MBean. I guess these JON Alerts could be accessed via the CLI.

So, you would have to setup an event that will be activated if a WARN message shows up in the server log. Then you can set up JON Alerts that will fire if a particular WARN message was logged. And then the CLI... but keep in mind that what you get with the CLI are not those "original" message alerts, but JON Alerts.

Hi Jarek
Can you add links to where to download the SOAP server used to reproduce this issue. Also we need the exact reproduction steps, e.g. which jmx console pages were visited to trigger the NPE.

1. download SOA-P 5 ER8 from
http://hudson.qa.jboss.com/hudson/view/SOA-Release/job/soa-5.0/49/artifact/binaries/soa-5.0.0.ER8.zip
http://jawa05.englab.brq.redhat.com/candidate/soa-5.0.0_ER8/soa-5.0.0.ER8.zip
md5sum: 66f4c2f48daafc7ae7ad41dd2f1b3762

2. unzip, add user admin=admin in jboss-soa-p.5.0.0/jboss-as/server/production/conf/props/soa-users.properties, start the production config

3. create jboss-soa-p.5.0.0/jboss-as/samples/quickstarts/quickstarts.properties
org.jboss.esb.server.config=production
jbpm.console.username=admin
jbpm.console.password=admin

4. go to jboss-soa-p.5.0.0/jboss-as/samples/quickstarts/messagealerts
$ ant deploy
verify esb has been deployed successfully
$ ant runtest

5. verify 4 WARN messages showed up in the server console

so far so good...

6. open the admin console and login as admin/admin
http://x.x.x.x:8080/admin-console/

7. wait for the NPE, it shows up every 60 seconds:
04:38:59,490 WARN  [EventManager] Failure to report Events to Server.
java.lang.NullPointerException
	at org.rhq.core.pc.event.EventManager.sendEventReport(EventManager.java:133)
	at org.rhq.core.pc.event.EventSenderRunner.call(EventSenderRunner.java:48)
	at org.rhq.core.pc.event.EventSenderRunner.run(EventSenderRunner.java:54)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:619)



If there wouldn't be any alerts, this NPE wouldn't show up.

To clarify for the 5.0.0 release notes: the NPE only appears in the log, no error messages appear in the JMX console itself ?

That's right.

Cool, will document as a known issue

This may be completely irrelevant but I noticed these warnings when accessing the admin-console for the first time.  This is on OSX 10.6.2
I was just wondering about the WARN messages regarding native libraries and if they might be contributing since this bug doesn't manifest until admin-console is initialized for the first time.

The last WARN is of course the one that repeats as described above with the exception


2010-02-17 11:02:41,868 INFO  [org.jboss.on.embedded.ui.BootstrapAction] (http-127.0.0.1-8080-3) Initializing Administration Console v1.3.2.GA...
2010-02-17 11:02:45,422 WARN  [org.rhq.core.system.SystemInfoFactory] (http-127.0.0.1-8080-3) System info API not accessible on this platform (native shared library not found in java.library.path).
2010-02-17 11:02:45,658 INFO  [org.jboss.on.embedded.manager.pc.PluginContainerResourceManager] (http-127.0.0.1-8080-3) Discovering Resources...
2010-02-17 11:02:45,731 WARN  [org.rhq.plugins.jmx.JMXDiscoveryComponent] (ResourceDiscoveryComponent.invoker.daemon-1) Unable to complete base jmx server discovery (enable DEBUG for stack): java.lang.UnsupportedOperationException: No native library available - Cannot get the process table information without native support
2010-02-17 11:03:15,674 WARN  [org.rhq.core.pc.event.EventManager] (EventManager.sender-1) Failure to report Events to Server.

yeah, this is already filed as SOA-1760 and SOA-1825

I did a draft document of this one for the Release Notes. Text reads:


https://jira.jboss.org/jira/browse/SOA-1660

    When sending messages, the alerts will appear in the JMX Console. If you then log into
    the Administration Console and refresh the Mbean view in the JMX Console, the
    java.lang.NullPointerException will appear once per minute. This exception only appears
    in the log; it is not visible in the JMX Console itself.

It turned out that you don't have to go to the jmx-console, the NPE shows up after you login to the admin console and is repeated every 60 secs, so please remove: 

"and refresh the Mbean view in the JMX Console,"

Besides that, IMHO it should be noted that the MessageAlerts queue is emptied when SOA is monitored by JON or the admin console has been accessed. Maybe something like this:

"You can use the jmx-console to display message alerts (jboss.esb:service=MessageAlerts). If SOA is monitored by JON or the admin console has been accessed, message alerts will disappear when the MBean view is refreshed because the MessageAlerts queue is emptied."

Text now reads:

https://jira.jboss.org/jira/browse/SOA-1660
    When sending messages, the alerts will appear in the JMX Console. If you then log into the
    Administration Console, the java.lang.NullPointerException will appear once per
    minute. This exception only appears in the log; it is not visible in the JMX Console itself.

    Note that you can use the JMX Console to display message alerts
    (jboss.esb:service=MessageAlerts). However, if the SOA Platform is monitored by JON or
    if the Administration Console has been accessed, message alerts will disappear when the
    Mbean view is refreshed because the MessageAlerts queue will be emptied.

Verified release notes.

Rel notes have been verified and it is expected that the queue is emptied by the JON plugin - alerts are displayed in the EVENTS tab of the corresponding ESB Service.

Link: Added: This issue related EMBJOPR-290