Bug 564514 - NPE in Agent when scanning JVM with password-protected JMX
Summary: NPE in Agent when scanning JVM with password-protected JMX
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: RHQ Project
Classification: Other
Component: Plugins
Version: 1.3
Hardware: All
OS: Linux
high
medium
Target Milestone: ---
: ---
Assignee: Stefan Negrea
QA Contact: Mike Foley
URL:
Whiteboard:
Depends On:
Blocks: ews1.0.2 rhq_triage 567481
TreeView+ depends on / blocked
 
Reported: 2010-02-12 22:45 UTC by Ondřej Žižka
Modified: 2013-09-02 07:25 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2013-09-02 07:25:32 UTC
Embargoed:


Attachments (Terms of Use)
Agent log. (18.08 KB, text/plain)
2010-02-12 22:48 UTC, Ondřej Žižka
no flags Details

Description Ondřej Žižka 2010-02-12 22:45:17 UTC
StR:
====

1) Install Tomcat 5 and make it be monitored in JON.
2) Secure it's JMX with a password. *
3) Agent: discovery -f
4) Check in JON. The JVM is red.
5) Check Agent's log. Contains the ERRORs as below.

*) I've put this to the startup.bat:

JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.port=12121"
JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.authenticate=true"
JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.ssl=false"
JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.access.file=/mnt/jqa/jdk1.6.0_17-x86_64/jre/lib/management/jmxremote.access"
JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.password.file=/mnt/jqa/jdk1.6.0_17-x86_64/jre/lib/management/jmxremote.password"

 ...then create and edit the .password file accordingly - see
 http://java.sun.com/j2se/1.5.0/docs/guide/management/agent.html




Full log attached.


2010-02-12 23:13:25,224 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=10360, type=VM Class Loading System, key=java.lang:type=ClassLoading, name=Class Loading, parent=Tomcat Server JVM] from synchronized merge.
org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=10360, type=VM Class Loading System, key=java.lang:type=ClassLoading, name=Class Loading, parent=Tomcat Server JVM].
	at org.rhq.core.pc.inventory.InventoryManager.activateResource(InventoryManager.java:1283)
	at org.rhq.core.pc.inventory.InventoryManager.refreshResourceComponentState(InventoryManager.java:2256)
	at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:2057)
	at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:2063)
	at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:2063)
	at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:2063)
	at org.rhq.core.pc.inventory.InventoryManager.synchInventory(InventoryManager.java:807)
	at org.rhq.core.pc.inventory.InventoryManager.handleReport(InventoryManager.java:787)
	at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.call(AutoDiscoveryExecutor.java:121)
	at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.run(AutoDiscoveryExecutor.java:92)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:146)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:170)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)
	at java.lang.Thread.run(Thread.java:595)
Caused by: java.lang.NullPointerException
	at org.rhq.plugins.jmx.MBeanResourceComponent.loadBean(MBeanResourceComponent.java:183)
	at org.rhq.plugins.jmx.MBeanResourceComponent.loadBean(MBeanResourceComponent.java:169)
	at org.rhq.plugins.jmx.MBeanResourceComponent.start(MBeanResourceComponent.java:107)
	at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:525)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
	at java.util.concurrent.FutureTask.run(FutureTask.java:123)
	... 3 more
2010-02-12 23:13:25,225 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=10356, type=Operating System, key=java.lang:type=OperatingSystem, name=Operating system information, parent=Tomcat Server JVM] from synchronized merge.

Comment 1 Ondřej Žižka 2010-02-12 22:48:37 UTC
Created attachment 390602 [details]
Agent log.

Comment 2 Ondřej Žižka 2010-02-12 22:56:56 UTC
I've removed " of Tomcat 5" from the bug's title.
Perhaps it's not Tomcat-related, but I found it when testing EWS.

Comment 3 Ondřej Žižka 2010-02-12 23:00:30 UTC
Tomcat 5.5.28 from EWS 1.0.1 CR1 RHEL x64, download here:
http://download.devel.redhat.com/devel/candidates/JBEWS/1.0.1_CR1/RHEL5/

Comment 4 Ondřej Žižka 2010-02-12 23:02:49 UTC
Note that other Tomcat sub-nodes are OK;
and JVM monitoring works with unprotected JMX.

Comment 5 wes hayutin 2010-02-16 16:57:52 UTC
Temporarily adding the keyword "SubBug" so we can be sure we have accounted for all the bugs.

keyword:
new = Tracking + FutureFeature + SubBug

Comment 6 wes hayutin 2010-02-16 17:02:49 UTC
making sure we're not missing any bugs in rhq_triage

Comment 7 Corey Welton 2010-09-30 13:56:43 UTC
mazz, jshaughn - any thoughts?

Comment 8 John Mazzitelli 2010-09-30 14:11:05 UTC
looks like an NPE in the jmx plugin:

Caused by: java.lang.NullPointerException
 at
org.rhq.plugins.jmx.MBeanResourceComponent.loadBean(MBeanResourceComponent.java:183)


and this might be something jay s fixed a while ago. i don't have the bz handy, but there was an issue where the EMS bean cache was invalid and I think caused an NPE. In other words, this might be fixed alreaedy.

Comment 9 Charles Crouch 2011-03-08 12:54:37 UTC
Needs to be reproduced to see if it is still a problem. In particular the following information would be helpful:

a) Were the jmxremote settings enabled in Tomcat prior to it being initially discovered or added as part of enabling security? 
b) Is this a problem if Tomcat 5 is secured before its discovered.

Comment 10 Ondřej Žižka 2011-03-16 00:22:33 UTC
Still happens. Actually I reported this as a new JIRA, but not BZ.
https://issues.jboss.org/browse/JBQA-4380

Summary:

1) Have a secured Tomcat
2) *Don't* set principal and credentials in Connection tab
3) discovery [-f]

To answer Charles' questions:

a) No, jmx were disabled when initially discovered.
b) Not tried yet. Will try.

Comment 11 Ondřej Žižka 2011-03-16 02:07:10 UTC
To clarify, the sequence was:

1) Instalation with EWS default - no JMX
2) Inventorized
3) Enabled JMX without security
4) discovery -f
5) Secured JMX
6) discovery -f => NPE
7) set credentials in Connection tab => OK

Comment 12 Stefan Negrea 2011-10-12 19:57:02 UTC
The errors reported above are displayed only in the logs, they are not visible
to the users. The users will sees the resources without correct credentials as Down until the correct credentials are provided.

Comment 13 Stefan Negrea 2011-10-14 01:59:27 UTC
Updated the JMX plugin code to gracefully handle the case when the ems connection is null due to missing JMX credentials. Also, updated mod_cluster plugin code that had a similar loadBean method implementation.

When retesting this issue, the null pointer exception is replaced by an Ems exception. This new exception is normal since the resource requires authentication but no credentials have been provided yet by the user. The exception is visible only in the agent logs; in UI the resource is down until the user provides a correct set of username/password.

Here is a snippet from the new exception:
org.mc4j.ems.connection.EmsConnectException: Could not connect [service:jmx:rmi:///jndi/rmi://localhost:12345/jmxrmi] java.lang.SecurityException: Authentication failed! Username or password is null

Comment 14 Heiko W. Rupp 2013-09-02 07:25:32 UTC
Bulk closing of issues that were VERIFIED, had no target release and where the status changed more than a year ago.


Note You need to log in before you can comment on or make changes to this bug.