Bug 564514
| Summary: | NPE in Agent when scanning JVM with password-protected JMX | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Other] RHQ Project | Reporter: | Ondřej Žižka <ozizka> | ||||
| Component: | Plugins | Assignee: | Stefan Negrea <snegrea> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Mike Foley <mfoley> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 1.3 | CC: | cwelton, mazz, snegrea | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | All | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2013-09-02 07:25:32 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 683054, 565628, 567481 | ||||||
| Attachments: |
|
||||||
Created attachment 390602 [details]
Agent log.
I've removed " of Tomcat 5" from the bug's title. Perhaps it's not Tomcat-related, but I found it when testing EWS. Tomcat 5.5.28 from EWS 1.0.1 CR1 RHEL x64, download here: http://download.devel.redhat.com/devel/candidates/JBEWS/1.0.1_CR1/RHEL5/ Note that other Tomcat sub-nodes are OK; and JVM monitoring works with unprotected JMX. Temporarily adding the keyword "SubBug" so we can be sure we have accounted for all the bugs. keyword: new = Tracking + FutureFeature + SubBug making sure we're not missing any bugs in rhq_triage mazz, jshaughn - any thoughts? looks like an NPE in the jmx plugin: Caused by: java.lang.NullPointerException at org.rhq.plugins.jmx.MBeanResourceComponent.loadBean(MBeanResourceComponent.java:183) and this might be something jay s fixed a while ago. i don't have the bz handy, but there was an issue where the EMS bean cache was invalid and I think caused an NPE. In other words, this might be fixed alreaedy. Needs to be reproduced to see if it is still a problem. In particular the following information would be helpful: a) Were the jmxremote settings enabled in Tomcat prior to it being initially discovered or added as part of enabling security? b) Is this a problem if Tomcat 5 is secured before its discovered. Still happens. Actually I reported this as a new JIRA, but not BZ. https://issues.jboss.org/browse/JBQA-4380 Summary: 1) Have a secured Tomcat 2) *Don't* set principal and credentials in Connection tab 3) discovery [-f] To answer Charles' questions: a) No, jmx were disabled when initially discovered. b) Not tried yet. Will try. To clarify, the sequence was: 1) Instalation with EWS default - no JMX 2) Inventorized 3) Enabled JMX without security 4) discovery -f 5) Secured JMX 6) discovery -f => NPE 7) set credentials in Connection tab => OK The errors reported above are displayed only in the logs, they are not visible to the users. The users will sees the resources without correct credentials as Down until the correct credentials are provided. Updated the JMX plugin code to gracefully handle the case when the ems connection is null due to missing JMX credentials. Also, updated mod_cluster plugin code that had a similar loadBean method implementation. When retesting this issue, the null pointer exception is replaced by an Ems exception. This new exception is normal since the resource requires authentication but no credentials have been provided yet by the user. The exception is visible only in the agent logs; in UI the resource is down until the user provides a correct set of username/password. Here is a snippet from the new exception: org.mc4j.ems.connection.EmsConnectException: Could not connect [service:jmx:rmi:///jndi/rmi://localhost:12345/jmxrmi] java.lang.SecurityException: Authentication failed! Username or password is null Bulk closing of issues that were VERIFIED, had no target release and where the status changed more than a year ago. |
StR: ==== 1) Install Tomcat 5 and make it be monitored in JON. 2) Secure it's JMX with a password. * 3) Agent: discovery -f 4) Check in JON. The JVM is red. 5) Check Agent's log. Contains the ERRORs as below. *) I've put this to the startup.bat: JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.port=12121" JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.authenticate=true" JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.ssl=false" JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.access.file=/mnt/jqa/jdk1.6.0_17-x86_64/jre/lib/management/jmxremote.access" JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.password.file=/mnt/jqa/jdk1.6.0_17-x86_64/jre/lib/management/jmxremote.password" ...then create and edit the .password file accordingly - see http://java.sun.com/j2se/1.5.0/docs/guide/management/agent.html Full log attached. 2010-02-12 23:13:25,224 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=10360, type=VM Class Loading System, key=java.lang:type=ClassLoading, name=Class Loading, parent=Tomcat Server JVM] from synchronized merge. org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=10360, type=VM Class Loading System, key=java.lang:type=ClassLoading, name=Class Loading, parent=Tomcat Server JVM]. at org.rhq.core.pc.inventory.InventoryManager.activateResource(InventoryManager.java:1283) at org.rhq.core.pc.inventory.InventoryManager.refreshResourceComponentState(InventoryManager.java:2256) at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:2057) at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:2063) at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:2063) at org.rhq.core.pc.inventory.InventoryManager.processSyncInfo(InventoryManager.java:2063) at org.rhq.core.pc.inventory.InventoryManager.synchInventory(InventoryManager.java:807) at org.rhq.core.pc.inventory.InventoryManager.handleReport(InventoryManager.java:787) at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.call(AutoDiscoveryExecutor.java:121) at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.run(AutoDiscoveryExecutor.java:92) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:146) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:170) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676) at java.lang.Thread.run(Thread.java:595) Caused by: java.lang.NullPointerException at org.rhq.plugins.jmx.MBeanResourceComponent.loadBean(MBeanResourceComponent.java:183) at org.rhq.plugins.jmx.MBeanResourceComponent.loadBean(MBeanResourceComponent.java:169) at org.rhq.plugins.jmx.MBeanResourceComponent.start(MBeanResourceComponent.java:107) at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:525) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269) at java.util.concurrent.FutureTask.run(FutureTask.java:123) ... 3 more 2010-02-12 23:13:25,225 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=10356, type=Operating System, key=java.lang:type=OperatingSystem, name=Operating system information, parent=Tomcat Server JVM] from synchronized merge.