Here's the stack trace: 2011-09-12 16:17:51,102 INFO [ResourceContainer.invoker.daemon-1] (org.mc4j.ems.connection.ConnectionFactory)- Discovered libraries in 0 ms 2011-09-12 16:17:51,194 INFO [ResourceContainer.invoker.daemon-1] (ems.impl.jmx.connection.DConnection)- Querying MBeanServer for all MBeans 2011-09-12 16:17:51,196 WARN [ResourceContainer.invoker.daemon-1] (jboss.on.common.jbossas.JmxConnectionHelper)- Could not establish connection to the instance [1] times. java.lang.reflect.UndeclaredThrowableException at $Proxy57.queryNames(Unknown Source) at org.mc4j.ems.impl.jmx.connection.DConnection.loadSynchronous(DConnection.java:138) at org.jboss.on.common.jbossas.JmxConnectionHelper.loadConnection(JmxConnectionHelper.java:213) at org.jboss.on.common.jbossas.JmxConnectionHelper.getEmsConnection(JmxConnectionHelper.java:101) at org.rhq.plugins.jbossas5.ApplicationServerComponent.initializeEmsConnection(ApplicationServerComponent.java:502) at org.rhq.plugins.jbossas5.ApplicationServerComponent.start(ApplicationServerComponent.java:180) at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:537) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.IllegalAccessException: Class org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBeanServerProxy can not access a member of class javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection with modifiers "public" at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65) at java.lang.reflect.Method.invoke(Method.java:588) at org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBeanServerProxy.invoke(GenericMBeanServerProxy.java:123) ... 15 more --- This is not as issue when connecting to EAP5, so there were obviously some changes made to the JBoss MBeanServer class in AS 6.0 that make the queryNames() method inaccessible via reflection. There are some tricks that can be done in EMS to workaround this.
This is fixed by [master 8bb8c6d], which upgrades EMS from 1.2.15.1 to 1.2.16. The fix that was added in EMS 1.2.16 was to ensure the java.lang.Method object used in the MBeanServer proxy (org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBeanServerProxy) is accessible, prior to invoking it. Here's the code that attempts to obtain an accessible version of the Method: private static Method getAccessibleMethod(Method method) { try { method.setAccessible(true); return method; } catch (SecurityException e) { // If the class declaring the method is itself not public, the method will not be accessible, so attempt to // find a method with the same signature on one of the server class's interfaces. if (!Modifier.isPublic(method.getDeclaringClass().getModifiers())) { Method interfaceMethod = getInterfaceMethod(method); if (interfaceMethod != null) { return interfaceMethod; } } return method; } } private static Method getInterfaceMethod(Method method) { Class<?>[] interfaceClasses = method.getClass().getInterfaces(); for (int i = 0, interfaceClassesLength = interfaceClasses.length; i < interfaceClassesLength; i++) { Class interfaceClass = interfaceClasses[i]; try { return interfaceClass.getMethod(method.getName(), method.getParameterTypes()); } catch (NoSuchMethodException e) { // ignore } catch (SecurityException e) { // ignore } } // Return null to indicate we were unable to find an interface method. return null; }
That change is pretty old and has been on the release branch for ever (as far as I can see). This version of EMS got overwritten on Nov 4th by the change of Bug 701375
Can monitor AS6 nicely, no more exception in the log. BUT after rebooting as6, I can no longer monitor the JVM part of it ResourceError is showing org.mc4j.ems.connection.EmsConnectException: java.io.IOException: The client has been closed. at org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBeanServerProxy.invoke(GenericMBeanServerProxy.java:135) at $Proxy68.isRegistered(Unknown Source) at org.mc4j.ems.impl.jmx.connection.bean.DMBean.isRegistered(DMBean.java:188) at org.rhq.plugins.jmx.MBeanResourceComponent.isMBeanAvailable(MBeanResourceComponent.java:240) at org.rhq.plugins.jmx.MBeanResourceComponent.getAvailability(MBeanResourceComponent.java:227) at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:552) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) Caused by: java.io.IOException: The client has been closed. at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.restart(ClientCommunicatorAdmin.java:74) at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.gotIOException(ClientCommunicatorAdmin.java:34) at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1437) at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.isRegistered(RMIConnector.java:841) at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBeanServerProxy.invokeInternal(GenericMBeanServerProxy.java:168) at org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBeanServerProxy.invoke(GenericMBeanServerProxy.java:103) ... 13 more
Heiko, please triage this legacy Ian issue. Thanks.
I think this got fixed with all the changes in JXM handling and re-establishing of connections.