Bug 717762

Summary: RHQ agent does not collect information for detected JBoss 5 servers on AIX 6.1
Product: [Other] RHQ Project Reporter: David <david.vanbalen>
Component: AgentAssignee: Nobody <nobody>
Status: NEW --- QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: 3.0.1CC: hrupp
Target Milestone: ---   
Target Release: ---   
Hardware: powerpc   
OS: Other   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description David 2011-06-29 19:43:26 UTC
Description of problem:

When running an RHQ agent on an AIX 6.1 server with two JBoss 5.0 servers running on it (one EAP and one SOA-P), the agent will detect both of them so they can be imported through the RHQ server's UI. However, no information or metrics are never provided to the RHQ server for either of the JBoss servers, with both always being shown as being down, regardless of their actual state.

Version-Release number of selected component (if applicable):

RHQ server 2.4.1
RHQ agent 3.0.1
JBoss EAP: 5.0
JBoss SOA-P 5.0
AIX 6.1
JDK: IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc64-64 jvmap6460sr7-20091214_49398 (JIT enabled, AOT enabled)

How reproducible:

Always

Steps to Reproduce:
1. Install RHQ agent, as per instructions in JBoss Operations Network 2.4 Installation Guide
2. Run the agent
3. Initiate a discovery (note that running JBoss 5 servers are detected)
4. Import servers via RHQ server UI (note that they will be shown as being down, even if they are running)
  
Actual results:

No information, including availability, is provided for running JBoss5 servers, and the stack trace from the log snippet below appears in the logs when the agent attempts an availability scan (note that it is only visible with debug turned on):

2011-06-23 14:46:51,191 DEBUG [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Running Availability Scan...
2011-06-23 14:46:54,298 DEBUG [ResourceContainer.invoker.daemon-11] (rhq.plugins.jbossas5.connection.RemoteProfileServiceConnectionProvider)- Connecting to Profile Service via remote JNDI using env [{java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory, java.naming.provider.url=jnp://10.2.201.77:11562, jnp.disableDiscovery=true, jnp.timeout=60000, jnp.sotimeout=60000}]...
2011-06-23 14:46:54,298 DEBUG [ResourceContainer.invoker.daemon-11] (rhq.plugins.jbossas5.connection.RemoteProfileServiceConnectionProvider)- Creating JNDI InitialContext with env [{java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory, java.naming.provider.url=jnp://10.2.201.77:11562, jnp.disableDiscovery=true, jnp.timeout=60000, jnp.sotimeout=60000}]...
2011-06-23 14:46:54,298 DEBUG [ResourceContainer.invoker.daemon-11] (rhq.plugins.jbossas5.connection.RemoteProfileServiceConnectionProvider)- Created JNDI InitialContext [javax.naming.InitialContext@49914991].
2011-06-23 14:46:54,298 DEBUG [ResourceContainer.invoker.daemon-11] (org.jnp.interfaces.TimedSocketFactory)- createSocket, hostAddr: /10.2.201.77, port: 11562, localAddr: null, localPort: 0, timeout: 60000
2011-06-23 14:46:54,301 DEBUG [ResourceContainer.invoker.daemon-11] (org.rhq.plugins.jbossas5.ApplicationServerComponent)- Failed to connect to Profile Service.
java.lang.RuntimeException: Failed to lookup JNDI name 'ProfileService' from InitialContext.
	at org.rhq.plugins.jbossas5.connection.AbstractProfileServiceConnectionProvider.lookup(AbstractProfileServiceConnectionProvider.java:84)
	at org.rhq.plugins.jbossas5.connection.RemoteProfileServiceConnectionProvider.doConnect(RemoteProfileServiceConnectionProvider.java:108)
	at org.rhq.plugins.jbossas5.connection.RemoteProfileServiceConnectionProvider.doConnect(RemoteProfileServiceConnectionProvider.java:43)
	at org.rhq.plugins.jbossas5.connection.AbstractProfileServiceConnectionProvider.connect(AbstractProfileServiceConnectionProvider.java:42)
	at org.rhq.plugins.jbossas5.ApplicationServerComponent.connectToProfileService(ApplicationServerComponent.java:388)
	at org.rhq.plugins.jbossas5.ApplicationServerComponent.getAvailabilityNow(ApplicationServerComponent.java:141)
	at org.rhq.plugins.jbossas5.ApplicationServerComponent.getAvailability(ApplicationServerComponent.java:136)
	at sun.reflect.GeneratedMethodAccessor72.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:600)
	at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:525)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:315)
	at java.util.concurrent.FutureTask.run(FutureTask.java:150)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:862)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:736)
Caused by: 
javax.naming.CommunicationException [Root exception is java.io.InvalidClassException: org.jboss.remoting.InvokerLocator; local class incompatible: stream classdesc serialVersionUID = -2909329895029296248, local class serialVersionUID = -4977622166779282521]
	at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:839)
	at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:686)
	at javax.naming.InitialContext.lookup(InitialContext.java:455)
	at org.rhq.plugins.jbossas5.connection.AbstractProfileServiceConnectionProvider.lookup(AbstractProfileServiceConnectionProvider.java:81)
	... 15 more
Caused by: 
java.io.InvalidClassException: org.jboss.remoting.InvokerLocator; local class incompatible: stream classdesc serialVersionUID = -2909329895029296248, local class serialVersionUID = -4977622166779282521
	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:574)
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1587)
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1500)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1736)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1333)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:352)
	at org.jboss.aop.metadata.SimpleMetaData.readExternal(SimpleMetaData.java:348)
	at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1796)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1755)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1333)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1951)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1875)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1757)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1333)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1951)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1875)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1757)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1333)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1951)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1875)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1757)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1333)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:352)
	at java.rmi.MarshalledObject.get(MarshalledObject.java:142)
	at org.jnp.interfaces.MarshalledValuePair.get(MarshalledValuePair.java:72)
	at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:769)
	... 18 more




Expected results:

After JBoss 5 servers are detected, the agent should collect information about them and send it to the RHQ server.


Additional info:

Comment 1 Marc Shirley 2011-07-20 16:12:44 UTC
Issue is due to 5.x Application Platforms using the "-Djboss.remoting.pre_2_0_compatible=true" Java option.  This results in an inability of the agent to connect to the ProfileService in some cases due to the class incompatibility caused by the JVM returning a different serialVersionUID.  

The AS/AS5 plugin should include a check for the "-Djboss.remoting.pre_2_0_compatible=true" option in order to account for this issue.