Bug 1035809

Summary: ConnectException: Connection refused to host: 127.0.0.1 with EPP 5.2.2 during server shutdown
Product: [JBoss] JBoss Enterprise Portal Platform 6 Reporter: mgottval <mgottval>
Component: JON PluginAssignee: Thomas Heute <theute>
Status: NEW --- QA Contact: Martin Vecera <mvecera>
Severity: low Docs Contact:
Priority: low    
Version: 6.1.0CC: atangrin, hbrock, loleary, mvecera, myarboro
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description mgottval 2013-11-28 14:22:42 UTC
1. Run EPP 5.2.2
2. Run PostgreSQL database
2. Install and start JON 3.2 by the script rhqctl

JON server discovers EPP and all metrics, when I try to shutdown the EPP server by right click to the EPP - operations - shutdown, in the agent log there appears warnings and then exceptions, but when the EPP server is started again, everything is alright:

WARN  [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Availability collection failed with exception on Resource[id=10198, uuid=a0dfd10b-0be2-45d0-99f7-58e9483156d6, type={GateIn}Portlet, key=portal@AdminToolbarPortlet/RENDER_PHASE, name=AdminToolbarPortlet/RENDER_PHASE, parent=portal], availability will be reported as DOWN
org.mc4j.ems.connection.EmsConnectException: java.rmi.ConnectException: Connection refused to host: 127.0.0.1; nested exception is: 
	java.net.ConnectException: Connection refused
	at org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBeanServerProxy.invoke(GenericMBeanServerProxy.java:110)
	at $Proxy51.isRegistered(Unknown Source)
	at org.mc4j.ems.impl.jmx.connection.bean.DMBean.isRegistered(DMBean.java:188)
	at org.rhq.plugins.jmx.MBeanResourceComponent.isMBeanAvailable(MBeanResourceComponent.java:242)
	at org.rhq.plugins.jmx.MBeanResourceComponent.getAvailability(MBeanResourceComponent.java:229)
	at org.gatein.rhq.plugins.jmx.MBeanAttributeResourceComponent.getAvailability(MBeanAttributeResourceComponent.java:118)
	at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocation.call(ResourceContainer.java:654)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
Caused by: java.rmi.ConnectException: Connection refused to host: 127.0.0.1; nested exception is: 
	java.net.ConnectException: Connection refused
	at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
	at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
	at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:128)
	at org.jboss.invocation.jrmp.server.JRMPInvoker_Stub.invoke(Unknown Source)
	at org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxy.invoke(JRMPInvokerProxy.java:133)
	at org.jboss.invocation.InvokerInterceptor.invokeInvoker(InvokerInterceptor.java:416)
.....

Comment 1 Larry O'Leary 2014-02-07 15:44:25 UTC
Moving this to JPP product as this is a management plug-in bug.

The plug-in should only log at debug when an availability check fails. Essentially, things are working as expected but we shouldn't see this warning. Instead, we should only see the above message when DEBUG is enabled.

Comment 3 Thomas Heute 2014-03-20 09:37:08 UTC
I don't think the JPP plugin does anything explicitly about availability and the stacktrace shows that RHQ generates the WARN:

https://github.com/rhq-project/rhq/blob/master/modules/core/plugin-container/src/main/java/org/rhq/core/pc/inventory/AvailabilityExecutor.java#L357

Larry, doesn't EAP 5 suffers from the same issue ?

Comment 4 Larry O'Leary 2014-03-20 15:41:23 UTC
No. The JBoss EAP 5 plug-in[1] doesn't suffer from this issue. It appears the problem in the JPP case is that the getAvailability method is throwing an exception. This is not expected. If the resource is down because the getAvailability method can't reach the server, then it should return a DOWN availability.

The plug-in container only expects to get a UP or DOWN response and an exception is considered a runtime failure. 

I am guessing that the GateIn plug-in's getAvailability method should handle its own exceptions and only throw an exception if something unexecpted happens (NPE, memory, etc).


[1]: https://github.com/rhq-project/rhq/blob/master/modules/plugins/jboss-as-5/src/main/java/org/rhq/plugins/jbossas5/ApplicationServerComponent.java#L163