Bug 949447

Summary:	Intermittent failures in RemoteEJBClientStatefulBeanFailoverTestCase
Product:	[JBoss] JBoss Enterprise Application Platform 6	Reporter:	Richard Janík <rjanik>
Component:	Clustering	Assignee:	Richard Achmatowicz <rachmato>
Status:	CLOSED WONTFIX	QA Contact:	Michal Vinkler <mvinkler>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	6.1.0	CC:	bbaranow, jkudrnac, lthon, myarboro, rachmato, rhusar
Target Milestone:	---
Target Release:	EAP 6.4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	Clustering testsuite
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-03-01 12:29:02 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	996500

Description Richard Janík 2013-04-08 08:11:23 UTC

Description of problem:

Two tests randomly fail:
org.jboss.as.test.clustering.cluster.ejb3.stateful.remote.failover.RemoteEJBClientStatefulBeanFailoverTestCase(SYNC-tcp).testFailoverFromRemoteClientWhenOneNodeGoesDown
org.jboss.as.test.clustering.cluster.ejb3.stateful.remote.failover.RemoteEJBClientStatefulBeanFailoverTestCase(SYNC-tcp).testFailoverFromRemoteClientWhenOneNodeUndeploys

(tcp is configured to use TCPGOSSIP as in https://bugzilla.redhat.com/show_bug.cgi?id=922662)

Stacktrace for the first test:

javax.ejb.EJBException: java.io.IOException: Channel Channel ID d34781b2 (outbound) of Remoting connection 0ac70ac7 to /127.0.0.1:4447 has been closed
	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:216)
	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:161)
	at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:124)
	at com.sun.proxy.$Proxy22.establishView(Unknown Source)
	at org.jboss.as.test.clustering.cluster.ejb3.stateful.remote.failover.RemoteEJBClientStatefulBeanFailoverTestCase.establishView(RemoteEJBClientStatefulBeanFailoverTestCase.java:245)
	at org.jboss.as.test.clustering.cluster.ejb3.stateful.remote.failover.RemoteEJBClientStatefulBeanFailoverTestCase.failoverFromRemoteClient(RemoteEJBClientStatefulBeanFailoverTestCase.java:160)
	at org.jboss.as.test.clustering.cluster.ejb3.stateful.remote.failover.RemoteEJBClientStatefulBeanFailoverTestCase.testFailoverFromRemoteClientWhenOneNodeGoesDown(RemoteEJBClientStatefulBeanFailoverTestCase.java:129)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.jboss.arquillian.junit.Arquillian$6$1.invoke(Arquillian.java:270)
	at org.jboss.arquillian.container.test.impl.execution.LocalTestExecuter.execute(LocalTestExecuter.java:60)
	at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:90)
	at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
	at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
	at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:135)
	at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:115)
	at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)
	at org.jboss.arquillian.container.test.impl.execution.ClientTestExecuter.execute(ClientTestExecuter.java:53)
	at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:90)
	at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
	at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
	at org.jboss.arquillian.container.test.impl.client.ContainerEventController.createContext(ContainerEventController.java:142)
	at org.jboss.arquillian.container.test.impl.client.ContainerEventController.createTestContext(ContainerEventController.java:129)
	at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:90)
	at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
	at org.jboss.arquillian.test.impl.TestContextHandler.createTestContext(TestContextHandler.java:89)
	at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:90)
	at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
	at org.jboss.arquillian.test.impl.TestContextHandler.createClassContext(TestContextHandler.java:75)
	at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:90)
	at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
	at org.jboss.arquillian.test.impl.TestContextHandler.createSuiteContext(TestContextHandler.java:60)
	at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:90)
	at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
	at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:135)
	at org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.test(EventTestRunnerAdaptor.java:111)
	at org.jboss.arquillian.junit.Arquillian$6.evaluate(Arquillian.java:263)
	at org.jboss.arquillian.junit.Arquillian$4.evaluate(Arquillian.java:226)
	at org.jboss.arquillian.junit.Arquillian.multiExecute(Arquillian.java:314)
	at org.jboss.arquillian.junit.Arquillian.access$100(Arquillian.java:46)
	at org.jboss.arquillian.junit.Arquillian$5.evaluate(Arquillian.java:240)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.jboss.arquillian.junit.Arquillian$2.evaluate(Arquillian.java:185)
	at org.jboss.arquillian.junit.Arquillian.multiExecute(Arquillian.java:314)
	at org.jboss.arquillian.junit.Arquillian.access$100(Arquillian.java:46)
	at org.jboss.arquillian.junit.Arquillian$3.evaluate(Arquillian.java:199)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:147)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:234)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:133)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:114)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:188)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:166)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:101)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
Caused by: java.io.IOException: Channel Channel ID d34781b2 (outbound) of Remoting connection 0ac70ac7 to /127.0.0.1:4447 has been closed
	at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:115)
	at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:107)
	at org.jboss.remoting3.spi.SpiUtils.safeHandleClose(SpiUtils.java:54)
	at org.jboss.remoting3.spi.AbstractHandleableCloseable$CloseHandlerTask.run(AbstractHandleableCloseable.java:501)
	at org.jboss.remoting3.spi.AbstractHandleableCloseable.runCloseTask(AbstractHandleableCloseable.java:406)
	at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeComplete(AbstractHandleableCloseable.java:277)
	at org.jboss.remoting3.remote.RemoteConnectionChannel.closeAction(RemoteConnectionChannel.java:515)
	at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeAsync(AbstractHandleableCloseable.java:359)
	at org.jboss.remoting3.remote.RemoteConnectionHandler.closeAllChannels(RemoteConnectionHandler.java:390)
	at org.jboss.remoting3.remote.RemoteConnectionHandler.sendCloseRequest(RemoteConnectionHandler.java:231)
	at org.jboss.remoting3.remote.RemoteConnectionHandler.closeAction(RemoteConnectionHandler.java:376)
	at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeAsync(AbstractHandleableCloseable.java:359)
	at org.jboss.remoting3.ConnectionImpl.closeAction(ConnectionImpl.java:52)
	at org.jboss.remoting3.spi.AbstractHandleableCloseable.close(AbstractHandleableCloseable.java:153)
	at org.jboss.ejb.client.remoting.ConnectionPool.safeClose(ConnectionPool.java:165)
	at org.jboss.ejb.client.remoting.ConnectionPool.release(ConnectionPool.java:99)
	at org.jboss.ejb.client.remoting.ConnectionPool$PooledConnection.close(ConnectionPool.java:186)
	at org.jboss.ejb.client.remoting.RemotingConnectionManager.safeClose(RemotingConnectionManager.java:65)
	at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector$ContextCloseListener.contextClosed(ConfigBasedEJBClientContextSelector.java:174)
	at org.jboss.ejb.client.EJBClientContext.close(EJBClientContext.java:1047)
	at org.jboss.ejb.client.EJBClientContext.finalize(EJBClientContext.java:1062)
	at java.lang.J9VMInternals.runFinalize(J9VMInternals.java:412)

The error is similar to that in https://bugzilla.redhat.com/show_bug.cgi?id=928223

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-JGroups/job/eap-6x-jgroups-tcpgossip-rhel-matrix/43/jdk=ibm16,label=RHEL6_x86_64/testReport/org.jboss.as.test.clustering.cluster.ejb3.stateful.remote.failover/RemoteEJBClientStatefulBeanFailoverTestCase%28SYNC-tcp%29/testFailoverFromRemoteClientWhenOneNodeGoesDown/

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-JGroups/job/eap-6x-jgroups-tcpgossip-rhel-matrix/43/jdk=ibm16,label=RHEL6_x86_64/testReport/org.jboss.as.test.clustering.cluster.ejb3.stateful.remote.failover/RemoteEJBClientStatefulBeanFailoverTestCase%28SYNC-tcp%29/testFailoverFromRemoteClientWhenOneNodeUndeploys/

Comment 1 Radoslav Husar 2013-07-19 13:39:51 UTC

Is this still an issue? I haven't seen this test failing on the ligtning PR runs for a while.

Comment 3 Richard Janík 2013-10-01 07:02:19 UTC

Not seen in 6.2.0.ER3 nor 6.2.0.ER1.

Comment 4 Richard Achmatowicz 2014-04-09 14:26:13 UTC

This test case was @Ignore'd in the EAP 6.3 sources, and so gets skipped.
I'm going to do some testing on the original problem platform mentioned above to see if I can reproduce and then investigate further.