Attachment 915677 Details for Bug 901016 – Comment

Comment

comment_5566441.txt (text/plain), 64.13 KB, created by Ladislav Thon on 2013-03-04 13:07:49 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Ladislav Thon

Created: 2013-03-04 13:07:49 UTC

Size: 64.13 KB

patch

obsolete

>I have seen this in https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbservlet-undeploy-repl-sync/14/ too. There are some new exceptions though, so I figured I will share them.
>
>Just after perf18 rejoins the cluster and receives new view, it logs these two IllegalStateExceptions:
>
>08:54:49,137 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 70) ISPN000094: Received new cluster view: [perf21/ejb|5] [perf21/ejb, perf19/ejb, perf20/ejb, perf18/ejb]
>08:54:49,137 ERROR [org.jgroups.protocols.pbcast.NAKACK2] (OOB-9,shared=udp) failed to deliver OOB message [dst: <null>, src: perf21/ejb (3 headers), size=5804 bytes, flags=OOB|DONT_BUNDLE, transient_flags=OOB_DELIVERED]: java.lang.IllegalStateException: channel is not connected
>	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:615)
>	at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:544)
>	at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:391)
>	at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:249)
>	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:598)
>	at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
>	at org.jgroups.JChannel.up(JChannel.java:707)
>	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
>	at org.jgroups.protocols.RSVP.up(RSVP.java:188)
>	at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
>	at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
>	at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
>	at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
>	at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
>	at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:453)
>	at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:721)
>	at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:574)
>	at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
>	at org.jgroups.protocols.FD.up(FD.java:253)
>	at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
>	at org.jgroups.protocols.MERGE3.up(MERGE3.java:290)
>	at org.jgroups.protocols.Discovery.up(Discovery.java:359)
>	at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2616)
>	at org.jgroups.protocols.TP.passMessageUp(TP.java:1263)
>	at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1825)
>	at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1798)
>	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_38]
>	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_38]
>	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
>
>08:54:49,138 ERROR [org.jgroups.protocols.pbcast.NAKACK2] (OOB-15,shared=udp) failed to deliver OOB message [dst: <null>, src: perf19/ejb (3 headers), size=5804 bytes, flags=OOB|DONT_BUNDLE, transient_flags=OOB_DELIVERED]: java.lang.IllegalStateException: channel is not connected
>	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:615)
>	at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:544)
>	at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:391)
>	at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:249)
>	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:598)
>	at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
>	at org.jgroups.JChannel.up(JChannel.java:707)
>	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
>	at org.jgroups.protocols.RSVP.up(RSVP.java:188)
>	at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
>	at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
>	at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
>	at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
>	at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
>	at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:453)
>	at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:721)
>	at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:574)
>	at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
>	at org.jgroups.protocols.FD.up(FD.java:253)
>	at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
>	at org.jgroups.protocols.MERGE3.up(MERGE3.java:290)
>	at org.jgroups.protocols.Discovery.up(Discovery.java:359)
>	at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2616)
>	at org.jgroups.protocols.TP.passMessageUp(TP.java:1263)
>	at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1825)
>	at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1798)
>	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_38]
>	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_38]
>	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
>
>15 seconds after that, perf19 logs these two TimeoutExceptions (as can be seen, it already has the new view as well):
>
>08:54:49,133 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-20,shared=udp) ISPN000094: Received new cluster view: [perf21/ejb|5] [perf21/ejb, perf19/ejb, perf20/ejb, perf18/ejb]
>08:55:06,636 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (ajp-perf19/10.16.90.56:8009-109) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf18/ejb
>	at org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:110)
>	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:541)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:175)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:197)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:254)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:241)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:236)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:215)
>	at org.infinispan.interceptors.ReplicationInterceptor.broadcastPrepare(ReplicationInterceptor.java:131)
>	at org.infinispan.interceptors.ReplicationInterceptor.visitPrepareCommand(ReplicationInterceptor.java:123)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:106)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:107)
>	at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:135)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128)
>	at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:115)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitPrepareCommand(TransactionSynchronizerInterceptor.java:61)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
>	at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:189)
>	at org.infinispan.statetransfer.StateTransferInterceptor.visitPrepareCommand(StateTransferInterceptor.java:93)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
>	at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:85)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
>	at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141)
>	at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
>	at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
>	at org.infinispan.transaction.tm.DummyTransaction.notifyBeforeCompletion(DummyTransaction.java:230)
>	at org.infinispan.transaction.tm.DummyTransaction.runPrepare(DummyTransaction.java:242)
>	at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:86)
>	at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:102)
>	at org.infinispan.batch.BatchContainer.resolveTransaction(BatchContainer.java:123)
>	at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:105)
>	at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:86)
>	at org.infinispan.CacheImpl.endBatch(CacheImpl.java:661)
>	at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:80)
>	at org.jboss.as.clustering.infinispan.invoker.BatchCacheInvoker.invoke(BatchCacheInvoker.java:53)
>	at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:81)
>	at org.jboss.as.clustering.ejb3.cache.backing.infinispan.InfinispanBackingCacheEntryStore.update(InfinispanBackingCacheEntryStore.java:214)
>	at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.update(SerializationGroupMemberContainer.java:280) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.update(SerializationGroupMemberContainer.java:51) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.impl.backing.PassivatingBackingCacheImpl.release(PassivatingBackingCacheImpl.java:227) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.release(AbstractCache.java:92) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.release(AbstractCache.java:39) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.releaseInstance(StatefulSessionSynchronizationInterceptor.java:199) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.access$200(StatefulSessionSynchronizationInterceptor.java:50) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization.afterCompletion(StatefulSessionSynchronizationInterceptor.java:273) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
>	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:402)
>	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:103)
>	at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
>	at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
>	at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
>	at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.stateful.StatefulComponentIdInterceptor.processInvocation(StatefulComponentIdInterceptor.java:52) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$$$view5.getSerialAndIncrement(Unknown Source) [clusterbench-ee6-ejb.jar:]
>	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.6.0_38]
>	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_38]
>	at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_38]
>	at org.jboss.weld.util.reflection.SecureReflections$13.work(SecureReflections.java:267) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflectionAccess.run(SecureReflectionAccess.java:52) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflectionAccess.runAsInvocation(SecureReflectionAccess.java:137) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflections.invoke(SecureReflections.java:263) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:111) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:105) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$Proxy$.getSerialAndIncrement(LocalStatefulSB$Proxy$_$$_Weld$Proxy$.java) [clusterbench-ee6-ejb.jar:]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(LocalStatefulSB$Proxy$_$$_WeldClientProxy.java) [clusterbench-ee6-ejb.jar:]
>	at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:21)
>	at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
>	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
>	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
>	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
>	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
>	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
>	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.event(JBossWebContext.java:67)
>	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.invoke(JBossWebContext.java:48)
>	at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:134)
>	at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:99)
>	at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:92)
>	at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:64)
>	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
>	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145)
>	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
>	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
>	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336)
>	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488)
>	at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420)
>	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920)
>	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
>
>08:55:06,659 ERROR [org.infinispan.transaction.TransactionCoordinator] (ajp-perf19/10.16.90.56:8009-109) Error while processing prepare: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf18/ejb
>	at org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:110)
>	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:541)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:175)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:197)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:254)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:241)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:236)
>	at org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:215)
>	at org.infinispan.interceptors.ReplicationInterceptor.broadcastPrepare(ReplicationInterceptor.java:131)
>	at org.infinispan.interceptors.ReplicationInterceptor.visitPrepareCommand(ReplicationInterceptor.java:123)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:106)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:107)
>	at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:135)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128)
>	at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:115)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitPrepareCommand(TransactionSynchronizerInterceptor.java:61)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
>	at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:189)
>	at org.infinispan.statetransfer.StateTransferInterceptor.visitPrepareCommand(StateTransferInterceptor.java:93)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
>	at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
>	at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:85)
>	at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
>	at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
>	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
>	at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141)
>	at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
>	at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
>	at org.infinispan.transaction.tm.DummyTransaction.notifyBeforeCompletion(DummyTransaction.java:230)
>	at org.infinispan.transaction.tm.DummyTransaction.runPrepare(DummyTransaction.java:242)
>	at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:86)
>	at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:102)
>	at org.infinispan.batch.BatchContainer.resolveTransaction(BatchContainer.java:123)
>	at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:105)
>	at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:86)
>	at org.infinispan.CacheImpl.endBatch(CacheImpl.java:661)
>	at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:80)
>	at org.jboss.as.clustering.infinispan.invoker.BatchCacheInvoker.invoke(BatchCacheInvoker.java:53)
>	at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:81)
>	at org.jboss.as.clustering.ejb3.cache.backing.infinispan.InfinispanBackingCacheEntryStore.update(InfinispanBackingCacheEntryStore.java:214)
>	at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.update(SerializationGroupMemberContainer.java:280) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.update(SerializationGroupMemberContainer.java:51) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.impl.backing.PassivatingBackingCacheImpl.release(PassivatingBackingCacheImpl.java:227) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.release(AbstractCache.java:92) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.release(AbstractCache.java:39) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.releaseInstance(StatefulSessionSynchronizationInterceptor.java:199) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.access$200(StatefulSessionSynchronizationInterceptor.java:50) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization.afterCompletion(StatefulSessionSynchronizationInterceptor.java:273) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
>	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:402)
>	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:103)
>	at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
>	at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
>	at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
>	at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.stateful.StatefulComponentIdInterceptor.processInvocation(StatefulComponentIdInterceptor.java:52) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$$$view5.getSerialAndIncrement(Unknown Source) [clusterbench-ee6-ejb.jar:]
>	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.6.0_38]
>	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_38]
>	at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_38]
>	at org.jboss.weld.util.reflection.SecureReflections$13.work(SecureReflections.java:267) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflectionAccess.run(SecureReflectionAccess.java:52) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflectionAccess.runAsInvocation(SecureReflectionAccess.java:137) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflections.invoke(SecureReflections.java:263) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:111) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:105) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$Proxy$.getSerialAndIncrement(LocalStatefulSB$Proxy$_$$_Weld$Proxy$.java) [clusterbench-ee6-ejb.jar:]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(LocalStatefulSB$Proxy$_$$_WeldClientProxy.java) [clusterbench-ee6-ejb.jar:]
>	at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:21)
>	at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
>	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
>	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
>	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
>	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
>	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
>	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.event(JBossWebContext.java:67)
>	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.invoke(JBossWebContext.java:48)
>	at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:134)
>	at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:99)
>	at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:92)
>	at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:64)
>	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
>	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145)
>	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
>	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
>	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336)
>	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488)
>	at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420)
>	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920)
>	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
>
>Immediately after these TimeoutExceptions, perf19 also logs these two CacheExceptions:
>
>08:55:06,679 ERROR [org.infinispan.transaction.tm.DummyTransaction] (ajp-perf19/10.16.90.56:8009-109) ISPN000109: beforeCompletion() failed for SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=[MarshalledValue{instance={[-17, -88, -39, 4, 35, -77, 68, 100, -71, -40, 53, -30, -44, -31, 126, 104]}, serialized=ByteArray{size=128, array=0x0301fe03ff810200..}, cachedHashCode=467025400}@22e308ee], topologyId=9, isFromStateTransfer=false} org.infinispan.transaction.synchronization.SyncLocalTransaction@360a0} org.infinispan.transaction.synchronization.SynchronizationAdapter@360bf: org.infinispan.CacheException: Could not prepare. 
>	at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
>	at org.infinispan.transaction.tm.DummyTransaction.notifyBeforeCompletion(DummyTransaction.java:230)
>	at org.infinispan.transaction.tm.DummyTransaction.runPrepare(DummyTransaction.java:242)
>	at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:86)
>	at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:102)
>	at org.infinispan.batch.BatchContainer.resolveTransaction(BatchContainer.java:123)
>	at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:105)
>	at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:86)
>	at org.infinispan.CacheImpl.endBatch(CacheImpl.java:661)
>	at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:80)
>	at org.jboss.as.clustering.infinispan.invoker.BatchCacheInvoker.invoke(BatchCacheInvoker.java:53)
>	at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:81)
>	at org.jboss.as.clustering.ejb3.cache.backing.infinispan.InfinispanBackingCacheEntryStore.update(InfinispanBackingCacheEntryStore.java:214)
>	at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.update(SerializationGroupMemberContainer.java:280) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.update(SerializationGroupMemberContainer.java:51) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.impl.backing.PassivatingBackingCacheImpl.release(PassivatingBackingCacheImpl.java:227) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.release(AbstractCache.java:92) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.release(AbstractCache.java:39) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.releaseInstance(StatefulSessionSynchronizationInterceptor.java:199) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.access$200(StatefulSessionSynchronizationInterceptor.java:50) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization.afterCompletion(StatefulSessionSynchronizationInterceptor.java:273) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
>	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:402)
>	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:103)
>	at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
>	at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
>	at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
>	at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.stateful.StatefulComponentIdInterceptor.processInvocation(StatefulComponentIdInterceptor.java:52) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$$$view5.getSerialAndIncrement(Unknown Source) [clusterbench-ee6-ejb.jar:]
>	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.6.0_38]
>	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_38]
>	at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_38]
>	at org.jboss.weld.util.reflection.SecureReflections$13.work(SecureReflections.java:267) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflectionAccess.run(SecureReflectionAccess.java:52) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflectionAccess.runAsInvocation(SecureReflectionAccess.java:137) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflections.invoke(SecureReflections.java:263) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:111) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:105) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$Proxy$.getSerialAndIncrement(LocalStatefulSB$Proxy$_$$_Weld$Proxy$.java) [clusterbench-ee6-ejb.jar:]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(LocalStatefulSB$Proxy$_$$_WeldClientProxy.java) [clusterbench-ee6-ejb.jar:]
>	at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:21)
>	at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
>	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
>	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
>	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
>	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
>	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
>	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.event(JBossWebContext.java:67)
>	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.invoke(JBossWebContext.java:48)
>	at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:134)
>	at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:99)
>	at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:92)
>	at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:64)
>	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
>	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145)
>	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
>	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
>	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336)
>	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488)
>	at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420)
>	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920)
>	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
>Caused by: javax.transaction.xa.XAException
>	at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
>	at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
>	at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
>	... 87 more
>
>08:55:06,693 WARN  [com.arjuna.ats.jta] (ajp-perf19/10.16.90.56:8009-109) ARJUNA016029: SynchronizationImple.afterCompletion - failed for org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization@14173840 with exception: org.infinispan.CacheException: Unable to end batch
>	at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:107)
>	at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:86)
>	at org.infinispan.CacheImpl.endBatch(CacheImpl.java:661)
>	at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:80)
>	at org.jboss.as.clustering.infinispan.invoker.BatchCacheInvoker.invoke(BatchCacheInvoker.java:53)
>	at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:81)
>	at org.jboss.as.clustering.ejb3.cache.backing.infinispan.InfinispanBackingCacheEntryStore.update(InfinispanBackingCacheEntryStore.java:214)
>	at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.update(SerializationGroupMemberContainer.java:280) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.update(SerializationGroupMemberContainer.java:51) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.impl.backing.PassivatingBackingCacheImpl.release(PassivatingBackingCacheImpl.java:227) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.release(AbstractCache.java:92) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.release(AbstractCache.java:39) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.releaseInstance(StatefulSessionSynchronizationInterceptor.java:199) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.access$200(StatefulSessionSynchronizationInterceptor.java:50) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization.afterCompletion(StatefulSessionSynchronizationInterceptor.java:273) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
>	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:402)
>	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:103)
>	at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
>	at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
>	at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
>	at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ejb3.component.stateful.StatefulComponentIdInterceptor.processInvocation(StatefulComponentIdInterceptor.java:52) [jboss-as-ejb3-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final-redhat-2.jar:1.1.1.Final-redhat-2]
>	at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$$$view5.getSerialAndIncrement(Unknown Source) [clusterbench-ee6-ejb.jar:]
>	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.6.0_38]
>	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_38]
>	at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_38]
>	at org.jboss.weld.util.reflection.SecureReflections$13.work(SecureReflections.java:267) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflectionAccess.run(SecureReflectionAccess.java:52) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflectionAccess.runAsInvocation(SecureReflectionAccess.java:137) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.util.reflection.SecureReflections.invoke(SecureReflections.java:263) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:111) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:105) [weld-core-1.1.10.Final-redhat-1.jar:1.1.10.Final-redhat-1]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$Proxy$.getSerialAndIncrement(LocalStatefulSB$Proxy$_$$_Weld$Proxy$.java) [clusterbench-ee6-ejb.jar:]
>	at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(LocalStatefulSB$Proxy$_$$_WeldClientProxy.java) [clusterbench-ee6-ejb.jar:]
>	at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:21)
>	at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
>	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
>	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
>	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
>	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
>	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
>	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.event(JBossWebContext.java:67)
>	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.invoke(JBossWebContext.java:48)
>	at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:134)
>	at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:99)
>	at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:92)
>	at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:64)
>	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
>	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145)
>	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
>	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
>	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336)
>	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488)
>	at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420)
>	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920)
>	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
>Caused by: javax.transaction.RollbackException: Exception rolled back, status is: 9
>	at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:89)
>	at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:102)
>	at org.infinispan.batch.BatchContainer.resolveTransaction(BatchContainer.java:123)
>	at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:105)
>	... 81 more
>
>08:55:06,706 WARN  [com.arjuna.ats.arjuna] (ajp-perf19/10.16.90.56:8009-109) ARJUNA012127: TwoPhaseCoordinator.afterCompletion - returned failure for SynchronizationImple< 0:ffff0a105a38:-152486fc:512e0eb9:159c1, org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization@14173840 >
>
>Maybe this particular situation can be explained by some network glitch, not sure.

Actions: View

Attachments on bug 901016: 915677