Created attachment 829633 [details] Server.log running with standalone-ha During running JBoss AS testsuite on testing EAP 6.2.0.CR3 we hit an error [1] in the log. As the periodic recovery runs asynchronously this error is intermittent and is hard to reproduce it. But I've checked the code of the EJBTransactionRecoveryService which looks in this way: final XAResource[] xaResources = new XAResource[receiverContexts.size()]; synchronized (receiverContexts) { for (int i = 0; i < receiverContexts.size(); i++) { xaResources[i] = EJBClientManagedTransactionContext.getEJBXAResourceForRecovery(receiverContexts.get(i), arjunaTxCoreEnvironmentBean.getValue().getNodeIdentifier()); } } and at the first look I wonder whether the problem couldn't be that the xaResources array is not created under synchronized block? I'm adding attachement of the testsuite run. [1] [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016009: Caught:: java.lang.ArrayIndexOutOfBoundsException: 0 at org.jboss.as.ejb3.remote.EJBTransactionRecoveryService.getXAResources(EJBTransactionRecoveryService.java:112) at com.arjuna.ats.internal.jbossatx.jta.XAResourceRecoveryHelperWrapper.getXAResources(XAResourceRecoveryHelperWrapper.java:51) [jbossjts-integration.jar:4.17.15.Final-redhat-4] at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.resourceInitiatedRecoveryForRecoveryHelpers(XARecoveryModule.java:516) [jbossjts-jacorb.jar:4.17.15.Final-redhat-4] at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkFirstPass(XARecoveryModule.java:182) [jbossjts-jacorb.jar:4.17.15.Final-redhat-4] at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:743) [jbossjts-jacorb.jar:4.17.15.Final-redhat-4] at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:371) [jbossjts-jacorb.jar:4.17.15.Final-redhat-4]
I agree to Ondrej. I think the line 109 "final XAResource[] xaResources = new XAResource[receiverContexts.size()];" should be placed inside synchronized block: ~~~ 107 @Override 108 public XAResource[] getXAResources() { 109 final XAResource[] xaResources = new XAResource[receiverContexts.size()]; 110 synchronized (receiverContexts) { 111 for (int i = 0; i < receiverContexts.size(); i++) { 112 xaResources[i] = EJBClientManagedTransactionContext.getEJBXAResourceForRecovery(receiverContexts.get(i), arjunaTxCoreEnvironmentBean.getValue().getNodeIdentifier()); 113 } 114 } 115 return xaResources; 116 } ~~~ I guess this "java.lang.ArrayIndexOutOfBoundsException: 0" will happen in the following scenario: 1. receiverContexts.size() = 0 at the line 109 xaResources is created with new XAResource[0] 2. receiverRegistered method is called from other thread between the line 109 and 110 Now receiverContexts.size() = 1 3. As receiverContexts.size() = 1, for-loop is executed Then the line 112 "xaResources[0] = ..." will get ArrayIndexOutOfBoundsException because xaResources is empty array
Noticed in EAP 6.3.0 ER1 too. David, could you please take a look at description and comment 2 about line 109? If it is the case, the fix will be easy.
Verified for EAP 6.3.0.ER3. Run like: export JAVA_OPTS="-Djboss.modules.system.pkgs=org.jboss.byteman -Dorg.jboss.byteman.transform.all -javaagent:${BYTEMAN_HOME}/lib/byteman.jar=script:/path/to/bz1035216-byteman-script.btm,boot:${BYTEMAN_HOME}/lib/byteman.jar,listener:false" ./bin/standalone.sh -c standalone-ha.xml For ER1 the error occurs after few server restarts (issue is reproducible with byteman script and the server which is restarted several times). For ER3 the failure was not spotted.