Bug 1090371

+++ This bug was initially created as a clone of Bug #1057207 +++

There is a strange deployment issue happening sometimes running EAP test suite. For example with IronJacamarDeploymentTestCase. Basically arquillian fails on deploying the test rar when it tries to read the resource through management API (attaching surefire report file).
What I could find out is that sometimes the server reports in the log that the deployment is successfully performed but listing deployments through cli or management API does not show them. Not sure it is a problem with ironjacamar, but we saw such issue only with it.

To understand what's going on, I modified the deploy() method from CommonDeployableContainer to block execution when such situation occurs(i.e. when getProtocolMetaData() throws). Connecting to server with cli in this state also does not list the ij.rar deployment. That rules out the possibility that it just needs some more time. Funnily the last line of server log is:
>  10:51:18,156 INFO  [org.jboss.as.server] (management-handler-thread - 4) JBAS018559: Deployed "ij.rar" (runtime-name : "ij.rar")

and in debug mode, the next line caused by the attempt to read the management resource is:
> 10:51:18,160 DEBUG [org.jboss.as.controller.management-operation] (management-handler-thread - 1) JBAS014616: Operation ("read-resource") failed - address: ([("deployment" => "ij.rar")]) - failure description: "JBAS014807: Management resource '[(\"deployment\" => \"ij.rar\")]' not found"

The issue can be reproduced if you run the test case 2 or 3 hundred times. I got it around 1% of the runs on a descent multicore physical machine if that matters. For convenience pasting a sample such loop here:
> for i in `seq 101 400`; do
>     ./integration-tests.sh -Dts.basic -Dpublic-repos -Dmaven.test.failure.ignore=true -fae -Dsurefire.forked.process.timeout=3600000 -Dmcast=230.5.6.7 -Dtest=IronJacamarDeploymentTestCase |& tee run.$i.log;
>     mv testsuite/integration/basic/target testsuite/integration/basic/target.$i;
> done

It may look as a small percentage but in QE we are running the TS hundreds of times and in a matrix job with 12 variants, it seems always a few of them get it wrong. So reproducibility depends on environment.

P.S. I could not reproduce the issue outside the EAP test suite deploying and undeploying the archives as used in the test suite thousands of times. So this issue looks like possibly related: https://issues.jboss.org/browse/WFLY-1701

--- Additional comment from Aleksandar Kostadinov on 2014-01-23 11:00:20 EST ---



--- Additional comment from Aleksandar Kostadinov on 2014-01-23 11:13:40 EST ---



--- Additional comment from Aleksandar Kostadinov on 2014-01-23 11:19:08 EST ---

Notice that the log from the good run is shorter because on the bad run, test execution was stopped for the night and server logged a lot of debug messages until I stopped it the next day.

--- Additional comment from Aleksandar Kostadinov on 2014-01-23 11:24:13 EST ---



--- Additional comment from Aleksandar Kostadinov on 2014-01-23 11:25:18 EST ---

attaching arquillian-service.jar and ij.rar as deployed by the test suite if that can help