Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1057207

Summary: ironjacamar transient bad deployment
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Aleksandar Kostadinov <akostadi>
Component: JCAAssignee: Martin Simka <msimka>
Status: CLOSED CURRENTRELEASE QA Contact: Martin Simka <msimka>
Severity: high Docs Contact: Russell Dickenson <rdickens>
Priority: unspecified    
Version: 6.2.0CC: pjelinek, pkremens
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1090371 (view as bug list) Environment:
RHEL
Last Closed: 2014-09-03 11:28:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1090371    
Attachments:
Description Flags
junit test error report
none
debug log from server when error occurred
none
debug log from server from a passed test run
none
arquillian-service.jar as deployed by the test suite
none
ij.rar as deployed by the test siute none

Description Aleksandar Kostadinov 2014-01-23 15:54:41 UTC
There is a strange deployment issue happening sometimes running EAP test suite. For example with IronJacamarDeploymentTestCase. Basically arquillian fails on deploying the test rar when it tries to read the resource through management API (attaching surefire report file).
What I could find out is that sometimes the server reports in the log that the deployment is successfully performed but listing deployments through cli or management API does not show them. Not sure it is a problem with ironjacamar, but we saw such issue only with it.

To understand what's going on, I modified the deploy() method from CommonDeployableContainer to block execution when such situation occurs(i.e. when getProtocolMetaData() throws). Connecting to server with cli in this state also does not list the ij.rar deployment. That rules out the possibility that it just needs some more time. Funnily the last line of server log is:
>  10:51:18,156 INFO  [org.jboss.as.server] (management-handler-thread - 4) JBAS018559: Deployed "ij.rar" (runtime-name : "ij.rar")

and in debug mode, the next line caused by the attempt to read the management resource is:
> 10:51:18,160 DEBUG [org.jboss.as.controller.management-operation] (management-handler-thread - 1) JBAS014616: Operation ("read-resource") failed - address: ([("deployment" => "ij.rar")]) - failure description: "JBAS014807: Management resource '[(\"deployment\" => \"ij.rar\")]' not found"

The issue can be reproduced if you run the test case 2 or 3 hundred times. I got it around 1% of the runs on a descent multicore physical machine if that matters. For convenience pasting a sample such loop here:
> for i in `seq 101 400`; do
>     ./integration-tests.sh -Dts.basic -Dpublic-repos -Dmaven.test.failure.ignore=true -fae -Dsurefire.forked.process.timeout=3600000 -Dmcast=230.5.6.7 -Dtest=IronJacamarDeploymentTestCase |& tee run.$i.log;
>     mv testsuite/integration/basic/target testsuite/integration/basic/target.$i;
> done

It may look as a small percentage but in QE we are running the TS hundreds of times and in a matrix job with 12 variants, it seems always a few of them get it wrong. So reproducibility depends on environment.

P.S. I could not reproduce the issue outside the EAP test suite deploying and undeploying the archives as used in the test suite thousands of times. So this issue looks like possibly related: https://issues.jboss.org/browse/WFLY-1701

Comment 1 Aleksandar Kostadinov 2014-01-23 16:00:20 UTC
Created attachment 854460 [details]
junit test error report

Comment 2 Aleksandar Kostadinov 2014-01-23 16:13:40 UTC
Created attachment 854466 [details]
debug log from server when error occurred

Comment 3 Aleksandar Kostadinov 2014-01-23 16:19:08 UTC
Created attachment 854467 [details]
debug log from server from a passed test run

Notice that the log from the good run is shorter because on the bad run, test execution was stopped for the night and server logged a lot of debug messages until I stopped it the next day.

Comment 4 Aleksandar Kostadinov 2014-01-23 16:24:13 UTC
Created attachment 854469 [details]
arquillian-service.jar as deployed by the test suite

Comment 5 Aleksandar Kostadinov 2014-01-23 16:25:18 UTC
Created attachment 854470 [details]
ij.rar as deployed by the test siute

attaching arquillian-service.jar and ij.rar as deployed by the test suite if that can help

Comment 6 Jesper Pedersen 2014-09-02 19:56:53 UTC
Martin, feel free to look at this - or close.

Comment 7 Martin Simka 2014-09-03 11:28:14 UTC
Not seen in last test cycles, most likely resolved in bug 984855