Bug 1397494 - (6.4.z) overlay tests fail elusively with EJBCLIENT000025: No EJB receiver available exception
Summary: (6.4.z) overlay tests fail elusively with EJBCLIENT000025: No EJB receiver av...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: EJB
Version: 6.4.19
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: CR1
: EAP 6.4.13
Assignee: Fedor Gavrilov
QA Contact: Jan Martiska
URL:
Whiteboard:
Depends On:
Blocks: 1404342 eap6413-payload 1389428 1403950
TreeView+ depends on / blocked
 
Reported: 2016-11-22 16:08 UTC by Fedor Gavrilov
Modified: 2017-02-03 16:44 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-02-03 16:44:52 UTC
Type: Bug


Attachments (Terms of Use)

Description Fedor Gavrilov 2016-11-22 16:08:02 UTC
Description of problem:
Overlay tests, such as org.jboss.as.test.integration.deployment.deploymentoverlay.ear.OverlayExistingResourceTestCase or org.jboss.as.test.integration.deployment.deploymentoverlay.jar.OverlayNonExistingResourceTestCase fail from time to time with exception like this:
java.lang.IllegalStateException: EJBCLIENT000025: No EJB receiver available for handling [appName:, moduleName:overlayed, distinctName:] combination for invocation context org.jboss.ejb.client.EJBClientInvocationContext@169076
    at org.jboss.ejb.client.EJBClientContext.requireEJBReceiver(EJBClientContext.java:747)
    at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:116)
    at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)
    at org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:255)
    at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:200)
    at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:183)
    at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)
    at com.sun.proxy.$Proxy51.fetchResource(Unknown Source)

Overlay tests seem to do lots of deploy/undeploy operations, which result in DeploymentRepository.add, DeploymentRepository.startDeployment and DeploymentRepository.remove calls. Exception like this might occur when remove call from the previous undeploy operation is being executed after add call of the following deploy operation.
 

How reproducible/Steps to Reproduce:
Can be reproduced locally by setting breakpoints in DeploymentRepository methods, executing ./integration-tests.sh clean install -Dtest=org.jboss.as.test.integration.deployment.deploymentoverlay.*.*TestCase.java -Dintegration.module -Dts.basic -DtestLogToFile -Ddebug
 and releasing execution shortly after it hits breakpoint, but unreliably. Also exceptions could be noticed in CI environment, supposedly when it is under the load. First noticed in https://github.com/jbossas/jboss-eap/pull/2878 test results, but PR itself does not seem to be the cause.

Actual results: no test failures


Expected results: failure around OverlayUtils.setupOverlay call or similar


Additional info:

Comment 1 Fedor Gavrilov 2016-11-23 11:22:41 UTC
UPD: surely,

Expected results: no test failures

Actual results: failure around OverlayUtils.setupOverlay call or similar

Comment 2 Enrique Gonzalez Martinez 2016-11-23 12:06:28 UTC
this looks like a race condition when the deployment happens before the ejb client tries to access the app... the tests need to ensure that deployment has been performed.

I'm not sure if there is a reliable way to know when a deployment has been performed. I guess something like ServerReload class should be tried here to fix a problem like this (not sure if it is backported in 6.4.x branch)

Comment 3 Enrique Gonzalez Martinez 2016-11-23 12:07:25 UTC
rectify my last comment.... it is the other way round... the ejb client tries to connect before the deployment

Comment 4 Fedor Gavrilov 2016-11-24 17:15:08 UTC
Enrique,
yes, I can imagine this being the cause. Remote client calls previously waited until deployment becomes available, but later the behaviour was changed to make them fail fast. If there is no happens-before relationship between deployment being done and client sending requests, the result might look like this.

Comment 11 Jiří Bílek 2017-01-02 13:22:45 UTC
Both tests is not fail on any platforms.

Verified with EAP 6.4.13.CP.CR1

Comment 13 Petr Penicka 2017-02-03 16:44:52 UTC
Released with EAP 6.4.13 on Feb 02 2017.


Note You need to log in before you can comment on or make changes to this bug.