Bug 1292367 - [GSS](6.4.z) Race condition during server start-up - RA not deployed
[GSS](6.4.z) Race condition during server start-up - RA not deployed
Status: CLOSED CURRENTRELEASE
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: JCA (Show other bugs)
6.4.4
Unspecified Unspecified
unspecified Severity unspecified
: CR1
: EAP 6.4.8
Assigned To: Lin Gao
Martin Simka
:
Depends On: 984855
Blocks: 1326951 1047961 eap648-payload 1322600
  Show dependency treegraph
 
Reported: 2015-12-17 03:30 EST by Hisanobu Okuda
Modified: 2017-01-17 07:38 EST (History)
18 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 984855
: 1326951 (view as bug list)
Environment:
Last Closed: 2017-01-17 07:38:37 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
onagano: needinfo+


Attachments (Terms of Use)
standalone.xml (65.06 KB, application/xml)
2015-12-17 04:03 EST, Hisanobu Okuda
no flags Details
ojdbc6_g.jar (3.69 MB, application/zip)
2015-12-17 04:05 EST, Hisanobu Okuda
no flags Details
startup_shutdown_100times (298 bytes, text/plain)
2015-12-21 22:18 EST, Hisanobu Okuda
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker JBEAP-2899 Blocker Closed Race condition during server start-up - config property of ConnectionDefinition is not injected 2018-03-30 00:22 EDT
JBoss Issue Tracker WFLY-1701 Blocker Closed Race condition during server start-up - RA not deployed 2018-03-30 00:22 EDT
JBoss Issue Tracker WFLY-6035 Critical Closed Race condition during server start-up - config property of ConnectionDefinition is not injected 2018-03-30 00:22 EDT

  None (edit)
Description Hisanobu Okuda 2015-12-17 03:30:13 EST
+++ This bug was initially created as a clone of Bug #984855 +++

Description of problem:
EAP 6.4.4 rarely fails to deploy the teiid RA and raises an exception:-

13:15:31,324 ERROR [org.jboss.msc.service.fail] (ResourceAdapterDeploymentService Thread Pool -- 1) MSC000001: Failed to start service jboss.ra.deployment."org.jboss.teiid.resource-adapter.file:main_fileQS39": org.jboss.msc.service.StartException in service jboss.ra.deplo
yment."org.jboss.teiid.resource-adapter.file:main_fileQS39": JBAS010446: Failed to start RA deployment [fileQS39]
        at org.jboss.as.connector.services.resourceadapters.deployment.AbstractResourceAdapterDeploymentService$1.run(AbstractResourceAdapterDeploymentService.java:279) [jboss-as-connector-7.5.4.Final-redhat-4.jar:7.5.4.Final-redhat-4]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
        at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.2.Final-redhat-1.jar:2.1.2.Final-redhat-1]
Caused by: org.jboss.jca.deployers.common.DeployException: IJ020056: Deployment failed: file:/home/opt/dv62x/modules/system/layers/dv/org/jboss/teiid/resource-adapter/file/main/
        at org.jboss.jca.deployers.common.AbstractResourceAdapterDeployer.createObjectsAndInjectValue(AbstractResourceAdapterDeployer.java:2518) [ironjacamar-deployers-common-1.0.32.Final-redhat-1.jar:1.0.32.Final-redhat-1]
        at org.jboss.as.connector.services.resourceadapters.deployment.ResourceAdapterXmlDeploymentService$AS7RaXmlDeployer.doDeploy(ResourceAdapterXmlDeploymentService.java:185) [jboss-as-connector-7.5.4.Final-redhat-4.jar:7.5.4.Final-redhat-4]
        at org.jboss.as.connector.services.resourceadapters.deployment.ResourceAdapterXmlDeploymentService.start(ResourceAdapterXmlDeploymentService.java:106) [jboss-as-connector-7.5.4.Final-redhat-4.jar:7.5.4.Final-redhat-4]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
Caused by: javax.resource.spi.InvalidPropertyException: ParentDirectory is not set
        at org.teiid.resource.adapter.file.FileManagedConnectionFactory.createConnectionFactory(FileManagedConnectionFactory.java:47)
        at org.teiid.resource.spi.BasicManagedConnectionFactory.createConnectionFactory(BasicManagedConnectionFactory.java:56)
        at org.jboss.jca.deployers.common.AbstractResourceAdapterDeployer.createObjectsAndInjectValue(AbstractResourceAdapterDeployer.java:2253) [ironjacamar-deployers-common-1.0.32.Final-redhat-1.jar:1.0.32.Final-redhat-1]
        ... 7 more



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. install EAP 6.4.4
2. install JDV 6.2.0 on the top of EAP 6.4.4
3. copy the attached standalone.xml
4. repeat start/stop server
5. You will see the exception a few times in every 100 startup.

Actual results:


Expected results:


Additional info:

I don't observe the issue if I remove the 4 oracle datasources in the standalone.xml.
Comment 1 Hisanobu Okuda 2015-12-17 04:03 EST
Created attachment 1106635 [details]
standalone.xml
Comment 2 Hisanobu Okuda 2015-12-17 04:05:08 EST
ERRATA

Steps to Reproduce:
1. install EAP 6.4.4
2. install JDV 6.2.0 on the top of EAP 6.4.4
3. copy the attached standalone.xml
4. copy the attached ojdbc6_g.jar
5. repeat start/stop server
6. You will see the exception a few times in every 100 startup.
Comment 3 Hisanobu Okuda 2015-12-17 04:05 EST
Created attachment 1106636 [details]
ojdbc6_g.jar
Comment 4 Hisanobu Okuda 2015-12-21 22:18 EST
Created attachment 1108554 [details]
startup_shutdown_100times
Comment 5 Enrique Gonzalez Martinez 2016-01-05 09:51:04 EST
Taking this. It does not seem connected to WFLY-1701 as the fix is already in 6.4.x branch. 

https://github.com/jbossas/jboss-eap/commit/8145eb54234120ecc5d2ff55f4720984bb125443
Comment 7 Enrique Gonzalez Martinez 2016-01-08 10:38:12 EST
This is a race condition. During the deployment there are a few services involved in the boot up:

a) 
  1) ConnectionDefinitionService
  2) CDConfigPropertiesService

b) AS7RaXmlDeployer (doDeploy)

There are two different race conditions (not just the one found in this BZ) but they are related.

1) if a) is executed after b), the RA deployer won't create the factory. (the ModifiableResourceAdapter has not been set at that point because the ConnectionDefinitionService has no been started yet; therefore the AS7RaXMLDeployer cannot deploy the connection definition during its execution).

2) if it is executed in the right order but a.2 (set the connector property) is executed after b then you get the stacktrace in the description. The problem is that the ModifiableConnDef has not registered the property by the CDConfigPropertiesService; It means that As7RaXmlDeployer cannot set the property before creating the factory during Ra deployment (org.teiid.resource.adapter.file.FileManagedConnectionFactory).
Comment 11 Osamu Nagano 2016-01-26 20:21:07 EST
@lgao, Right, my mistake.  Fixed these fields.  Thanks!
Comment 13 JBoss JIRA Server 2016-03-06 22:33:42 EST
Lin Gao <lgao@redhat.com> updated the status of jira JBEAP-2899 to Resolved
Comment 15 Mike McCune 2016-03-28 18:55:34 EDT
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions
Comment 18 Jiří Bílek 2016-05-10 02:07:06 EDT
Verified with EAP 6.4.8.CP.CR2
Comment 19 JBoss JIRA Server 2016-06-14 07:38:14 EDT
Jiri Pallich <jpallich@redhat.com> updated the status of jira JBEAP-2899 to Closed
Comment 21 Petr Penicka 2017-01-17 07:38:37 EST
Retroactively bulk-closing issues from released EAP 6.4 cumulative patches.

Note You need to log in before you can comment on or make changes to this bug.