Bug 1009931 - EAP fails to :reload with transactions configured to be run with JDBCObject store
Summary: EAP fails to :reload with transactions configured to be run with JDBCObject s...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Transaction Manager
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ER6
: EAP 6.2.0
Assignee: tom.jenkinson
QA Contact: Ondrej Chaloupka
URL:
Whiteboard:
Depends On: 1012986
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-09-19 14:00 UTC by Ondrej Chaloupka
Modified: 2017-10-10 00:20 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-12-15 16:12:19 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Oracle driver (2.61 MB, application/x-java-archive)
2013-09-19 14:02 UTC, Ondrej Chaloupka
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker JBTM-1938 0 Major Closed Caching of datasource for JDBC store incompatible with WildFly :reload operation 2018-01-25 05:14:55 UTC

Description Ondrej Chaloupka 2013-09-19 14:00:05 UTC
I have problem to reload server when I have configured transactions to use the JDBC store. 
After reload I'm getting following error message and the TM stops to work and the following transactions are rollbacked.

WARN  [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) ARJUNA012251: allObjUids caught exception: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:jboss/datasources/jdbc-store
 at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:147)
 at com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess.getConnection(DataSourceJDBCAccess.java:51) [jbossjts-jacorb-4.17.9.Final-redhat-1.jar:4.17.9.Final-redhat-1]
 at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCImple_driver.allObjUids(JDBCImple_driver.java:316) [jbossjts-jacorb-4.17.9.Final-redhat-1.jar:4.17.9.Final-redhat-1]
 at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCStore.allObjUids(JDBCStore.java:179) [jbossjts-jacorb-4.17.9.Final-redhat-1.jar:4.17.9.Final-redhat-1]
 at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCStore.allObjUids(JDBCStore.java:119) [jbossjts-jacorb-4.17.9.Final-redhat-1.jar:4.17.9.Final-redhat-1]
 at com.arjuna.ats.internal.arjuna.recovery.ExpiredTransactionStatusManagerScanner.scan(ExpiredTransactionStatusManagerScanner.java:81) [jbossjts-jacorb-4.17.9.Final-redhat-1.jar:4.17.9.Final-redhat-1]
 at com.arjuna.ats.internal.arjuna.recovery.ExpiredEntryMonitor.run(ExpiredEntryMonitor.java:171) [jbossjts-jacorb-4.17.9.Final-redhat-1.jar:4.17.9.Final-redhat-1]
Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:jboss/datasources/jdbc-store
 at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:321)
 at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:302)
 at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464)
 at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:139)
 ... 6 more

Comment 2 Ondrej Chaloupka 2013-09-19 14:02:23 UTC
Created attachment 799966 [details]
Oracle driver

Comment 5 Ondrej Chaloupka 2013-09-19 14:15:11 UTC
This could influence the customer as well. Supposing not being just testing blocker. Setting blocker to ?.

Comment 6 tom.jenkinson 2013-09-19 14:44:37 UTC
Hi Ondra,

Which version of EAP 6.2 did you try this on and where can I download it from please?

Thanks,
Tom

Comment 8 tom.jenkinson 2013-09-19 16:17:19 UTC
Hi Ondra,

Wouldn't we need:
https://bugzilla.redhat.com/show_bug.cgi?id=971358

It doesn't look like stefano has resolved that yet?

Tom

Comment 9 tom.jenkinson 2013-09-19 22:21:25 UTC
OK, so I have had a look at this. The problem is that the JDBCstore caches the lookup of the datasource before calling getconnection. Assuming there isn't a performance issue with re-resolving the JDBC store every time we call getConnection I can change getConnection to lookup the datasource each time.

I have solved this in JBTM-1938

Comment 10 Ondrej Chaloupka 2013-09-20 05:57:59 UTC
Hi Tom,

thanks for the fix.

From my point of view the bz
https://bugzilla.redhat.com/show_bug.cgi?id=971358
was fixed. But Stefano could have some more input on it. I'm not sure.

Ondra

Comment 11 tom.jenkinson 2013-09-20 14:45:20 UTC
So, having made the fix in the narayana repo,

Comment 12 tom.jenkinson 2013-09-27 10:34:42 UTC
For information: this has the same severity as 1011379, both are fixed in 4.17.11.

Comment 13 tom.jenkinson 2013-10-01 11:15:47 UTC
https://github.com/jbossas/jboss-eap/pull/478

Comment 14 JBoss JIRA Server 2013-10-01 16:07:42 UTC
Michael Musgrove <mmusgrov> updated the status of jira JBTM-1938 to Closed

Comment 18 Ondrej Chaloupka 2013-10-17 12:18:37 UTC
Verified on EAP 6.2.0.ER6. It's possible to reload server and no error messages are received on the console.

Comment 19 Ondrej Chaloupka 2013-10-17 14:03:31 UTC
Hi,

I'm sorry I've found out that my previous statement is not true. 

The problem with reloading the server set with JDBC driver still occur in some cases. But there is a new error now. I'm running on EAP 6.2.0.ER6.

From the log it seems to me as a timing issue. Time to time the data source could be bound after the transactions subsystem is started.

Console log:
15:05:13,272 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) JBAS010403: Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 11.2)
15:05:13,283 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) JBAS010400: Bound data source [java:jboss/datasources/jdbc-store]
15:05:13,284 ERROR [stderr] (Transaction Expired Entry Monitor) Exception in thread "Transaction Expired Entry Monitor" com.arjuna.ats.arjuna.exceptions.FatalError: com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess@1fc261da : javax.naming.NameNotFoundException: Error looking up datasources/jdbc-store, service service jboss.naming.context.java.jboss.datasources.jdbc-store is not started
15:05:13,285 ERROR [stderr] (Transaction Expired Entry Monitor)         at com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess.getConnection(DataSourceJDBCAccess.java:55)
15:05:13,285 ERROR [stderr] (Transaction Expired Entry Monitor)         at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCImple_driver.allObjUids(JDBCImple_driver.java:316)
15:05:13,286 ERROR [stderr] (Transaction Expired Entry Monitor)         at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCStore.allObjUids(JDBCStore.java:179)
15:05:13,286 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
15:05:13,286 ERROR [stderr] (Transaction Expired Entry Monitor)         at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCStore.allObjUids(JDBCStore.java:119)
15:05:13,287 ERROR [stderr] (Transaction Expired Entry Monitor)         at com.arjuna.ats.internal.arjuna.recovery.ExpiredTransactionStatusManagerScanner.scan(ExpiredTransactionStatusManagerScanner.java:81)
15:05:13,287 ERROR [stderr] (Transaction Expired Entry Monitor)         at com.arjuna.ats.internal.arjuna.recovery.ExpiredEntryMonitor.run(ExpiredEntryMonitor.java:171)
15:05:13,288 ERROR [stderr] (Transaction Expired Entry Monitor) Caused by: javax.naming.NameNotFoundException: Error looking up datasources/jdbc-store, service service jboss.naming.context.java.jboss.datasources.jdbc-store is not started
15:05:13,288 ERROR [stderr] (Transaction Expired Entry Monitor)         at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:132)
15:05:13,288 ERROR [stderr] (Transaction Expired Entry Monitor)         at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:80)
15:05:13,288 ERROR [stderr] (Transaction Expired Entry Monitor)         at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)
15:05:13,289 ERROR [stderr] (Transaction Expired Entry Monitor)         at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:174)
15:05:13,289 ERROR [stderr] (Transaction Expired Entry Monitor)         at org.jboss.as.naming.InitialContext.lookup(InitialContext.java:122)
15:05:13,289 ERROR [stderr] (Transaction Expired Entry Monitor)         at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:183)
15:05:13,289 ERROR [stderr] (Transaction Expired Entry Monitor)         at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)
15:05:13,289 ERROR [stderr] (Transaction Expired Entry Monitor)         at javax.naming.InitialContext.lookup(InitialContext.java:411)
15:05:13,290 ERROR [stderr] (Transaction Expired Entry Monitor)         at com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess.getConnection(DataSourceJDBCAccess.java:53)
15:05:13,290 ERROR [stderr] (Transaction Expired Entry Monitor)         ... 5 more
15:05:13,317 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 26) JBAS018559: Deployed "oracle11gR2-jdbc-driver.jar" (runtime-name : "oracle11gR2-jdbc-driver.jar")

Comment 20 tom.jenkinson 2013-10-18 09:17:11 UTC
Hi Ondra,

I think it is a different cause. The last issue was that the JDBC object store caches connections, this new issue seems to be related to there not being a dependency between the TM and JCA services (i.e. from a BZ point of view, this new issue could affect the initial load, not just a :reload)?

I will look further...

Tom

Comment 21 Ondrej Chaloupka 2013-10-18 10:35:42 UTC
Hi Tom,

I agree this is different issue which is related to module dependency management in AS rather than to transactions subsystem.
Just to clarify the problem happens to me just after the reload. I didn't experience the problem after the initial load. But it could be a coincidence.

I will report this problem as a new bz and I will verify this one.

Thanks
Ondra

Comment 23 Ondrej Chaloupka 2013-10-18 12:37:57 UTC
Verifying this issue.

I created new bugzilla as mentioned above.
https://bugzilla.redhat.com/show_bug.cgi?id=1020881


Note You need to log in before you can comment on or make changes to this bug.