Bug 1020881

Summary:	EAP stucks after :reload with transactions configured running with JDBCObject store
Product:	[JBoss] JBoss Enterprise Application Platform 6	Reporter:	Ondrej Chaloupka <ochaloup>
Component:	Transaction Manager	Assignee:	Stefano Maestri <smaestri>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Ondrej Chaloupka <ochaloup>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	6.2.0	CC:	brian.stansberry, jkudrnac, myarboro, tom.jenkinson
Target Milestone:	ER7	Keywords:	TestBlocker
Target Release:	EAP 6.2.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-12-15 16:55:33 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ondrej Chaloupka 2013-10-18 12:35:00 UTC

This bz refeference to bz
https://bugzilla.redhat.com/show_bug.cgi?id=1009931
which was filled for EAP 6.2.0.ER4.

Now in EAP 6.2.0.ER6 the server fails to reload with another reason. The fail does not happen all the times as it seems to me being a timing issue. I get two types of errors. One [1] causes erros to be put into console but server continues to respond but second one [2] causes the server being stuck (no respond from the server after reload).

I'm proposing this bug as a blocker ? as in second case the reload causes the server being stuck (no reaction) and need to be hardly restarted by hand. This could cause problems when using web console and in case of using jdbc object store under domain mode.



[1]
INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) JBAS010403: Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 11.2)
INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) JBAS010400: Bound data source [java:jboss/datasources/jdbc-store]
ERROR [stderr] (Transaction Expired Entry Monitor) Exception in thread "Transaction Expired Entry Monitor" com.arjuna.ats.arjuna.exceptions.FatalError: com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess@1fc261da : javax.naming.NameNotFoundException: Error looking up datasources/jdbc-store, service service jboss.naming.context.java.jboss.datasources.jdbc-store is not started
ERROR [stderr] (Transaction Expired Entry Monitor)   at com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess.getConnection(DataSourceJDBCAccess.java:55)
ERROR [stderr] (Transaction Expired Entry Monitor)   at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCImple_driver.allObjUids(JDBCImple_driver.java:316)
ERROR [stderr] (Transaction Expired Entry Monitor)   at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCStore.allObjUids(JDBCStore.java:179)
INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
ERROR [stderr] (Transaction Expired Entry Monitor)   at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCStore.allObjUids(JDBCStore.java:119)
ERROR [stderr] (Transaction Expired Entry Monitor)   at com.arjuna.ats.internal.arjuna.recovery.ExpiredTransactionStatusManagerScanner.scan(ExpiredTransactionStatusManagerScanner.java:81)
ERROR [stderr] (Transaction Expired Entry Monitor)   at com.arjuna.ats.internal.arjuna.recovery.ExpiredEntryMonitor.run(ExpiredEntryMonitor.java:171)
ERROR [stderr] (Transaction Expired Entry Monitor) Caused by: javax.naming.NameNotFoundException: Error looking up datasources/jdbc-store, service service jboss.naming.context.java.jboss.datasources.jdbc-store is not started
ERROR [stderr] (Transaction Expired Entry Monitor)   at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:132)
ERROR [stderr] (Transaction Expired Entry Monitor)   at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:80)
ERROR [stderr] (Transaction Expired Entry Monitor)   at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)
ERROR [stderr] (Transaction Expired Entry Monitor)   at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:174)
ERROR [stderr] (Transaction Expired Entry Monitor)   at org.jboss.as.naming.InitialContext.lookup(InitialContext.java:122)
ERROR [stderr] (Transaction Expired Entry Monitor)   at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:183)
ERROR [stderr] (Transaction Expired Entry Monitor)   at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)
ERROR [stderr] (Transaction Expired Entry Monitor)   at javax.naming.InitialContext.lookup(InitialContext.java:411)
ERROR [stderr] (Transaction Expired Entry Monitor)   at com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess.getConnection(DataSourceJDBCAccess.java:53)
ERROR [stderr] (Transaction Expired Entry Monitor)   ... 5 more
INFO  [org.jboss.as.server] (ServerService Thread Pool -- 26) JBAS018559: Deployed "oracle11gR2-jdbc-driver.jar" (runtime-name : "oracle11gR2-jdbc-driver.jar")


[2]
ERROR [stderr] (Periodic Recovery) Exception in thread "Periodic Recovery" com.arjuna.ats.arjuna.exceptions.FatalError: com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess@12f3754a : javax.naming.NameNotFoundException: Error looking up datasources/jdbc-store, service service jboss.naming.context.java.jboss.datasources.jdbc-store is not started
ERROR [stderr] (Periodic Recovery)   at com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess.getConnection(DataSourceJDBCAccess.java:55)
ERROR [stderr] (Periodic Recovery)   at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCImple_driver.currentState(JDBCImple_driver.java:225)
ERROR [stderr] (Periodic Recovery)   at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCStore.currentState(JDBCStore.java:139)
ERROR [stderr] (Periodic Recovery)   at com.arjuna.ats.internal.txoj.recovery.TORecoveryModule.periodicWorkSecondPass(TORecoveryModule.java:181)
ERROR [stderr] (Periodic Recovery)   at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:789)
ERROR [stderr] (Periodic Recovery)   at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:371)
ERROR [stderr] (Periodic Recovery) Caused by: javax.naming.NameNotFoundException: Error looking up datasources/jdbc-store, service service jboss.naming.context.java.jboss.datasources.jdbc-store is not started
ERROR [stderr] (Periodic Recovery)   at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:132)
ERROR [stderr] (Periodic Recovery)   at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:80)
ERROR [stderr] (Periodic Recovery)   at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)
ERROR [stderr] (Periodic Recovery)   at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:174)
ERROR [stderr] (Periodic Recovery)   at org.jboss.as.naming.InitialContext.lookup(InitialContext.java:122)
ERROR [stderr] (Periodic Recovery)   at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:183)
ERROR [stderr] (Periodic Recovery)   at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)
ERROR [stderr] (Periodic Recovery)   at javax.naming.InitialContext.lookup(InitialContext.java:411)
ERROR [stderr] (Periodic Recovery)   at com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess.getConnection(DataSourceJDBCAccess.java:53)
ERROR [stderr] (Periodic Recovery)   ... 5 more

Comment 1 Ondrej Chaloupka 2013-10-22 09:30:45 UTC

I found out that this happens not only with reload but as well when server is stopping. Which means that when the JDBC object store is configured and you stopped the server the server stucks and it has to be killed by 'kill -9' or similar way.

This problem seems to me quite urgent. I can see that the issue is already solved for Wildfly then please check the process of the stopping the server as well.

Comment 2 Ondrej Chaloupka 2013-10-22 10:13:49 UTC

I'm blocked by this issue in my testing I'm adding TestBlocker keyword.

As server stucks for reload and shutdown in most of the cases this issue blocks my testing of the JDBC object store.

Comment 4 Ondrej Chaloupka 2013-10-31 13:13:09 UTC

Verified 6.2.0.ER7. 
The reload works in my case.