Bug 1027126 - Server failed to start with standalone-xts.xml and jdbc object store running on mysql 5.5 [NEEDINFO]
Server failed to start with standalone-xts.xml and jdbc object store running ...
Status: CLOSED CURRENTRELEASE
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Transaction Manager (Show other bugs)
6.2.0
Unspecified Unspecified
unspecified Severity medium
: ER5
: EAP 6.3.0
Assigned To: tom.jenkinson
Hayk Hovsepyan
Russell Dickenson
:
Depends On:
Blocks: 1097166 1097673
  Show dependency treegraph
 
Reported: 2013-11-06 03:34 EST by Ondrej Chaloupka
Modified: 2014-09-15 08:49 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
In previous versions of JBoss EAP 6, when the transaction manager was configured to run XTS transactions, use a JDBC object store hosted on MySQL 5.5, and the driver was put in the `deployments` directory, the server failed to start. This issue has been resolved in this release.
Story Points: ---
Clone Of:
: 1097166 (view as bug list)
Environment:
Last Closed: 2014-06-28 11:28:32 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
ochaloup: needinfo? (smaestri)


Attachments (Terms of Use)
server.log running with standalone-xts.xml (205.45 KB, text/x-log)
2013-11-06 03:34 EST, Ondrej Chaloupka
no flags Details
server.log running with standalone-xts.xml (42.79 KB, text/x-log)
2013-11-06 03:37 EST, Ondrej Chaloupka
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker JBTM-2121 Major Closed Add a jdbc object store implementation for later revisions of MySQL driver 2015-11-12 04:41 EST

  None (edit)
Description Ondrej Chaloupka 2013-11-06 03:34:31 EST
Created attachment 820248 [details]
server.log running with standalone-xts.xml

I've hit a problem in starting EAP 6.2.0.ER7 with standalone-xts.xml when configured to use jdbc object store running on MySQL 5.5 and driver is put under deployments folder.

When I tried to run the server in the same configuration against other databases there was no problem. The problematic seems to be myssql.

When driver is defined as module then the server starts fine.


The error which could be observed in the console is following (I'm adding the whole log as the attachment).
JBAS014775:    New missing/unsatisfied dependencies:
      service jbosgi.BootstrapBundles.COMPLETE (missing) dependents: [service jbosgi.internal.framework.INIT, service jbosgi.PersistentBundles.INSTALL, service jboss.osgi.as.module.registration] 
      service jbosgi.PersistentBundles.COMPLETE (missing) dependents: [service jbosgi.internal.framework.INIT] 
      service jboss.jdbc-driver.mysql55-jdbc-driver_jar (missing) dependents: [service jboss.driver-demander.java:jboss/datasources/jdbc-store, service jboss.data-source.java:jboss/datasources/jdbc-store] 
      service jboss.osgi.as.initial.deployments.COMPLETE (missing) dependents: [service jbosgi.PersistentBundles.INSTALL]

How to reproduce:
 - copy the standalone-xts.xml from $JBOSS_HOME/docs/examples/configs to standalone/configuration
 - configure the transactions to use the jdbc object store
  1) under datasources subsystem
  <datasource jta="false" jndi-name="java:jboss/datasources/jdbc-store"   pool-name="JDBCPool" enabled="true" use-java-context="true">
   <connection-url>jdbc:mysql://...</connection-url>
   <driver>mysql55-jdbc-driver.jar</driver>
   <security>
    <user-name>user</user-name>
    <password>pass</password>
   </security>
  </datasource>
  2) under transactions subsystem
  <jdbc-store datasource-jndi-name="java:jboss/datasources/jdbc-store"/>
 
 - copy the mysql jdbc driver to deployment directory
 - start server
   ./bin/standalone.sh -c standalone-xts.xml
Comment 1 Ondrej Chaloupka 2013-11-06 03:37:00 EST
Created attachment 820249 [details]
server.log running with standalone-xts.xml
Comment 3 Ondrej Chaloupka 2013-11-22 07:51:22 EST
This should be documented as a known issue.
Comment 4 Russell Dickenson 2013-11-26 22:04:41 EST
Attention; Ondrej

I have written a release notes entry for this ticket but I would appreciate your review to that I can be sure it is accurate.
Comment 5 Ondrej Chaloupka 2013-11-27 03:15:53 EST
Hi Russel,

thank you. I've added information that this occurs when TM is configured to run XTS transactions what is important as for JTA/JTS transactions the jdbc store works fine on mysql 5.5.

Ondra
Comment 6 Scott Mumford 2013-12-03 18:54:42 EST
Reflagged as Known Issue as per bug 1036648
Comment 7 Ondrej Chaloupka 2014-03-25 05:38:11 EDT
Hi Stefano,

may I ask you what is current status of this issue for EAP 6.3.0?

Thanks
Ondra
Comment 8 tom.jenkinson 2014-05-07 07:21:14 EDT
Hi Ondra,

I was not seeing this myself when I am using the 5.1.8 driver, I then tried the 5.1.30 driver and can see that it does not work with that one.

I get the same error as you:
11:48:25,730 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
JBAS014775:    New missing/unsatisfied dependencies:
      service jbosgi.BootstrapBundles.COMPLETE (missing) dependents: [service jbosgi.internal.framework.INIT, service jbosgi.PersistentBundles.INSTALL, service jboss.osgi.as.module.registration] 
      service jbosgi.PersistentBundles.COMPLETE (missing) dependents: [service jbosgi.internal.framework.INIT] 
      service jboss.jdbc-driver.mysql55-jdbc-driver_jar (missing) dependents: [service jboss.driver-demander.java:jboss/datasources/jdbc-store, service jboss.data-source.java:jboss/datasources/jdbc-store] 
      service jboss.osgi.as.initial.deployments.COMPLETE (missing) dependents: [service jbosgi.PersistentBundles.INSTALL

I don't think this is a transaction manager issue, I think it looks to be something in the way OSGi is trying to load the driver.

I then noticed that OSGi is not enabled in the full or standard profiles - I removed that service and got a little further. The server was still failing to start up but just with:

12:13:11,143 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
JBAS014775:    New missing/unsatisfied dependencies:
      service jboss.jdbc-driver.mysql55-jdbc-driver_jar (missing) dependents: [service jboss.driver-demander.java:jboss/datasources/jdbc-store, service jboss.data-source.java:jboss/datasources/jdbc-store] 

12:13:11,156 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:9990/management
12:13:11,157 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://127.0.0.1:9990
12:13:11,157 ERROR [org.jboss.as] (Controller Boot Thread) JBAS015875: JBoss EAP 6.3.0.Beta1 (AS 7.4.0.Final-redhat-SNAPSHOT) started (with errors) in 4581ms - Started 192 of 260 services (30 services failed or missing dependencies, 68 services are lazy, passive or on-demand).

So then I looked in the mysql jar and saw some "fabric" stuff that wasn't in the old driver. I therefore doctored the mysql jar to remove that and got:

/META-INF/services/java.sql.Driver
com.mysql.fabric.jdbc.FabricMySQLDriver


This removed that error message but created another:
12:15:35,755 FATAL [com.arjuna.ats.arjuna] (MSC service thread 1-7) ARJUNA012260: Received exception for com.arjuna.ats.internal.arjuna.objectstore.jdbc.accessors.DataSourceJDBCAccess:JBossTSTxTable: java.lang.ClassNotFoundException: com.arjuna.ats.internal.arjuna.objectstore.jdbc.drivers.mysql_driver from [Module "org.jboss.jts:main" from local module loader @3bfa3fa9 (finder: local module finder @2a698f02 (roots: /home/tom/projects/jbosstm/jboss-eap/build/target/jboss-as-7.4.0.Final-redhat-SNAPSHOT/modules,/home/tom/projects/jbosstm/jboss-eap/build/target/jboss-as-7.4.0.Final-redhat-SNAPSHOT/modules/system/layers/base))]
	at  <SNIP/>
	at com.arjuna.ats.internal.arjuna.objectstore.jdbc.JDBCStore.<init>(JDBCStore.java:252) [jbossjts-jacorb-4.17.19.Final-redhat-1.jar:4.17.19.Final-redhat-1]

Sure enough the class is called com.arjuna.ats.internal.arjuna.objectstore.jdbc.drivers.mysql_ab_driver in our codebase. I therefore created a ..mysql_driver.class and the server would start up!

So there are 3 things:
1. Why is OSGi enabled in the standalone-xts.xml, it doesn't seem to be used - I will ask Paul
2. Something can't parse the services.xml in mySQL when the com.mysql.fabric.jdbc.FabricMySQLDriver driver is in there - I can't help with that sorry
3. We are missing a class in Narayana to handle the latest mysql code. I will add that.

Thanks,
Tom
Comment 9 Paul Robinson 2014-05-07 07:24:26 EDT
standalone-xts.xml just extends standalone-full.xml. All we add is the XTS subsytem. Can this issue be reproduced on standalone-full.xml (without XTS)?
Comment 10 tom.jenkinson 2014-05-07 07:25:10 EDT
Here is the PR with the mysql_driver: https://github.com/jbosstm/narayana/pull/650
Comment 11 tom.jenkinson 2014-05-07 07:30:26 EDT
@paul, Its strange, I tried it with standalone-full and the resultant build doesn't have OSGi in standalone-full.xml but it is in standalone-xts.xml

[tom@dhcppc1 jboss-as-7.4.0.Final-redhat-SNAPSHOT](master) (10:55:18) $ diff standalone/configuration/standalone-full.xml docs/examples/configs/standalone-xts.xml 
6a7
>         <extension module="org.jboss.as.configadmin"/>
22a24
>         <extension module="org.jboss.as.osgi"/>
31a34
>         <extension module="org.jboss.as.xts"/>
133a137
>         <subsystem xmlns="urn:jboss:domain:configadmin:1.0"/>
345a350,373
>         <subsystem xmlns="urn:jboss:domain:osgi:1.2" activation="lazy">
>             <properties>
>                 <!-- Specifies the beginning start level of the framework -->
>                 <property name="org.osgi.framework.startlevel.beginning">1</property>
>             </properties>
>             <capabilities>
>                 <!-- modules registered with the OSGi layer on startup -->
>                 <capability name="javax.jws.api"/>
>                 <capability name="javax.persistence.api"/>
>                 <capability name="javax.servlet.api"/>
>                 <capability name="javax.transaction.api"/>
>                 <capability name="javax.ws.rs.api"/>
>                 <capability name="javax.xml.bind.api"/>
>                 <capability name="javax.xml.ws.api"/>
>                 <capability name="org.slf4j"/>
>                 <!-- bundles started in startlevel 1 -->
>                 <capability name="org.apache.felix.log" startlevel="1"/>
>                 <capability name="org.jboss.osgi.logging" startlevel="1"/>
>                 <capability name="org.apache.felix.configadmin" startlevel="1"/>
>                 <capability name="org.jboss.as.osgi.configadmin" startlevel="1"/>
>                 <capability name="org.jboss.as.osgi.http" startlevel="1"/>
>                 <capability name="org.jboss.as.osgi.jpa" startlevel="1"/>
>             </capabilities>
>         </subsystem>
404a433,435
>         <subsystem xmlns="urn:jboss:domain:xts:1.0">
>             <xts-environment url="http://${jboss.bind.address:127.0.0.1}:8080/ws-c11/ActivationService"/>
>         </subsystem>

I take it that is not intentional?
Comment 12 Paul Robinson 2014-05-07 07:38:45 EDT
Ok, it looks like the way these config files are created, has changed. 

Gytis, can you investigate how they are now generated and ensure that standalone-xts.xml is standalone-full.xml with just the XTS subsytem added.
Comment 13 Gytis Trikleris 2014-05-08 07:44:48 EDT
EAP pull request with standalone-xts.xml modification: https://github.com/jbossas/jboss-eap/pull/1312
Comment 14 tom.jenkinson 2014-05-08 10:55:37 EDT
Looks like we have the XTS configuration and internal Narayana parsing of MySQL driver urls in hand. There is still the issue that the mySQL driver will not load if its placed in JBossAS deployments folder. I think that should be a broader issue than this one. I don't think the tx team could fix it either so it will probably need a new owner when the other fixes are in the product.
Comment 15 Brian Stansberry 2014-05-12 12:39:21 EDT
Tom, can you create a separate BZ for the deployments folder issue and devel_ack this one so I can merge https://github.com/jbossas/jboss-eap/pull/1312? Thanks.
Comment 16 tom.jenkinson 2014-05-13 05:39:59 EDT
OK, I have cloned the bug and modified its description.

Thanks
Tom
Comment 17 Kabir Khan 2014-05-15 04:05:29 EDT
Fixed by upgrade https://bugzilla.redhat.com/show_bug.cgi?id=1097673
Comment 18 Hayk Hovsepyan 2014-05-29 07:46:13 EDT
Verified on revision 6.3.0 ER5
Comment 19 JBoss JIRA Server 2014-09-15 08:49:38 EDT
Tom Jenkinson <tom.jenkinson@redhat.com> updated the status of jira JBTM-2121 to Closed

Note You need to log in before you can comment on or make changes to this bug.