Created attachment 889778 [details] server.log Description of problem: When datasource is configured to use security domain with Kerberos authetication and application (ejb) tries to call em.persist(entity), exception is raised. Server tries to enlist / obtain connections when the transaction is in rollback mode (ABORT_ONLY). See attached server.log for full stacktrace. When I change datasource to use user-name/password, it works. Adjusting of pool size have no effect. 15:33:11,543 TRACE [org.jboss.jca.core.connectionmanager.listener.TxConnectionListener] (http-/127.0.0.1:8080-4) Failed to enlist resource org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@48b75e7f[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@7e9d0924 connection handles=0 lastUse=1398432791336 trackByTx=true pool=org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubjectAndCri@682d524c pool internal context=SemaphoreArrayListManagedConnectionPool@124473c4[pool=TestDatasource] xaResource=LocalXAResourceImpl@77c35b6c[connectionListener=48b75e7f connectionManager=54a935d warned=false currentXid=null productName=Oracle productVersion=Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options jndiName=java:jboss/datasources/TestDatasource] txSync=TxSync@984042721{tx=TransactionImple < ac, BasicAction: 0:ffff7f000101:-63ea678b:535a63fd:e status: ActionStatus.ABORT_ONLY > wasTrackByTx=true enlisted=false}]: java.lang.Throwable: Unabled to enlist resource, see the previous warnings. at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener$TransactionSynchronization.<init>(TxConnectionListener.java:630) at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.enlist(TxConnectionListener.java:316) at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.managedConnectionReconnected(TxConnectionManagerImpl.java:483) at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.reconnectManagedConnection(AbstractConnectionManager.java:623) at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:491) at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) ... EJB: @PersistenceContext(unitName = "defaultPU") private EntityManager em; public void test() { TestEntity entity = new TestEntity(); entity.setValue("some value"); em.persist(entity); } Datasource: <datasource jndi-name="java:jboss/datasources/TestDatasource" pool-name="TestDatasource" enabled="true" spy="true"> <connection-url>jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=db04.mw.lab.eng.bos.redhat.com)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=qaora11.jbossqa)))</connection-url> <connection-property name="oracle.net.authentication_services"> (KERBEROS5) </connection-property> <driver>oracle11gR2.jar</driver> <pool> <min-pool-size>0</min-pool-size> <max-pool-size>1</max-pool-size> <allow-multiple-users>true</allow-multiple-users> </pool> <security> <security-domain>DatabaseUser</security-domain> </security> </datasource> *Version-Release number of selected component (if applicable): EAP 6.3.0.ER2 IronJacamar 1.0.26 *Steps to Reproduce: 1. clone datasource-kerberos test suite git://git.app.eng.bos.redhat.com/jbossqe-eap-tests-domain-mode.git 2. extract eap somewhere 3. run mvn integration-test -Djboss.dist=<path-to-eap> -Ddb.profile=oracle11gR2 -Djdbc.installation.type=deployment -Dit.test=org.jboss.qa.krbds.test.DatasourceWithAllowMultipleUsersTestCase#testDatasourceJPA 4. test fails, see server.log for manual testing you can now run eap and deploy app target/deployment/_DEFAULT__jpaTestDeployment_HostTestWebApp-jpa.war and then access http://localhost:8080/HostTestWebApp/HostTestServlet
You can't obtain connections once a transaction is in rollback mode. This needs further investigation.
this issue seems to be oracle exclusive, mentioned tests pass on mssql
The reason for roll-back only state is as follows: 15:33:11,341 WARN [com.arjuna.ats.arjuna] (http-/127.0.0.1:8080-4) ARJUNA012140: Adding multiple last resources is disallowed. Trying to add LastResourceRecord(XAOnePhaseResource(LocalXAResourceImpl@77c35b6c[connectionListener=48b75e7f connectionManager=54a935d warned=false currentXid=< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff7f000101:-63ea678b:535a63fd:e, node_name=1, branch_uid=0:ffff7f000101:-63ea678b:535a63fd:17, subordinatenodename=null, eis_name=java:jboss/datasources/TestDatasource > productName=Oracle productVersion=Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options jndiName=java:jboss/datasources/TestDatasource])), but already have LastResourceRecord(XAOnePhaseResource(LocalXAResourceImpl@36b6ed73[connectionListener=44b40d42 connectionManager=54a935d warned=false currentXid=< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff7f000101:-63ea678b:535a63fd:e, node_name=1, branch_uid=0:ffff7f000101:-63ea678b:535a63fd:16, subordinatenodename=null, eis_name=java:jboss/datasources/TestDatasource > productName=Oracle productVersion=Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options jndiName=java:jboss/datasources/TestDatasource])) It doesn't correctly track connections in the pool for Oracle database. After em.persist(entity) it creates a new connection and a new LocalXAResourceImpl by PoolBySubject#getTransactionNewConnection(). But then within beforeCompletion() it creates another new connection by PoolBySubject#getTransactionNewConnection() and thus another new LocalXAResourceImpl which it tries to enlist to the non-XA tx which fails. The second time, i.e. within beforeCompletion(), when it tries to find an old connection by AbstractPool#getTransactionOldConnection(Transaction trackByTransaction, ManagedConnectionPool mcp) the mcp managed connection pool is different to the previous one and so {{ConnectionListener cl = (ConnectionListener)tsr.getResource(mcp);}} returns null instead of a connection listener which is stored in tsr at that time. I am still not sure however why the mcp differs in this case.
The issue cannot be debugged anymore due to outage of the kerberos server. Anyway, I have a suspicion that Oracle database changes the Subject somehow which leads to this issue. What happens there according to the attached server.log is that a new instance of SemaphoreArrayListManagedConnectionPool appears within beforeCompletion() which seems to be caused by a different key created by Object key = getKey(subject, cri, separateNoTx); at AbstractPool#getConnection() method. In this case the pool is a subject based one, thus the subject has to be different (as per SubjectKey code) which leads to a different key which then leads to a new managed connection pool created in AbsractPool#getManagedConnectionPool(). We need the kerberos server being up back to confirm this.
Jesper Pedersen <jpederse> updated the status of jira JBJCA-1179 to Closed
Jesper Pedersen <jpederse> updated the status of jira JBJCA-1179 to Reopened
Jesper Pedersen <jpederse> updated the status of jira JBJCA-1179 to Resolved
It has been fixed in IJ.
It will be fixed by upgrade to 1.0.28.Final.
verified on EAP 6.4.0.DR4
Kabir Khan <kabir.khan> updated the status of jira EAP6-215 to Resolved
John Doyle <jdoyle> updated the status of jira EAP6-215 to Closed