Bug 1081144

Summary: [RFE] EAP6-17 XAException.XAER_PROTO when CMR datasource transaction fails during commit phase
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Ondrej Chaloupka <ochaloup>
Component: Transaction ManagerAssignee: Michael <mmusgrov>
Status: CLOSED CURRENTRELEASE QA Contact: Ondrej Chaloupka <ochaloup>
Severity: high Docs Contact: Russell Dickenson <rdickens>
Priority: unspecified    
Version: 6.3.0CC: hhovsepy, kkhan, mmusgrov, smumford
Target Milestone: ER2Flags: smumford: needinfo-
Target Release: EAP 6.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-28 15:26:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1085877    
Bug Blocks: 1051640    
Attachments:
Description Flags
server.log none

Description Ondrej Chaloupka 2014-03-26 16:18:54 UTC
Created attachment 879117 [details]
server.log

It seems that there is some troubles in finishing transaction on CMR connection when RuntimeException is thrown during CMR commit.

In such case the second access to the connection causes XAException.XAER_PROTO being thrown.

The log contains error like this:
WARN  [com.arjuna.ats.jta] (EJB default - 8) ARJUNA016061: TransactionImple.enlistResource - XAResource.start returned: XAException.XAER_PROTO for < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff7f000001:-863b1bb:5332e12f:4a, node_name=1, branch_uid=0:ffff7f000001:-863b1bb:5332e12f:4e, subordinatenodename=null, eis_name=java:jboss/xa-datasources/CrashRecoveryDS >: org.jboss.jca.core.spi.transaction.local.LocalXAException: IJ001151: Trying to start a new transaction when old is not complete: Old: < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff7f000001:-863b1bb:5332e12f:3c, node_name=1, branch_uid=0:ffff7f000001:-863b1bb:5332e12f:41, subordinatenodename=null, eis_name=java:jboss/xa-datasources/CrashRecoveryDS >, New < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff7f000001:-863b1bb:5332e12f:4a, node_name=1, branch_uid=0:ffff7f000001:-863b1bb:5332e12f:4e, subordinatenodename=null, eis_name=java:jboss/xa-datasources/CrashRecoveryDS >, Flags 0

The test flow is following:
1) prepare jdbc cmr resource
2) prepare test xa resource
3) commit jdbc cmr resource - RuntimeException is thrown
4) commit test xa resource
5) checking values in the database table - for this check is used CMR datasource - at this point the start of the transaction fails with the exception mentioned above

Comment 1 Ondrej Chaloupka 2014-03-26 16:21:08 UTC
Created attachment 879118 [details]
ds.properties file for oracle

This could be reproduced by test case of crash recovery testsuite

git clone -b lrco git://git.app.eng.bos.redhat.com/jbossqe/eap-tests-transactions.git
cd eap-tests-transactions/jbossts
export JBOSS_HOME=path/to/jboss-eap
mvn clean verify -Dtest=JPAInjectedFailureCMRTestCase#injectRuntimeExceptionOnCommit -Dno.cleanup.at.teardown -Djbossts.noJTS -Dds.properties=path/to/ds.properties

Jboss eap 6.3.0.DR5 could be downloaded from: 
http://download.devel.redhat.com/devel/candidates/JBEAP/JBEAP-6.3.0.DR5/jboss-eap-6.3.0.DR5.zip

Comment 3 Ondrej Chaloupka 2014-03-28 07:39:19 UTC
This is connected with upstream jira:
https://issues.jboss.org/browse/JBTM-2132

In current state there is not needed to throw RuntimeException (which is maybe incorrect test scenario) but when LocalXAException (excepted exception which RM could throw) is thrown then the result with not finished transaction on connection is the same. Which means that the second call using the same connection is failing because of not finished txn.

Comment 6 Ondrej Chaloupka 2014-04-03 16:49:32 UTC
Created attachment 882377 [details]
ds.properties file for oracle

Comment 10 Kabir Khan 2014-04-09 22:29:40 UTC
Reading comment #9, I am setting this to MODIFIED in line with the component upgrade

Comment 11 Ondrej Chaloupka 2014-04-15 08:02:32 UTC
I've done checks with Narayana 4.17.19.Final (expected to be in ER2) and I agree with Mike that this is probably not an issue which could be found in real application.

What I think is:
The only possible "real" exception seems to be the ResourceException when commit fails on javax.resource.spi.LocalTransaction.
The JCA layer expects that local transaction is not part of the any global one. The catch block catches the the fail of resource and returns connection to pool. When the catch block is not reached then the second usage of the connection causes the exception being thrown. So there could be any exception being thrown - runtime or not runtime in any part of the LocalXAResourceImpl class and it would cause that the connection won't be usable next time. But as I tried to explain - my point of view is that it seems that this will not happen.

The code of the LocalXAResourceImpl is here:
https://github.com/ironjacamar/ironjacamar/blob/e45ca1459f4b960e81ab6cfa6322cb4464a5cf68/core/src/main/java/org/jboss/jca/core/tx/jbossts/LocalXAResourceImpl.java#L163

When I will get the EAP 6.3.0.ER2 I will check the change of the narayana version and run our tests against it. If there is no other issue found then this issue is fixed from my point of view.

Comment 12 Ondrej Chaloupka 2014-04-23 13:44:27 UTC
As mentioned in comment above I consider this as fixed issue.
Verified for EAP 6.3.0.ER2.

Comment 13 Scott Mumford 2014-05-14 00:18:11 UTC
Micheal, does this issue require a release note in the 6.3.0 Beta documentation? 

Ondrej's comment 11 suggests this wasn't an issue that would impact a customer.

If this does need to be documented in the release notes, could you please provide a draft note in the Doc Text field above?

Comment 14 Scott Mumford 2014-05-14 00:32:46 UTC
Marking as not requiring a release note as per doc lead instruction.

If this is incorrect, please supply a draft release note in the Doc Text field and set the requires_doc_text flag to '?'

Cancelling NEEDINFO request.