Bug 998449 - Incorrect return code on second rollback call from TM running with JTS for crash recovery
Incorrect return code on second rollback call from TM running with JTS for cr...
Status: NEW
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: JMS (Show other bugs)
6.1.0
Unspecified Unspecified
unspecified Severity medium
: ---
: EAP 6.4.0
Assigned To: Gary Tully
Miroslav Novak
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-08-19 06:52 EDT by Ondrej Chaloupka
Modified: 2017-10-09 20:24 EDT (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
JMSCrashRecoveryRemoteTestCase#prepareHalt jts server.log (in html format) (1.56 MB, text/html)
2014-07-07 04:36 EDT, Ondrej Chaloupka
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker ENTMQ-717 Minor Resolved Incorrect return code on second rollback call from TM running with JTS for crash recovery 2017-07-24 09:42 EDT

  None (edit)
Description Ondrej Chaloupka 2013-08-19 06:52:23 EDT
It seems that ActiveMQ returns incorrect return code after second call of the rollback of transaction manager doing crash recovery. This seems to be a similar issue as we found for PostgreSQL (https://bugzilla.redhat.com/show_bug.cgi?id=988724).

There is exception thrown on the second rollback() call on XAResource [1].

The JTS specification defines that the rollback() can be called twice on the same XAResource. The XAException seems to carry value 0.
As I referred we found out the similar issue for driver postgresql and the return value was changed to be XAException.XAER_NOTA in such case.

More details on test setup could be found here:
https://docspace.corp.redhat.com/docs/DOC-146452#crash-recovery

In case technical details could be discussed with Tom Jenkinson from Narayana team.



[1]
12:15:57,770 WARN  [com.arjuna.ats.jtax] (RequestProcessor-10) ARJUNA024023: XAResourceRecord.rollback caused an XA error: ARJUNA016099: Unknown error code:0 from resource XAResourceWrapperImpl@2874fb7a[xaResource=org.apache.activemq.ra.ActiveMQManagedConnection$1@38665ee7 pad=false overrideRmValue=null productName=ActiveMQ productVersion=5.8.0.redhat-60024 jndiName=java:/JmsXA] in transaction < 131072, 29, 36, 0000000000-1-1127001-11735-31-518217-16460005149, 292929292929292929292828156292930-8864-2-22111461375292929807829292929292929 >: javax.transaction.xa.XAException: Transaction 'XID:[131072,globalId=0000000000ffffffffffffffff7f001ffffff8b23ffffffe1ffffffcd5211fffffff02e0003331,branchId=0000000000ffffffffffffffff7f001ffffff8b23ffffffe1ffffffcd5211fffffff02e0003800000000]' has not been started.
 at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:773)
 at org.apache.activemq.TransactionContext.rollback(TransactionContext.java:526)
 at org.apache.activemq.ra.LocalAndXATransaction.rollback(LocalAndXATransaction.java:128)
 at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.rollback(XAResourceWrapperImpl.java:186)
 at com.arjuna.ats.internal.jta.resources.jts.orbspecific.XAResourceRecord.rollback(XAResourceRecord.java:369) [jbossjts-jacorb-4.17.7.Final-redhat-3.jar:4.17.7.Final-redhat-3]
 at com.arjuna.ArjunaOTS.OTSAbstractRecordPOA._invoke(OTSAbstractRecordPOA.java:213) [jbossjts-jacorb-4.17.7.Final-redhat-3.jar:4.17.7.Final-redhat-3]
 at org.jacorb.poa.RequestProcessor.invokeOperation(RequestProcessor.java:306) [jacorb-2.3.2-redhat-4.jar:2.3.2-redhat-4]
 at org.jacorb.poa.RequestProcessor.process(RequestProcessor.java:626) [jacorb-2.3.2-redhat-4.jar:2.3.2-redhat-4]
 at org.jacorb.poa.RequestProcessor.run(RequestProcessor.java:769) [jacorb-2.3.2-redhat-4.jar:2.3.2-redhat-4]
Caused by: javax.transaction.xa.XAException: Transaction 'XID:[131072,globalId=0000000000ffffffffffffffff7f001ffffff8b23ffffffe1ffffffcd5211fffffff02e0003331,branchId=0000000000ffffffffffffffff7f001ffffff8b23ffffffe1ffffffcd5211fffffff02e0003800000000]' has not been started.
 at org.apache.activemq.broker.TransactionBroker.getTransaction(TransactionBroker.java:380)
 at org.apache.activemq.broker.TransactionBroker.rollbackTransaction(TransactionBroker.java:267)
 at org.apache.activemq.broker.BrokerFilter.rollbackTransaction(BrokerFilter.java:125)
 at org.apache.activemq.broker.BrokerFilter.rollbackTransaction(BrokerFilter.java:125)
 at org.apache.activemq.broker.MutableBrokerFilter.rollbackTransaction(MutableBrokerFilter.java:131)
 at org.apache.activemq.broker.TransportConnection.processRollbackTransaction(TransportConnection.java:474)
 at org.apache.activemq.command.TransactionInfo.visit(TransactionInfo.java:104)
 at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:329)
 at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:184)
 at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
 at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
 at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:288)
 at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
 at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214)
 at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
 at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]
Comment 1 Gary Tully 2013-08-20 07:28:56 EDT
fix on apache trunk - http://svn.apache.org/r1515783

the problem relates to sparse unmarshalling of exceptions in openwire, so the broker assigned xa_error is not visible to the client and uninitialised to 0 which is invalid xa_ok.
The fix replaces xa_ok with xa_rmerr (a generic failure notification)

this fix will make 5.9 and A-MQ 6.1
Comment 2 Ondrej Chaloupka 2013-09-17 08:38:30 EDT
Hi Gary,

I've took the RAR from 
https://repository.apache.org/content/repositories/snapshots/org/apache/activemq/activemq-rar/5.9-SNAPSHOT/activemq-rar-5.9-20130905.131830-99.rar
and your fix seems to be inside.

But the error exception is still visible because of the incorrect return code.
ARJUNA024023: XAResourceRecord.rollback caused an XA error: XAException.XAER_RMERR from resource XAResourceWrapperImpl@22f01dd3[xaResource=org.apache.activemq.ra.ActiveMQManagedConnection$1@4c848346 pad=false overrideRmValue=null productName=ActiveMQ productVersion=5.9-SNAPSHOT jndiName=java:/JmsXA] in transaction < 131072, 29, 36, 0000000000-1-112700184-20-5322825675-680003749, 2929292929292929292928281562929301139-245111185104-39292929667829292929292929 >: javax.transaction.xa.XAException: Transaction 'XID:[131072,globalId=0000000000ffffffffffffffff7f00154ffffffecffffffcb1652384bffffffbc0002531,branchId=0000000000ffffffffffffffff7f00154ffffffecffffffcb1652384bffffffbc0002a00000000]' has not been started.

What I understand there should be returned return code XA_NOTA to be consistent with Narayana behavior.
Comment 3 Miroslav Novak 2014-07-04 09:04:11 EDT
Hi Ondra,

can you check this one whether it's still a problem.

Thanks,
Mirek
Comment 4 Ondrej Chaloupka 2014-07-07 04:36:04 EDT
Created attachment 915999 [details]
JMSCrashRecoveryRemoteTestCase#prepareHalt jts server.log (in html format)

Hi Mirek,

I've rerun the test and pass through the test server log and the issue is still valid. There is returned incorrect return code (XAException.XAER_RMERR instead of XAException.XAER_NOTA) when JTS periodic recovery calls rollback on the same XID for second time.

Adding the server log with Narayana TRACE as attachment.

Ondra
Comment 5 Miroslav Novak 2014-07-07 04:47:00 EDT
Thanks Ondra! I'm setting jboss-eap-6.4 ? flag and release as this should get fix to EAP 6.4 release. Creating jira ENTMQ-717 for JBoss A-MQ dev to notice this issue.
Comment 6 JBoss JIRA Server 2014-08-07 09:02:16 EDT
Gary Tully <gary.tully@gmail.com> updated the status of jira ENTMQ-717 to Resolved

Note You need to log in before you can comment on or make changes to this bug.