It seems that there could be some problem in registration of the JMS RA for inflow transactions. Arjuna does not know eis_name to be stored in txid. This could cause problems during recovery.
When you check the trace message below you can see that the eis_name is put as uknown. Normally there could be mentioned the name of connection factory (e.g. eis_name=java:/JmsXA)
TRACE [com.arjuna.ats.jta] (default-threads - 2) XAResourceRecord.topLevelPrepare for XAResourceRecord < resource:org.apache.activemq.ra.LocalAndXATransaction@129fbf95, txid:< formatId=131077, gtrid_length=29, bqual_length=36 , tx_uid=0:ffff7f000001:67287e1c:523c1b3e:16, node_name=1, branch_uid=0:ffff7f000001:67287e1c:523c1b3e:17, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@67f06391 >
This occurs on all message services (HornetQ, WSMQ, ActiveMQ).
- let the MDB working above the XA connection factory and let it receive a message
- set the logging for com.arjuna to TRACE
- check the server log to see the topLevelPrepare trace log
I forgot to add whole stack trace for the correct log output. This trace could be seen in case that a message producer is used in the code.
[com.arjuna.ats.jta] (default-threads - 2) XAResourceRecord.topLevelPrepare for XAResourceRecord < resource:XAResourceWrapperImpl@56152711[xaResource=org.apache.activemq.ra.ActiveMQManagedConnection$1@6b7f93e pad=false overrideRmValue=null productName=ActiveMQ productVersion=5.9-SNAPSHOT jndiName=java:/JmsXA], txid:< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff7f000001:67287e1c:523c1b3e:16, node_name=1, branch_uid=0:ffff7f000001:6728 7e1c:523c1b3e:1f, subordinatenodename=null, eis_name=java:/JmsXA >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.9-SNAPSHOT, jndiName: java:/JmsXA com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@30bf69fc >
Look if javax.resource.spi.ManagedConnectionMetaData is implemented correctly
And the information fields in META-INF/ra.xml is correctly filed out
Hi Tom, Hi Clebert and Andy,
I was trying to investigate this problem that Xid does not contain eisname (is null) for inflow transactions. Normally I would expect the eisname contains jndi name of the resource.
When checking process of the inflow transaction enlistment I've got to this stackgrace (EAP 6.2.0.ER5):
XAResourceRecordWrappingPluginImpl.getEISName(XAResource) line: 69
TransactionImple.createXid(boolean, XAModifier, XAResource) line: 1497
TransactionImple.enlistResource(XAResource, Object) line: 581
TransactionImple.enlistResource(XAResource) line: 397
MessageEndpointInvocationHandler.beforeDelivery(Method) line: 110
NativeMethodAccessorImpl.invoke0(Method, Object, Object) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object) line: 57
DelegatingMethodAccessorImpl.invoke(Object, Object) line: 43
Method.invoke(Object, Object...) line: 606
MessageEndpointInvocationHandler(AbstractInvocationHandler).handle(Method, Object) line: 60
MessageEndpointInvocationHandler.doInvoke(Object, Method, Object) line: 136
MessageEndpointInvocationHandler(AbstractInvocationHandler).invoke(Object, Method, Object) line: 73
$Proxy20.beforeDelivery(Method) line: not available
HornetQMessageHandler.onMessage(ClientMessage) line: 308
ClientConsumerImpl.callOnMessage() line: 1117
ClientConsumerImpl.access$500(ClientConsumerImpl) line: 57
ClientConsumerImpl$Runner.run() line: 1252
OrderedExecutorFactory$OrderedExecutor$1.run() line: 106
ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1145
ThreadPoolExecutor$Worker.run() line: 615
Thread.run() line: 724
And it seems to me that the reason why the eisname is null is the fact that the XAResource is not XAResourceWrapper but DelegationSession. Do you think that the HornetQ should somelike pass the XAResource as a wrapper? Do you think that there could be some possibility how the inflow transaction could get info about jndi name of resource?
It should be possible, Im assuming you will want this for outgoing and recevery connections too?
getEISName is only called when the Xid is first created so I don't believe it will be required in recovery except to say that https://issues.jboss.org/browse/JBTM-860 does require it to be the same for the recovery connection too (though that Jira is not implemented yet)
the outgoing connections work fine in this point. The eisname is filled. The "issue" concerns just the inbound connections.
ok. Does it mean that you do not consider this as an issue?
Thank you for the reactions
And one more question - is this possibly problem on side of the TM or HornetQ. As you, Tom, talks about the JBTM-860 that is planned for 6.0.0.Final then this issue could block the resolving that issue. Then maybe I could close this bugzilla and create jira about it but I'm not sure what component is involved in?
actually, I dont think this would be an issue as we handle enlisting recovery resources ourselves.
As we agreed with Tom and Andy I removing the 6.2.0 flag and setting low priority. See below.
[10:38] <AndyTaylor> tomjenkinson: ping
[10:47] <tomjenkinson> AndyTaylor: hi
[10:48] <AndyTaylor> tomjenkinson: just commented on that BZ, i dont think it is an issue
[10:48] <AndyTaylor> tomjenkinson: as we register our own recovery
[10:48] <tomjenkinson> AndyTaylor: just checking the BZ
[10:49] <tomjenkinson> ochaloup: ping you may be interested in this discussion with AndyTaylor
[10:50] <tomjenkinson> AndyTaylor: I didn't raise the issue so I am not fully versed on this but as I understand it the Xids don't have the EIS name in them
[10:50] <ochaloup> tomjenkinson: yes, I'm, thanks
[10:50] <tomjenkinson> AndyTaylor: because the resource you have enlisted does not implement XAResourceWrapper or something?
[10:51] <AndyTaylor> tomjenkinson: yeah, but i dont think we need to, altho Im assuming that this is just needed for JCA recovery registration or something
[10:51] <tomjenkinson> AndyTaylor: this is a "problem" in the sense that the logs do not meaningfully contain debug data
[10:51] <AndyTaylor> tomjenkinson: well yes
[10:51] <tomjenkinson> AndyTaylor: so when the TM is logging "I can't recover Xid foo" it can't say
[10:52] <AndyTaylor> tomjenkinson: it would be helpful, but not a priorirty if it isnt actually a bug
[10:52] <tomjenkinson> "I can't recover Xid foo for HornetQ"
[10:52] <tomjenkinson> AndyTaylor: I didn't raise it :)
[10:52] <AndyTaylor> tomjenkinson: i know :)
[10:52] <tomjenkinson> AndyTaylor: I would say low priority enhancement
[10:52] <tomjenkinson> AndyTaylor: well low|high priority _enhancement_
[10:53] <AndyTaylor> tomjenkinson: very low, on my very long todo list :)
[10:53] <tomjenkinson> AndyTaylor: for me I would say it would be very useful as its hard enough trawling through logs
[10:53] <tomjenkinson> AndyTaylor: but either way, I can't see it getting in EAP 6.2
[10:53] <AndyTaylor> tomjenkinson: i couldnt agree more, but still not high priority
[10:54] <tomjenkinson> AndyTaylor: that said when I implement JBTM-860 I will be after you to do it :)
[10:54] <jbossbot> jira [JBTM-860] use XAResourceWrapper metadata for assume complete [Open (Unresolved) Enhancement, Major, Tom Jenkinson] https://issues.jboss.org/browse/JBTM-860
[10:54] <AndyTaylor> tomjenkinson: altho a small change it will semantically change how stuff works so would need thorough testing
[10:54] <AndyTaylor> tomjenkinson: well ping me when you have done that
[10:54] <tomjenkinson> AndyTaylor: JBTM-860 closes the loop when we call say commit/rollback on you and then crash and we still have the Xid in our log therefore and we constantly print "cannot recovery Xid foo"
[10:55] <tomjenkinson> AndyTaylor: because then we can say, hang on, this is for HQ and HQ doesn't have xid foo in XAR::recover so it must have completed that branch
[10:55] <AndyTaylor> tomjenkinson: that would be good, it annoys customers
[10:55] <AndyTaylor> ochaloup: you raised it yes?
[10:57] <tomjenkinson> AndyTaylor: ochaloup: I guess you guys can handle this for now? AndyTaylor I still can't parse "actually, I dont think this would be an issue as we handle enlisting recovery resources ourselves." does that apply still given the additional context we have covered above?
[10:58] <ochaloup> AndyTaylor: yeap, I raised it
[11:00] <AndyTaylor> ochaloup: could you set it to low priority
[11:02] <ochaloup> tomjenkinson, AndyTaylor: I agree. I would like to preserve info about this somewhere to anybody working on JBTM-860 could check it - do you agree if I paste there this conversation + link to the bz? then I would close the BZ for now. as the BZ probably can't be "solved" before the JBTM-860 will be ready.
[11:03] <AndyTaylor> tomjenkinson: as far as i am aware this is needed for when the JCA registers resources automatically
[11:03] <AndyTaylor> tomjenkinson: but since we do our own
This is fixed upstream in HornetQ, commits:
This fix is not in HornetQ 2.3.21.Final (in EAP 6.4.0.DR1) and based on last comment probably will not be in EAP 6.4.0 at all. Returning to modified.
The above commits are (it seems) not dealing with Arunja treating "dont know TXID" as "must be commited". Is theresomewhere a different bug or commit for this?
(In reply to Miroslav Novak from comment #15)
> This fix is not in HornetQ 2.3.21.Final (in EAP 6.4.0.DR1) and based on last
> comment probably will not be in EAP 6.4.0 at all. Returning to modified.
I am setting this back to NEW, which is the correct state for issues which have not made it in and noone working on them. MODIFIED (at least for 6.4.0) is for issues which are merged and usable from a built EAP 6.4 instance
After Hornet crash we also get these after restarting the server. Pretty annoying. Is the only option really manyally deleting tx w/ jboss CLI, or deleting the data folder ?
@matzew: I would bet that you refer to a little bit different case as you talk about having log messages being emitted after restarting the server. This issue talks about XAResource data being part of an active transaction. When the server is restarted then you probably see warnings from periodic recovery. Could that be those described at the forum : https://developer.jboss.org/message/901308#901308 ?
If yes there is optional system property -DJTAEnvironmentBean.xaAssumeRecoveryComplete=true - it works only for JTA and you need to use it with care as it could have consequences for data consistency.