Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1160761

Summary: Intermittent failures in SendMessagesTestCase especially when transactions run with JTS
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Ondrej Chaloupka <ochaloup>
Component: TestsuiteAssignee: Ondrej Chaloupka <ochaloup>
Status: CLOSED CURRENTRELEASE QA Contact: Petr Kremensky <pkremens>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.4.0CC: cdewolf, kkhan
Target Milestone: DR9   
Target Release: EAP 6.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-08-19 12:44:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
server.log when server runs with jts - case when failed and case when didn't fail none

Description Ondrej Chaloupka 2014-11-05 15:01:22 UTC
There could be hit intermittent failure in test org.jboss.as.test.integration.ejb.mdb.containerstart.SendMessagesTestCase. This could be observed especially when transaction is set to be run with JTS.

The failure exception is following:
java.lang.AssertionError: did not receive message with ordered number 62 in 90000 seconds
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.junit.Assert.assertNotNull(Assert.java:621)
	at org.jboss.as.test.integration.ejb.mdb.containerstart.SendMessagesTestCase.testShutdown(SendMessagesTestCase.java:248)

Comment 1 Ondrej Chaloupka 2014-11-05 15:16:05 UTC
Created attachment 954094 [details]
server.log when server runs with jts - case when failed and case when didn't fail

From my observation this is test issue and the reason under this issue is timing. 

If message driven bean is undeployed before the onMessage method got to "await" on barrier then await throws InterruptedException and the message is stuck in hornetq and not redelivered. This situation happens with jts as it takes a bit more for transaction manager to start txn and undeploy thread has time to go on.

If message driven bean is undeployed after the onMessage method got to "await" (what is expected behaior) then timeout of transaction could happen and txn is rollbacked and the message is redelivered.

Comment 2 Ondrej Chaloupka 2014-11-05 15:33:51 UTC
Thanks to @mnovak I've understood the reason of the problem. I will try to describe down here.

Method onMessage starts the transaction and as undeploy starts to be active the barrier await throws InterruptedException.
Transaction manager starts transaction at the entry of onMessage method. Transaction manager set the transaction to be rollbacked and do nothing as it there was no action done which could be set to rollback (TM had not started 2PC).
But when HornetQ passed the message to MDB it put message to "internal indoubt state". Message in such state is returned back to queue after transaction timeout period of time (see list of attributes e.g. at https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Operations_Network/3.2/html/Complete_Resource_Reference/JBossAS7-JBossAS7_Standalone_Server-Messaging.html). This timeout is timeout of hornetq and not of TM. 

The test passes if the receiver waits for longer time to message timeouted from hornetq "internal indoubt state". This is by default 5min.


The reference runs of jenkins job:
failing run: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-64-jbossas-testsuite-jts-matrix-ochaloup/7/

Comment 3 Ondrej Chaloupka 2014-11-06 10:48:13 UTC
One other thing is that setting rollback only on thread for jta does not mean invoking interrupted exception on sleeping/waiting methods. But jts does throw such exception.

passing run: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-64-jbossas-testsuite-jts-matrix-ochaloup/13/

Comment 4 Ondrej Chaloupka 2014-11-06 14:36:01 UTC
There is fix in my branch here
https://github.com/ochaloup/jboss-eap/commits/6.x-SendMessagesTestCase

Comment 5 JBoss JIRA Server 2014-11-06 16:21:34 UTC
Ondřej Chaloupka <ochaloup> updated the status of jira WFLY-4060 to Resolved

Comment 6 Kabir Khan 2014-11-07 13:54:09 UTC
https://github.com/jbossas/jboss-eap/pull/1901

Comment 7 Ondrej Chaloupka 2014-11-12 13:31:35 UTC
Run with DR9 and JTS and the failure was not shown.
Putting as verified.