Bug 1020953 - JTS qa test CrashRecovery12_Test03 fails running with HornetQ object store
Summary: JTS qa test CrashRecovery12_Test03 fails running with HornetQ object store
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Transaction Manager
Version: 6.2.0,6.2.4
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: GA
: EAP 6.3.0
Assignee: Michael
QA Contact: Ondrej Chaloupka
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-10-18 15:15 UTC by Ondrej Chaloupka
Modified: 2019-08-19 12:42 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-08-19 12:42:45 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker JBTM-1980 0 Major Closed QA test suite failure on HQStore: CrashRecovery12_Test03 2017-03-06 07:56:37 UTC

Description Ondrej Chaloupka 2013-10-18 15:15:38 UTC
There are a failure at test crashrecovery12 on method CrashRecovery12_Test03. This was reported for narayana 5.0.0 but the error occurs in 4.17.12.Final as well (which is version for EAP 6.2.0.ER6).

Running like:
./build.sh clean install -Pall  -Dmaven.test.skip.exec=true
cd qa/
ant dist
ant -f run-tests.xml -Dtest.methods="CrashRecovery12_Test03" -Dtest=crashrecovery12 -Dprofile=hornetq test

Fail:
Testcase: CrashRecovery12_Test03 took 31.498 sec
 FAILED
task outcome0 printed Failed.
junit.framework.AssertionFailedError: task outcome0 printed Failed.
 at org.jboss.jbossts.qa.junit.TaskImpl$TaskReaderThread.checkPassFail(TaskImpl.java:671)
 at org.jboss.jbossts.qa.junit.TaskImpl.waitFor(TaskImpl.java:430)
 at org.jboss.jbossts.qa.junit.testgroup.TestGroup_crashrecovery12.CrashRecovery12_Test03(TestGroup_crashrecovery12.java:80)
 at org.jboss.jbossts.qa.junit.QATestNameRule$1.evaluate(QATestNameRule.java:89)

Comment 2 Michael 2013-10-25 10:29:15 UTC
Note that this failure is a bug in the test design itself and does not affect the functionality of narayana itself - my fix is to the test suite and I did not touch the code that goes into the product

Comment 3 JBoss JIRA Server 2013-11-07 13:49:28 UTC
Gytis Trikleris <gytis> updated the status of jira JBTM-1980 to Closed

Comment 4 JBoss JIRA Server 2013-11-11 11:27:13 UTC
Gytis Trikleris <gytis> updated the status of jira JBTM-1980 to Reopened

Comment 5 JBoss JIRA Server 2013-11-11 11:27:13 UTC
Gytis Trikleris <gytis> made a comment on jira JBTM-1980

Looks like the same issue http://172.17.131.2/view/Narayana+BlackTie/job/narayana-hqstore-jacorb/22/TESTS=QA_JTS_JACORB,jdk=jdk7.latest,label=linux/

Comment 6 JBoss JIRA Server 2013-11-11 12:46:17 UTC
Michael Musgrove <mmusgrov> updated the status of jira JBTM-1980 to Resolved

Comment 7 JBoss JIRA Server 2013-11-11 12:46:17 UTC
Michael Musgrove <mmusgrov> made a comment on jira JBTM-1980

The referenced CI failure on the previous comment affects 5.0.0.CR1 only so I am resolving the JIRA again since the problem is fixed on the 4.17 branch.

I have removed 5.0.0.CR1 from the fix for field. See linked JIRA to track the problem on the 5.0.0.CR1 branch)

Comment 8 JBoss JIRA Server 2013-11-11 16:03:33 UTC
Michael Musgrove <mmusgrov> updated the status of jira JBTM-1980 to Reopened

Comment 9 JBoss JIRA Server 2013-11-11 16:03:33 UTC
Michael Musgrove <mmusgrov> made a comment on jira JBTM-1980

The fix is in both branches (4.17 and master) so even though the failure was seen against master it could occur in 4.17 also. 

From the logs it looks like the test task is being reaped after 58 seconds (although the test config says it should only be reaped afater 240 seconds). This means that my fix to overcome port recycling did not have sufficient time to take effect.

My line of investigation is to determine why the task was reaped early.

Comment 10 JBoss JIRA Server 2013-12-03 14:14:50 UTC
Michael Musgrove <mmusgrov> updated the status of jira JBTM-1980 to Resolved

Comment 11 JBoss JIRA Server 2013-12-03 14:14:50 UTC
Michael Musgrove <mmusgrov> made a comment on jira JBTM-1980

I am unable to reproduce this issue - the PR adds extra logging information to help diagnose if it does ruccur.

Comment 12 JBoss JIRA Server 2013-12-05 09:55:38 UTC
Michael Musgrove <mmusgrov> updated the status of jira JBTM-1980 to Reopened

Comment 13 JBoss JIRA Server 2013-12-05 09:55:38 UTC
Michael Musgrove <mmusgrov> made a comment on jira JBTM-1980

Recurrence of the same issue (with extra trace)

Comment 14 JBoss JIRA Server 2013-12-05 16:50:36 UTC
Michael Musgrove <mmusgrov> made a comment on jira JBTM-1980

The task that checks for recovery runs before the task that generates the crash scenario has finished. I have added a fix that delays the second task until the first task is ready.
I'm testing it with a dedicated job: http://172.17.131.2/job/JBTM-1980-hqstore/ to test my fix

Comment 15 JBoss JIRA Server 2013-12-08 17:59:12 UTC
Michael Musgrove <mmusgrov> updated the status of jira JBTM-1980 to Resolved

Comment 16 JBoss JIRA Server 2013-12-08 17:59:12 UTC
Michael Musgrove <mmusgrov> made a comment on jira JBTM-1980

The latest failure is because the client task that creates the crash record takes a while to start (because of port recycling) and the checker task that validates recovery occurred runs before the client has had time to generate the crash rec.

The fix is to wait for the client task to become ready before launching the checker task. I have only applied the fix to CrashRecovery12_Test03. If we ever get similar failures for other tests then we should apply the same fix as necessary.

Comment 17 JBoss JIRA Server 2013-12-11 17:37:06 UTC
Tom Jenkinson <tom.jenkinson> updated the status of jira JBTM-1980 to Closed

Comment 18 Michael 2014-08-21 10:36:48 UTC
Can we close this one yet or are you still seeing the failure?

Comment 19 Ondrej Chaloupka 2014-08-21 12:26:28 UTC
Hi, 
I can't see the problem for 4.17.21.Final which is under 6.3.0.GA. So setting as verified from QA point of view.

Thanks
Ondra


Note You need to log in before you can comment on or make changes to this bug.