Bug 895502

Summary:	org.infinispan.distexec.mapreduce.TopologyAwareTwoNodesMapReduceTest.testInvokeMapperCancellation test fails randomly
Product:	[JBoss] JBoss Data Grid 6	Reporter:	Anna Manukyan <amanukya>
Component:	Infinispan	Assignee:	Tristan Tarrant <ttarrant>
Status:	VERIFIED ---	QA Contact:	Martin Gencur <mgencur>
Severity:	high	Docs Contact:
Priority:	medium
Version:	6.1.0	CC:	jdg-bugs
Target Milestone:	ER10
Target Release:	6.1.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Anna Manukyan 2013-01-15 11:15:36 UTC

org.infinispan.distexec.mapreduce.TopologyAwareTwoNodesMapReduceTest.testInvokeMapperCancellation test fails randomly on all environments.

The error log is:

org.testng.TestException: 
Expected exception java.util.concurrent.CancellationException but got java.lang.AssertionError: Mapper not cancelled, root cause org.jgroups.TimeoutException: timeout sending message to TopologyAwareTwoNodesMapReduceTest-NodeB-22523(test2)
	at org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1503)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:764)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:907)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1237)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:767)
	at org.testng.TestRunner.run(TestRunner.java:617)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
	at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.AssertionError: Mapper not cancelled, root cause org.jgroups.TimeoutException: timeout sending message to TopologyAwareTwoNodesMapReduceTest-NodeB-22523(test2)
	at org.infinispan.distexec.mapreduce.SimpleTwoNodesMapReduceTest.testInvokeMapperCancellation(SimpleTwoNodesMapReduceTest.java:106)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:715)
	... 15 more

You can find the Jenkins failures here:

http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-testsuite-rhel/jdk=openjdk-1.7.0-local,label=RHEL6_x86/74/testReport/org.infinispan.distexec.mapreduce/TopologyAwareTwoNodesMapReduceTest/testInvokeMapperCancellation/

Comment 2 JBoss JIRA Server 2013-01-23 09:17:42 UTC

Anna Manukyan <amanukya> made a comment on jira ISPN-2714

Seems that the issue was just test thing - timing issue. The cancellation of the map reduce task was done with separate thread which probably was getting the priority to work before map reduce task execution start which was causing no task cancellation.

Comment 3 Anna Manukyan 2013-01-31 13:12:51 UTC

This bug doesn't appear any more on JDG 6.1.0.ER10. Can be set to Verified.

Comment 4 Anna Manukyan 2013-02-22 10:02:02 UTC

The issue didn't appear for recent ER builds. Marking the bug as verified!

Comment 6 JBoss JIRA Server 2013-06-28 14:41:59 UTC

Dan Berindei <dberinde> updated the status of jira ISPN-2714 to Reopened

Comment 7 JBoss JIRA Server 2013-06-28 14:41:59 UTC

Dan Berindei <dberinde> made a comment on jira ISPN-2714

The problem seems to be that TopologyAwareTwoNodesMapReduceTest and SimpleTwoNodesMapReduceTest share a static counter variable, and when they execute in parallel one of them will fail.