Bug 1284534

Summary: DistributedIntermediateSharedCacheFourNodesMapReduceTest>BaseWordCountMapReduceTest.testInvokeMapReduceOnAllKeysWithCollatorAsync fails randomly
Product: [JBoss] JBoss Data Grid 6 Reporter: Martin Vrabel <mvrabel>
Component: TestSuiteAssignee: Martin Gencur <mgencur>
Status: NEW --- QA Contact: Martin Gencur <mgencur>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.6.0CC: afield, jdg-bugs, ttarrant
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Martin Vrabel 2015-11-23 14:32:40 UTC
Description of problem:

testInvokeMapReduceOnAllKeysWithCollatorAsync(org.infinispan.distexec.mapreduce.DistributedIntermediateSharedCacheFourNodesMapReduceTest)  Time elapsed: 15.051 sec  <<< FAILURE!
java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: org.infinispan.commons.CacheException: Could not m$
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.infinispan.commons.util.concurrent.NotifyingFutureImpl.get(NotifyingFutureImpl.java:73)
        at org.infinispan.distexec.mapreduce.BaseWordCountMapReduceTest.testInvokeMapReduceOnAllKeysWithCollatorAsync(BaseWordCountMapReduceTest.java:308)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
        at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
        at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
        at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
        at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
        at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
        at org.testng.TestRunner.privateRun(TestRunner.java:767)
        at org.testng.TestRunner.run(TestRunner.java:617)
        at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
        at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
        at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
        at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.commons.CacheException: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: org.infinispan.commons.CacheException: Could not move intermediate keys/values f$
        at org.infinispan.distexec.mapreduce.MapReduceTask.executeHelper(MapReduceTask.java:482)
        at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:412)
        at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:852)
        at org.infinispan.distexec.mapreduce.MapReduceTask$3.call(MapReduceTask.java:872)
        ... 4 more
Caused by: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: org.infinispan.commons.CacheException: Could not move intermediate keys/values for M/R task d4bd10ca-51f4-45bf-b640-a5c$
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.infinispan.distexec.mapreduce.MapReduceTask$TaskPart.get(MapReduceTask.java:1067)
        at org.infinispan.distexec.mapreduce.MapReduceTask.executeMapPhase(MapReduceTask.java:604)
        at org.infinispan.distexec.mapreduce.MapReduceTask.executeHelper(MapReduceTask.java:476)
        ... 7 more
Caused by: org.infinispan.commons.CacheException: org.infinispan.commons.CacheException: Could not move intermediate keys/values for M/R task d4bd10ca-51f4-45bf-b640-a5ccbfbd25c5
        at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:116)
        at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.invokeMapCombineLocally(MapReduceTask.java:1193)
        at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.access$200(MapReduceTask.java:1120)
        at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:1142)
        at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:1138)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        ... 4 more
Caused by: org.infinispan.commons.CacheException: Could not move intermediate keys/values for M/R task d4bd10ca-51f4-45bf-b640-a5ccbfbd25c5
        at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.migrateIntermediateKeysAndValues(MapReduceManagerImpl.java:375)
        at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombine(MapReduceManagerImpl.java:282)
        at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:114)
        ... 10 more
Caused by: org.infinispan.util.concurrent.TimeoutException: Node DistributedIntermediateSharedCacheFourNodesMapReduceTest-NodeD-50733 timed out
        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:217)
        at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:564)
        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:292)
        at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.handleNonTxWriteCommand(BaseDistributionInterceptor.java:316)
        at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutKeyValueCommand(NonTxDistributionInterceptor.java:88)
        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
        at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:366)
        at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:454)
        at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:188)
        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
        at org.infinispan.expiration.impl.ExpirationInterceptor.handleWriteCommand(ExpirationInterceptor.java:64)
        at org.infinispan.expiration.impl.ExpirationInterceptor.visitPutKeyValueCommand(ExpirationInterceptor.java:35)
        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
        at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:46)
        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
        at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
        at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:336)
        at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:268)
        at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:123)
        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
        at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
        at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1607)
        at org.infinispan.CacheImpl.putInternal(CacheImpl.java:1079)
        at org.infinispan.CacheImpl.put(CacheImpl.java:1070)
        at org.infinispan.DecoratedCache.put(DecoratedCache.java:413)
        at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.migrateIntermediateKeysAndValues(MapReduceManagerImpl.java:367)
        ... 12 more
Caused by: org.jgroups.TimeoutException: timeout waiting for response from DistributedIntermediateSharedCacheFourNodesMapReduceTest-NodeD-50733, request: org.jgroups.blocks.UnicastRequest@1ca814e8, req_id=14087$
        at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:427)
        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:401)
        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:210)
        ... 54 more



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: