Bug 855350
Summary: | org.infinispan.distribution.topologyaware.TopologyAwareStateTransferTest.testNodeDown3 fails randomly | ||
---|---|---|---|
Product: | [JBoss] JBoss Data Grid 6 | Reporter: | Anna Manukyan <amanukya> |
Component: | Infinispan | Assignee: | Tristan Tarrant <ttarrant> |
Status: | ASSIGNED --- | QA Contact: | Martin Gencur <mgencur> |
Severity: | medium | Docs Contact: | |
Priority: | unspecified | ||
Version: | 6.0.1 | CC: | jdg-bugs |
Target Milestone: | ER5 | ||
Target Release: | 6.2.0 | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | Type: | Bug | |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Anna Manukyan
2012-09-07 13:43:05 UTC
The bug doesn't appear when the test is run separately. Per ttarrant and mgencur, these are related to tests running in non-representative loading environments and will likely not occur in actual customer deployments. These are not regressions or blockers so will be moved to 6.1. They do not need to be release noted for 6.0. ttarrant will add jira links as appropriate. For ER5, the testNodeDown has been failed, and just in case the provided links will be broken, the error message is: Error Message Timed out waiting for rebalancing to complete on node TopologyAwareStateTransferTest-NodeA-23408, current topology is CacheTopology{id=9, currentCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[TopologyAwareStateTransferTest-NodeA-23408, TopologyAwareStateTransferTest-NodeB-47156, TopologyAwareStateTransferTest-NodeC-50852, TopologyAwareStateTransferTest-NodeD-5971], owners={0: 0 1, 1: 0 1, 2: 0 1, 3: 0 1, 4: 0 1, 5: 0 1, 6: 0 1, 7: 0 1, 8: 0 1, 9: 0 1, 10: 0 1, 11: 0 1, 12: 0, 13: 0, 14: 0, 15: 3 0, 16: 3 0, 17: 3 0, 18: 3 0, 19: 3 0, 20: 2 0, 21: 2 0, 22: 2 0, 23: 2 0, 24: 2 0, 25: 2 0, 26: 2 0, 27: 2 0, 28: 2 0, 29: 2 0, 30: 1 0, 31: 1 0, 32: 1 0, 33: 1 0, 34: 1 0, 35: 1 0, 36: 1 0, 37: 1 0, 38: 1 0, 39: 1 0, 40: 1 0, 41: 1 0, 42: 1, 43: 1, 44: 1, 45: 3 1, 46: 3 1, 47: 3 1, 48: 3 1, 49: 3 1, 50: 2 1, 51: 2 1, 52: 2, 53: 2, 54: 2, 55: 3 2, 56: 3 2, 57: 3, 58: 3, 59: 3}, pendingCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[TopologyAwareStateTransferTest-NodeA-23408, TopologyAwareStateTransferTest-NodeB-47156, TopologyAwareStateTransferTest-NodeC-50852, TopologyAwareStateTransferTest-NodeD-5971], owners={0: 0 1, 1: 0 1, 2: 0 1, 3: 0 1, 4: 0 1, 5: 0 1, 6: 0 1, 7: 0 1, 8: 0 1, 9: 0 1, 10: 0 1, 11: 0 1, 12: 0 2, 13: 0 2, 14: 0 2, 15: 3 0, 16: 3 0, 17: 3 0, 18: 3 0, 19: 3 0, 20: 2 0, 21: 2 0, 22: 2 0, 23: 2 0, 24: 2 0, 25: 2 0, 26: 2 0, 27: 2 0, 28: 2 0, 29: 2 0, 30: 1 0, 31: 1 0, 32: 1 0, 33: 1 0, 34: 1 0, 35: 1 0, 36: 1 0, 37: 1 0, 38: 1 0, 39: 1 0, 40: 1 0, 41: 1 0, 42: 1 2, 43: 1 2, 44: 1 2, 45: 3 1, 46: 3 1, 47: 3 1, 48: 3 1, 49: 3 1, 50: 2 1, 51: 2 1, 52: 2 3, 53: 2 3, 54: 2 3, 55: 3 2, 56: 3 2, 57: 3 2, 58: 3 2, 59: 3 2}} Stacktrace java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node TopologyAwareStateTransferTest-NodeA-23408, current topology is CacheTopology{id=9, currentCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[TopologyAwareStateTransferTest-NodeA-23408, TopologyAwareStateTransferTest-NodeB-47156, TopologyAwareStateTransferTest-NodeC-50852, TopologyAwareStateTransferTest-NodeD-5971], owners={0: 0 1, 1: 0 1, 2: 0 1, 3: 0 1, 4: 0 1, 5: 0 1, 6: 0 1, 7: 0 1, 8: 0 1, 9: 0 1, 10: 0 1, 11: 0 1, 12: 0, 13: 0, 14: 0, 15: 3 0, 16: 3 0, 17: 3 0, 18: 3 0, 19: 3 0, 20: 2 0, 21: 2 0, 22: 2 0, 23: 2 0, 24: 2 0, 25: 2 0, 26: 2 0, 27: 2 0, 28: 2 0, 29: 2 0, 30: 1 0, 31: 1 0, 32: 1 0, 33: 1 0, 34: 1 0, 35: 1 0, 36: 1 0, 37: 1 0, 38: 1 0, 39: 1 0, 40: 1 0, 41: 1 0, 42: 1, 43: 1, 44: 1, 45: 3 1, 46: 3 1, 47: 3 1, 48: 3 1, 49: 3 1, 50: 2 1, 51: 2 1, 52: 2, 53: 2, 54: 2, 55: 3 2, 56: 3 2, 57: 3, 58: 3, 59: 3}, pendingCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[TopologyAwareStateTransferTest-NodeA-23408, TopologyAwareStateTransferTest-NodeB-47156, TopologyAwareStateTransferTest-NodeC-50852, TopologyAwareStateTransferTest-NodeD-5971], owners={0: 0 1, 1: 0 1, 2: 0 1, 3: 0 1, 4: 0 1, 5: 0 1, 6: 0 1, 7: 0 1, 8: 0 1, 9: 0 1, 10: 0 1, 11: 0 1, 12: 0 2, 13: 0 2, 14: 0 2, 15: 3 0, 16: 3 0, 17: 3 0, 18: 3 0, 19: 3 0, 20: 2 0, 21: 2 0, 22: 2 0, 23: 2 0, 24: 2 0, 25: 2 0, 26: 2 0, 27: 2 0, 28: 2 0, 29: 2 0, 30: 1 0, 31: 1 0, 32: 1 0, 33: 1 0, 34: 1 0, 35: 1 0, 36: 1 0, 37: 1 0, 38: 1 0, 39: 1 0, 40: 1 0, 41: 1 0, 42: 1 2, 43: 1 2, 44: 1 2, 45: 3 1, 46: 3 1, 47: 3 1, 48: 3 1, 49: 3 1, 50: 2 1, 51: 2 1, 52: 2 3, 53: 2 3, 54: 2 3, 55: 3 2, 56: 3 2, 57: 3 2, 58: 3 2, 59: 3 2}} at org.infinispan.test.TestingUtil.waitForRehashToComplete(TestingUtil.java:198) at org.infinispan.test.TestingUtil.waitForRehashToComplete(TestingUtil.java:208) at org.infinispan.distribution.topologyaware.TopologyAwareStateTransferTest.testNodeDown(TopologyAwareStateTransferTest.java:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80) at org.testng.internal.Invoker.invokeMethod(Invoker.java:715) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:907) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1237) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at org.testng.TestRunner.privateRun(TestRunner.java:767) at org.testng.TestRunner.run(TestRunner.java:617) at org.testng.SuiteRunner.runTest(SuiteRunner.java:334) at org.testng.SuiteRunner.access$000(SuiteRunner.java:37) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368) at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Anna Manukyan <amanukya> updated the status of jira ISPN-2516 to Resolved Anna Manukyan <amanukya> made a comment on jira ISPN-2516 The issue doesn't appear any more. I've run the build for Infinispan 5.4.2.Final for several times, and the issue didn't appear. Hi, the failure reappears on java16_default,w2k8r2 && x86_64 environment. The failure is: Error Message Timed out before caches had complete views. Expected 2 members in each view. Views are as follows: [[TopologyAwareStateTransferTest-NodeA-65287], [TopologyAwareStateTransferTest-NodeA-65287, TopologyAwareStateTransferTest-NodeB-35578, TopologyAwareStateTransferTest-NodeD-57223]] Stacktrace java.lang.RuntimeException: Timed out before caches had complete views. Expected 2 members in each view. Views are as follows: [[TopologyAwareStateTransferTest-NodeA-65287], [TopologyAwareStateTransferTest-NodeA-65287, TopologyAwareStateTransferTest-NodeB-35578, TopologyAwareStateTransferTest-NodeD-57223]] at org.infinispan.test.TestingUtil.viewsTimedOut(TestingUtil.java:248) at org.infinispan.test.TestingUtil.viewsTimedOut(TestingUtil.java:238) at org.infinispan.test.TestingUtil.blockUntilViewsReceived(TestingUtil.java:333) at org.infinispan.test.TestingUtil.blockUntilViewsReceived(TestingUtil.java:907) at org.infinispan.distribution.topologyaware.TopologyAwareStateTransferTest.testNodeDown3(TopologyAwareStateTransferTest.java:122) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80) at org.testng.internal.Invoker.invokeMethod(Invoker.java:715) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:907) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1237) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at org.testng.TestRunner.privateRun(TestRunner.java:767) at org.testng.TestRunner.run(TestRunner.java:617) at org.testng.SuiteRunner.runTest(SuiteRunner.java:334) at org.testng.SuiteRunner.access$000(SuiteRunner.java:37) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368) at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662) Standard Output [testng-TopologyAwareStateTransferTest] Test testNodeDown3(org.infinispan.distribution.topologyaware.TopologyAwareStateTransferTest) failed. Test suite progress: tests succeeded: 1651, failed: 1, skipped: 0. Anna Manukyan <amanukya> updated the status of jira ISPN-2516 to Reopened Anna Manukyan <amanukya> made a comment on jira ISPN-2516 Hi, the issue again appears on Windows 2008R2 64 bit machine with JDK6. You can find the infinispan tracelog for core module attached. Regards, Anna. Dan Berindei <dberinde> updated the status of jira ISPN-2516 to Resolved |