Bug 1027253 - JGroups: intermittent failures in ExecutingServiceTest
JGroups: intermittent failures in ExecutingServiceTest
Status: CLOSED NOTABUG
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Clustering (Show other bugs)
6.2.0
Unspecified Unspecified
unspecified Severity medium
: ---
: EAP 6.3.0
Assigned To: Richard Achmatowicz
Jitka Kozana
Russell Dickenson
JGroups testsuite
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-06 07:36 EST by Richard Janík
Modified: 2014-10-25 08:01 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-03-27 10:36:53 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
testsuiteLog-3.txt (123.40 KB, text/plain)
2013-11-06 07:36 EST, Richard Janík
no flags Details
testsuiteLog-2.txt (122.64 KB, text/plain)
2013-11-06 07:37 EST, Richard Janík
no flags Details
testsuiteLog-1.txt (121.60 KB, text/plain)
2013-11-06 07:38 EST, Richard Janík
no flags Details

  None (edit)
Description Richard Janík 2013-11-06 07:36:13 EST
Created attachment 820312 [details]
testsuiteLog-3.txt

Description of problem:

EAP 6.2.0.ER7, but I've seen this in previous versions as well. It seems to me that it happens _slightly_ more often now.
JGroups 3.2.12.Final, also seen in previous versions.

There is an intermittent failure in configuration of tests of org.jgroups.blocks.executor.ExecutingServiceTest:

   [testng] OK:   [48] org.jgroups.blocks.executor.ExecutingServiceTest.testCoordinatorWentDownWhileSendingMessage()
   [testng] failed config: java.lang.AssertionError: Unfinished e2 futures should be empty!
   [testng] FAIL: [48] org.jgroups.blocks.executor.ExecutingServiceTest.makeSureClean()
   [testng] SKIP: [48] org.jgroups.blocks.executor.ExecutingServiceTest.testExecutionCompletionService()

This causes the rest of the tests from ExecutingServiceTest to be skipped, but then there are other tests, that to me seem unrelated, skipped as well. Parallel running of tests and methods is set to true. A snippet from a log would look like this (different log than the one above):

...
   [testng] OK:   [48] org.jgroups.tests.MessageBundlingTest.testLatencyWithMessageBundling()
   [testng] OK:   [51] org.jgroups.tests.DiscardTest.testFastProperties()
   [testng] OK:   [49] org.jgroups.blocks.executor.ExecutingServiceTest.testCoordinatorWentDownWhileSendingMessage()
   [testng] OK:   [49] org.jgroups.blocks.executor.ExecutingServiceTest.testExecutionCompletionService()
   [testng] OK:   [49] org.jgroups.blocks.executor.ExecutingServiceTest.testExecutorAwaitTerminationInterrupt()
   [testng] OK:   [49] org.jgroups.blocks.executor.ExecutingServiceTest.testExecutorAwaitTerminationNoInterrupt()
   [testng] FAIL: [49] org.jgroups.blocks.executor.ExecutingServiceTest.testInterruptTaskRequestBeforeRunning()
   [testng] failed config: java.lang.AssertionError: Unfinished e1 futures should be empty!
   [testng] FAIL: [49] org.jgroups.blocks.executor.ExecutingServiceTest.makeSureClean()
   [testng] SKIP: [49] org.jgroups.blocks.executor.ExecutingServiceTest.testInterruptWhileRunningAlot()
   [testng] SKIP: [49] org.jgroups.blocks.executor.ExecutingServiceTest.testInvokeAnyCalls()
   [testng] SKIP: [49] org.jgroups.blocks.executor.ExecutingServiceTest.testNonSerializableCallable()
   [testng] SKIP: [49] org.jgroups.blocks.executor.ExecutingServiceTest.testSimpleSerializableCallableConcurrently()
   [testng] SKIP: [49] org.jgroups.blocks.executor.ExecutingServiceTest.testSimpleSerializableCallableSubmit()
   [testng] SKIP: [49] org.jgroups.blocks.executor.ExecutingServiceTest.testSubmittingNonSerializeCallable()
   [testng] SKIP: [49] org.jgroups.blocks.executor.ExecutingServiceTest.testSubmittingSerializeCallableWithNonSerializableComponent()
   [testng] SKIP: [49] org.jgroups.tests.Deadlock2Test.testOneChannel()
   [testng] SKIP: [49] org.jgroups.tests.Deadlock2Test.testTwoChannels()
   [testng] SKIP: [49] org.jgroups.tests.Deadlock2Test.testTwoChannelsWithInitialMulticast()
   [testng] SKIP: [49] org.jgroups.blocks.RpcDispatcherInterruptTest.testMethodCallWithTimeoutNoInterrupt()
   [testng] SKIP: [49] org.jgroups.blocks.MuxRpcDispatcherTest.testMulticastRPCs()
   [testng] SKIP: [49] org.jgroups.blocks.MuxRpcDispatcherTest.testUnicastRPCs()
   [testng] SKIP: [49] org.jgroups.tests.ChannelTest.testBasicOperations()
   [testng] SKIP: [49] org.jgroups.tests.ChannelTest.testIsConnectedOnFirstViewChange()
...

A note: it happens with udp as well as tcp config. Stacktraces of the failed tests would look like this:

org.jgroups.blocks.executor.ExecutingServiceTest.testInterruptTaskRequestBeforeRunning (from org.jgroups.blocks.executor.ExecutingServiceTest-udp)

Error Message

The object in queue doesn't match

Stacktrace

java.lang.AssertionError 
at org.jgroups.blocks.executor.ExecutingServiceTest.testInterruptTaskRequestBeforeRunning(ExecutingServiceTest.java:419) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:616) 
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80) 
at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:46) 
at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:37) 
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
at java.util.concurrent.FutureTask.run(FutureTask.java:166) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:679) 


and for makeSureClean():


Error Message

Unfinished e1 futures should be empty!

Stacktrace

java.lang.AssertionError 
at org.jgroups.blocks.executor.ExecutingServiceTest.makeSureClean(ExecutingServiceTest.java:110) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:616) 
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80) 
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:564) 
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:213) 
at org.testng.internal.Invoker.invokeMethod(Invoker.java:786) 
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) 
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) 
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) 
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:679) 


Links to test results for some of the configurations affected with this are below, testsuite logs are attached:
[1]: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-manu-jgroups-rhel/26/jdk=ibm16,label=RHEL5_x86/testReport/
[2]: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-manu-jgroups-rhel/26/jdk=openjdk-1.6.0-local,label=RHEL5_x86/testReport/
[3]: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-manu-jgroups-rhel/26/jdk=openjdk-1.7.0-local,label=RHEL6_x86/testReport/
Comment 1 Richard Janík 2013-11-06 07:37:53 EST
Created attachment 820313 [details]
testsuiteLog-2.txt
Comment 2 Richard Janík 2013-11-06 07:38:32 EST
Created attachment 820314 [details]
testsuiteLog-1.txt
Comment 3 Richard Achmatowicz 2014-03-27 10:26:53 EDT
NOTE: A number of EAP clustering issues relate to failures in the JGroups testsuite. Some of these failures relate to tests of features of JGroups which are not used by EAP.

As of EAP 6.3, a modified version of the JGroups testsuite (JGroups version 3.2.13) is being executed which excludes tests of JGroups features which are not used by EAP. Only the testsuite has been modified/fixed; the product is unchanged. The list of features can be found here: https://mojo.redhat.com/docs/DOC-938761

EAP 6.3 Clustering issues relating to failures in the JGroups testsuite are being closed if they relate to features not used by EAP.
Comment 4 Richard Achmatowicz 2014-03-27 10:36:53 EDT
This test is not within the scope of EAP 6.3 testing.

Note You need to log in before you can comment on or make changes to this bug.