Bug 1298119 - ClientEventsOOMTest.testOOM fails on RHEL 7
ClientEventsOOMTest.testOOM fails on RHEL 7
Status: NEW
Product: JBoss Data Grid 6
Classification: JBoss
Component: Server (Show other bugs)
6.6.0
Unspecified Unspecified
unspecified Severity unspecified
: ---
: 7.0.0
Assigned To: Galder Zamarreño
Martin Gencur
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-13 05:04 EST by Roman Macor
Modified: 2016-03-11 04:55 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker JDG-15 Major Verified ClientEventsOOMTest.testOOM fails on RHEL 7 2016-08-08 09:27 EDT

  None (edit)
Description Roman Macor 2016-01-13 05:04:27 EST
Description of problem:

org.infinispan.client.hotrod.event.ClientEventsOOMTest.testOOM fails with:

Error Message

java.net.SocketTimeoutException

Stacktrace

org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException
	at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:184)
	at org.infinispan.client.hotrod.impl.protocol.Codec20.readMagic(Codec20.java:305)
	at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeaderOrEvent(Codec20.java:205)
	at org.infinispan.client.hotrod.impl.operations.AddClientListenerOperation.executeOperation(AddClientListenerOperation.java:92)
	at org.infinispan.client.hotrod.impl.operations.AddClientListenerOperation.executeOperation(AddClientListenerOperation.java:25)
	at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:56)
	at org.infinispan.client.hotrod.impl.RemoteCacheImpl.addClientListener(RemoteCacheImpl.java:516)
	at org.infinispan.client.hotrod.event.ClientEventsOOMTest.testOOM(ClientEventsOOMTest.java:86)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:767)
	at org.testng.TestRunner.run(TestRunner.java:617)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
	at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException
	at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:211)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
	at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:179)
	... 27 more
Comment 3 Vojtech Juranek 2016-01-14 16:06:37 EST
So here is quick summary what I did:
I took one RHEL7 machine from our Jenkins lab and also provision one fresh RHEL 7 machine from Beaker.

* on Beaker machine test never fails (run 10-15 time with JDK8, didn't fail even with JDK 7)

* on Jenkins machine failed in cca 80% (again 10-15 runs, all with JDK8), but never failed when run with trace logging

* I suspected it's some environment issue, so I setup same ulimits on both machine and also sync some kernel parameters (otherwise JDK versions were same, also same kernels), but the result was same

* I took thread dumps on both machines during registering listeners, but didn't spot anything suspicious (as far as I can tell, both stack traces were very similar)
Comment 4 Martin Gencur 2016-01-15 04:29:28 EST
Nice analysis, Vojta! It looks like the issue is both random and depending on environment setup.
Comment 5 JBoss JIRA Server 2016-03-07 05:17:39 EST
Sebastian Łaskawiec <slaskawi@redhat.com> updated the status of jira JDG-15 to Coding In Progress
Comment 6 JBoss JIRA Server 2016-03-11 04:55:14 EST
Vaclav Dedik <vdedik@redhat.com> updated the status of jira JDG-15 to Resolved

Note You need to log in before you can comment on or make changes to this bug.