Bug 1426471 - [GSS] (6.4.z) deadlock in org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.onWritePossible
Summary: [GSS] (6.4.z) deadlock in org.apache.tomcat.websocket.server.WsRemoteEndpoint...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Web
Version: 6.4.12
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: CR1
: EAP 6.4.15
Assignee: Radovan Netuka
QA Contact: Radim Hatlapatka
URL:
Whiteboard:
Depends On:
Blocks: eap6415-payload 1427196 1427753
TreeView+ depends on / blocked
 
Reported: 2017-02-24 01:51 UTC by Masanobu Hatanaka
Modified: 2020-08-13 08:53 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-05-19 08:08:12 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3024581 0 None None None 2017-05-05 17:47:41 UTC

Description Masanobu Hatanaka 2017-02-24 01:51:22 UTC
Description of problem:
The deadlock is defected in WebSocket on EAP 6.4.12. 

Found one Java-level deadlock:
=============================
"WebSocketServer-/asahi/i-smart-54":
  waiting to lock monitor 0x00007fb16819df98 (object 0x00000004127d7d00, a java.lang.Object),
  which is held by "WebSocketServer-/asahi/i-smart-53"
"WebSocketServer-/asahi/i-smart-53":
  waiting to lock monitor 0x00007fb26c1f9e68 (object 0x000000040631dd00, a org.apache.coyote.http11.Http11NioProcessor),
  which is held by "HTTP-1831"
"HTTP-1831":
  waiting to lock monitor 0x00007fb16819df98 (object 0x00000004127d7d00, a java.lang.Object),
  which is held by "WebSocketServer-/asahi/i-smart-53"

Java stack information for the threads listed above:
===================================================
"WebSocketServer-/asahi/i-smart-54":
	at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.onWritePossible(WsRemoteEndpointImplServer.java:81)
	- waiting to lock <0x00000004127d7d00> (a java.lang.Object)
	at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.doWrite(WsRemoteEndpointImplServer.java:75)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.writeMessagePart(WsRemoteEndpointImplBase.java:445)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.endMessage(WsRemoteEndpointImplBase.java:363)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$EndMessageHandler.onResult(WsRemoteEndpointImplBase.java:483)
	at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer$OnResultRunnable.run(WsRemoteEndpointImplServer.java:247)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
"WebSocketServer-/asahi/i-smart-53":
	at org.apache.coyote.http11.Http11NioProcessor.writeEvent(Http11NioProcessor.java:760)
	- waiting to lock <0x000000040631dd00> (a org.apache.coyote.http11.Http11NioProcessor)
	at org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:859)
	at org.apache.coyote.Response.action(Response.java:188)
	at org.apache.catalina.connector.OutputBuffer.lastWrite(OutputBuffer.java:392)
	at org.apache.catalina.connector.CoyoteOutputStream.isReady(CoyoteOutputStream.java:112)
	at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.onWritePossible(WsRemoteEndpointImplServer.java:89)
	- locked <0x00000004127d7d00> (a java.lang.Object)
	at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.doWrite(WsRemoteEndpointImplServer.java:75)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.writeMessagePart(WsRemoteEndpointImplBase.java:445)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.startMessage(WsRemoteEndpointImplBase.java:335)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$TextMessageSendHandler.write(WsRemoteEndpointImplBase.java:741)
	- locked <0x00000004127d7d00> (a java.lang.Object)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$TextMessageSendHandler.onResult(WsRemoteEndpointImplBase.java:757)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.endMessage(WsRemoteEndpointImplBase.java:371)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$EndMessageHandler.onResult(WsRemoteEndpointImplBase.java:483)
	at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer$OnResultRunnable.run(WsRemoteEndpointImplServer.java:245)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
"HTTP-1831":
	at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.onWritePossible(WsRemoteEndpointImplServer.java:81)
	- waiting to lock <0x00000004127d7d00> (a java.lang.Object)
	at org.apache.tomcat.websocket.server.WsHttpUpgradeHandler$WsWriteListener.onWritePossible(WsHttpUpgradeHandler.java:243)
	at org.apache.catalina.core.StandardWrapperValve.async(StandardWrapperValve.java:605)
	at org.apache.catalina.core.StandardWrapperValve.event(StandardWrapperValve.java:350)
	at org.apache.catalina.core.StandardContextValve.event(StandardContextValve.java:171)
	at org.apache.catalina.valves.ValveBase.event(ValveBase.java:185)
	at org.apache.catalina.core.StandardHostValve.event(StandardHostValve.java:252)
	at org.apache.catalina.valves.ValveBase.event(ValveBase.java:185)
	at org.apache.catalina.core.StandardEngineValve.event(StandardEngineValve.java:121)
	at org.apache.catalina.connector.CoyoteAdapter.event(CoyoteAdapter.java:228)
	at org.apache.coyote.http11.Http11NioProcessor.event(Http11NioProcessor.java:232)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.event(Http11NioProtocol.java:819)
	- locked <0x000000040631dd00> (a org.apache.coyote.http11.Http11NioProcessor)
	at org.apache.tomcat.util.net.NioEndpoint$ChannelProcessor.run(NioEndpoint.java:940)
	- locked <0x00000004127c9560> (a java.lang.Object)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
	at org.jboss.threads.JBossThread.run(JBossThread.java:122)

Found 1 deadlock.


How reproducible:
No reproducible steps but the customer told that this deadlock can be found a few times per day.

I thought this was similar but no related and it seems that Bug#1410869 would resolve the issue.
[GSS](6.4.z) Deadlock in WsRemoteEndpointImplServer.onWritePossible
https://bugzilla.redhat.com/show_bug.cgi?id=1299057

Comment 4 Michael Cada 2017-04-26 12:53:49 UTC
As we do not have reproducer, I only verified that commit which is removing unnecessary synchronization is present. No regressions were found, marking as verified. EAP 6.4.15.CP.CR2 was used for verification.

Comment 5 Petr Penicka 2017-05-19 08:08:12 UTC
Released on May 18 as part of EAP 6.4.15.


Note You need to log in before you can comment on or make changes to this bug.