Bug 1248299 - [GSS] (6.4.z) Deadlock in remoting during channel shutdown
[GSS] (6.4.z) Deadlock in remoting during channel shutdown
Status: CLOSED DUPLICATE of bug 1215714
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Remoting (Show other bugs)
6.4.0
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: James Livingston
Jitka Kozana
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-07-30 00:20 EDT by James Livingston
Modified: 2016-02-21 19:56 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-07-30 00:28:48 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker WFLY-4113 Major Resolved 8.2.0.Final build hangs 2016-03-18 15:20 EDT

  None (edit)
Description James Livingston 2015-07-30 00:20:00 EDT
There is a deadlock risk in remoting which produces thread like:
"Remoting "host:MANAGEMENT" read-1" #149 prio=5 os_prio=0 tid=0x00007fbecc038800 nid=0x3f12 waiting for monitor entry [0x00007fbe136b0000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.jboss.remoting3.remote.OutboundMessage.cancel(OutboundMessage.java:289)
	- waiting to lock <0x00000007ae29fc60> (a org.xnio.streams.BufferPipeOutputStream)
	at org.jboss.remoting3.remote.RemoteConnectionChannel.closeMessages(RemoteConnectionChannel.java:560)
	at org.jboss.remoting3.remote.RemoteConnectionChannel.closeAction(RemoteConnectionChannel.java:542)
	at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeAsync(AbstractHandleableCloseable.java:359)
	at org.jboss.remoting3.remote.RemoteConnectionHandler.closeAllChannels(RemoteConnectionHandler.java:423)
	- locked <0x00000007ad9c3030> (a java.util.ArrayDeque)
	at org.jboss.remoting3.remote.RemoteConnectionHandler.handleConnectionClose(RemoteConnectionHandler.java:114)
	at org.jboss.remoting3.remote.RemoteReadListener.handleEvent(RemoteReadListener.java:81)
	- locked <0x00000007ad9c3030> (a java.util.ArrayDeque)
	at org.jboss.remoting3.remote.RemoteReadListener.handleEvent(RemoteReadListener.java:45)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.channels.TranslatingSuspendableChannel.handleReadable(TranslatingSuspendableChannel.java:189)
	at org.xnio.channels.TranslatingSuspendableChannel$1.handleEvent(TranslatingSuspendableChannel.java:103)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.channels.TranslatingSuspendableChannel.handleReadable(TranslatingSuspendableChannel.java:189)
	at org.xnio.ssl.JsseConnectedSslStreamChannel.handleReadable(JsseConnectedSslStreamChannel.java:183)
	at org.xnio.channels.TranslatingSuspendableChannel$1.handleEvent(TranslatingSuspendableChannel.java:103)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.nio.NioHandle.run(NioHandle.java:90)
	at org.xnio.nio.WorkerThread.run(WorkerThread.java:198)


"Remoting "host:MANAGEMENT" task-8" #336 prio=5 os_prio=0 tid=0x00007fbdec01c800 nid=0x4105 waiting for monitor entry [0x00007fbcefd79000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.jboss.remoting3.remote.RemoteConnection$RemoteWriteListener.send(RemoteConnection.java:294)
	- waiting to lock <0x00000007ad9c3030> (a java.util.ArrayDeque)
	at org.jboss.remoting3.remote.RemoteConnection.send(RemoteConnection.java:122)
	at org.jboss.remoting3.remote.OutboundMessage$1.accept(OutboundMessage.java:154)
	at org.xnio.streams.BufferPipeOutputStream.send(BufferPipeOutputStream.java:126)
	at org.xnio.streams.BufferPipeOutputStream.send(BufferPipeOutputStream.java:114)
	at org.xnio.streams.BufferPipeOutputStream.flush(BufferPipeOutputStream.java:143)
	- locked <0x00000007ae29fc60> (a org.xnio.streams.BufferPipeOutputStream)
	at org.xnio.streams.BufferPipeOutputStream.flush(BufferPipeOutputStream.java:135)
	at org.jboss.remoting3.remote.OutboundMessage.flush(OutboundMessage.java:277)
	at java.io.DataOutputStream.flush(DataOutputStream.java:123)
	at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
	at org.jboss.remotingjmx.DelegatingRemotingConnectorServer.writeVersionHeader(DelegatingRemotingConnectorServer.java:208)
	at org.jboss.remotingjmx.DelegatingRemotingConnectorServer.access$200(DelegatingRemotingConnectorServer.java:60)
	at org.jboss.remotingjmx.DelegatingRemotingConnectorServer$ChannelOpenListener.channelOpened(DelegatingRemotingConnectorServer.java:288)
	at org.jboss.remoting3.spi.SpiUtils$ServiceOpenTask.run(SpiUtils.java:126)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)



This was reported upstream in WFLY-4113, and resolved by upgrading to remoting 4.0.7 which incorporated the commit https://github.com/jboss-remoting/jboss-remoting/commit/afd95bbf3b27af677fc74908515972b4ddb4452a.

To prevent this deadlock, we need to backport the commit to remoting 3.3.x and pull in a fixed version.
Comment 1 James Livingston 2015-07-30 00:28:48 EDT
And never mind, this is already fixed in 6.4.2 due to the remoting 3.3.5 upgrade, but I didn't manage to find bug 1215714.

*** This bug has been marked as a duplicate of bug 1215714 ***

Note You need to log in before you can comment on or make changes to this bug.