Created attachment 1148107 [details]
remote-async-void-slsb-dbuffer-leak.zip

Description of problem:
JBoss Remoting enabled direct buffer from 3.3.7.Final, which is included in EAP 6.4.6 (by BZ-1288664/REM3-208).  This causes an OOME in direct buffer when a remote client calls an asynchronous method which has return type void.

~~~
Apr 18, 2016 2:22:26 PM org.xnio.ChannelListeners invokeChannelListener
ERROR: A channel event listener threw an exception
java.lang.OutOfMemoryError: Direct buffer memory
        at java.nio.Bits.reserveMemory(Bits.java:693)
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
        at org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:60)
        at org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:58)
        at org.xnio.ByteBufferSlicePool.allocate(ByteBufferSlicePool.java:86)
        at org.jboss.remoting3.remote.RemoteConnection.allocate(RemoteConnection.java:75)
        at org.jboss.remoting3.remote.RemoteConnectionChannel.allocate(RemoteConnectionChannel.java:581)
        at org.jboss.remoting3.remote.InboundMessage.allocate(InboundMessage.java:190)
        at org.jboss.remoting3.remote.InboundMessage.doSendCloseMessage(InboundMessage.java:104)
        at org.jboss.remoting3.remote.InboundMessage.handleIncoming(InboundMessage.java:243)
        at org.jboss.remoting3.remote.RemoteConnectionChannel.handleMessageData(RemoteConnectionChannel.java:474)
        at org.jboss.remoting3.remote.RemoteReadListener.handleEvent(RemoteReadListener.java:291)
        at org.jboss.remoting3.remote.RemoteReadListener.handleEvent(RemoteReadListener.java:46)
        at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
        at org.xnio.channels.TranslatingSuspendableChannel.handleReadable(TranslatingSuspendableChannel.java:189)
        at org.xnio.channels.TranslatingSuspendableChannel$1.handleEvent(TranslatingSuspendableChannel.java:103)
        at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
        at org.xnio.nio.NioHandle.run(NioHandle.java:90)
        at org.xnio.nio.WorkerThread.run(WorkerThread.java:198)
~~~


Version-Release number of selected component (if applicable):
EJB client (jboss-client.jar) of EAP 6.4.6


How reproducible:
Always.


Steps to Reproduce:
1. Unzip the reproducer and build an EJB and its client.
~~~
cd remote-async-void-slsb-dbuffer-leak
vi src/test/java/TestMainDBufferLeak.java #The default is against localhost without authentication.
mvn package
~~~
2. Deploy target/remote-async-void-slsb-dbuffer-leak-1.0.0.jar to a standalone server
3. Run the test client
~~~
java -cp $EAP646_HOME/bin/client/jboss-client.jar:target/test-classes:target/remote-async-void-slsb-dbuffer-leak-1.0.0.jar \
     -Xms100m -Xmx100m TestMainDBufferLeak
~~~


Actual results:
"java.lang.OutOfMemoryError: Direct buffer memory" as in the description.


Expected results:
No memory leaks.


Additional info:
Add command line option "-Djboss.remoting.pooled-buffers=false" for the client as a workaround.