Bug 997890 - consumer.receive() thows IndexOutOfBoundException in standalone JMS client
consumer.receive() thows IndexOutOfBoundException in standalone JMS client
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: HornetQ (Show other bugs)
Unspecified Unspecified
unspecified Severity high
: ER9
: EAP 6.3.0
Assigned To: Clebert Suconic
Miroslav Novak
Russell Dickenson
Depends On:
  Show dependency treegraph
Reported: 2013-08-16 07:55 EDT by Miroslav Novak
Modified: 2014-10-25 08:01 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2014-07-07 03:37:03 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Miroslav Novak 2013-08-16 07:55:20 EDT
Description of problem:

IndexOutOfBoundException crashed standalone JMS client in one of failover tests for replicated journal.
This can have negative impact to customer's production environment.  

Test scenario:

1. Start two EAP 6.1.1.ER6 in colocated topology with replicated journal
2. Start publishers and one subscriber sending/receiving messages to/from topic
3. Kill first server
4. Wait for failover to occur
5. Start first server again
6. Wait for failback
7. Stop publishers and wait for subcriber to read all messages

In step 7. IndexOutOfBoundException was thrown to subscriber [1]:
03:39:04,654 Thread-253 ERROR [org.jboss.qa.hornetq.apps.clients.SubscriberTransAck:138] Exception was thrown during receiving messages:
java.lang.IndexOutOfBoundsException: Not enough readable bytes - Need 409600, maximum is 204795
	at org.jboss.netty.buffer.AbstractChannelBuffer.checkReadableBytes(AbstractChannelBuffer.java:661)
	at org.jboss.netty.buffer.AbstractChannelBuffer.readBytes(AbstractChannelBuffer.java:338)
	at org.jboss.netty.buffer.AbstractChannelBuffer.readBytes(AbstractChannelBuffer.java:344)
	at org.hornetq.core.buffers.impl.ChannelBufferWrapper.readSimpleStringInternal(ChannelBufferWrapper.java:71)
	at org.hornetq.core.buffers.impl.ChannelBufferWrapper.readNullableSimpleString(ChannelBufferWrapper.java:49)
	at org.hornetq.jms.client.HornetQTextMessage.doBeforeReceive(HornetQTextMessage.java:141)
	at org.hornetq.jms.client.HornetQMessageConsumer.getMessage(HornetQMessageConsumer.java:232)
	at org.hornetq.jms.client.HornetQMessageConsumer.receive(HornetQMessageConsumer.java:129)
	at org.jboss.qa.hornetq.apps.clients.SubscriberTransAck.receiveMessage(SubscriberTransAck.java:391)
	at org.jboss.qa.hornetq.apps.clients.SubscriberTransAck.run(SubscriberTransAck.java:110)

Client is receiving mix of small and large messages of various types (Stream, Object, Byte, Text and Map messages). Exception was thrown after succesful failback of client from backup to live during receiving large TextMessage with body of size 400KB.

This issue appeared only once. There is no reproducer. 

[1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-HornetQ/job/eap-60-hornetq-ha-failover-colocated-replicated-journal-rhel-ipv4/IOTYPE=aio,jdk=java16_default,label=messaging-10/lastCompletedBuild/testReport/org.jboss.qa.hornetq.test.failover/ReplicatedColocatedClusterFailoverTestCase/testFailbackTransAckTopic/
Comment 1 Clebert Suconic 2013-08-20 11:55:11 EDT
It seems you just need a retry mechanism on the client for large messages...

I would mark this as a wont-fix or works for me.. I have no substantial information to work here
Comment 3 Miroslav Novak 2013-09-06 04:15:34 EDT
Hi Clebert,

thanks for your feedback. 

There is IndexOutOfBoundException thrown from consumer.receive() method which is unexpected and unchecked. Consumer can retry receive only if JMSException is thrown. 

In this case buffer for receiving large message was not allocated properly.

From the log it appears that IndexOutOfBoundException is not related to failover/failback scenario. It was thrown when no kind of server failure was in progress. Subscriber is using transacted session and commits every 100th message. 
I could not replicate the issue but I suspect that certain sequence of large and small messages can lead to this exception.

We should not close this bz.


Adding updated link to "raw" log from jenkins job:
Comment 8 Miroslav Novak 2014-07-07 03:37:03 EDT
Closing as I don't see IndexOutOfBoundException any more.

Note You need to log in before you can comment on or make changes to this bug.