Red Hat Bugzilla – Bug 997890
consumer.receive() thows IndexOutOfBoundException in standalone JMS client
Last modified: 2014-10-25 08:01:33 EDT
Description of problem: IndexOutOfBoundException crashed standalone JMS client in one of failover tests for replicated journal. This can have negative impact to customer's production environment. Test scenario: 1. Start two EAP 6.1.1.ER6 in colocated topology with replicated journal 2. Start publishers and one subscriber sending/receiving messages to/from topic 3. Kill first server 4. Wait for failover to occur 5. Start first server again 6. Wait for failback 7. Stop publishers and wait for subcriber to read all messages Result: In step 7. IndexOutOfBoundException was thrown to subscriber [1]: 03:39:04,654 Thread-253 ERROR [org.jboss.qa.hornetq.apps.clients.SubscriberTransAck:138] Exception was thrown during receiving messages: java.lang.IndexOutOfBoundsException: Not enough readable bytes - Need 409600, maximum is 204795 at org.jboss.netty.buffer.AbstractChannelBuffer.checkReadableBytes(AbstractChannelBuffer.java:661) at org.jboss.netty.buffer.AbstractChannelBuffer.readBytes(AbstractChannelBuffer.java:338) at org.jboss.netty.buffer.AbstractChannelBuffer.readBytes(AbstractChannelBuffer.java:344) at org.hornetq.core.buffers.impl.ChannelBufferWrapper.readSimpleStringInternal(ChannelBufferWrapper.java:71) at org.hornetq.core.buffers.impl.ChannelBufferWrapper.readNullableSimpleString(ChannelBufferWrapper.java:49) at org.hornetq.jms.client.HornetQTextMessage.doBeforeReceive(HornetQTextMessage.java:141) at org.hornetq.jms.client.HornetQMessageConsumer.getMessage(HornetQMessageConsumer.java:232) at org.hornetq.jms.client.HornetQMessageConsumer.receive(HornetQMessageConsumer.java:129) at org.jboss.qa.hornetq.apps.clients.SubscriberTransAck.receiveMessage(SubscriberTransAck.java:391) at org.jboss.qa.hornetq.apps.clients.SubscriberTransAck.run(SubscriberTransAck.java:110) Client is receiving mix of small and large messages of various types (Stream, Object, Byte, Text and Map messages). Exception was thrown after succesful failback of client from backup to live during receiving large TextMessage with body of size 400KB. This issue appeared only once. There is no reproducer. [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-HornetQ/job/eap-60-hornetq-ha-failover-colocated-replicated-journal-rhel-ipv4/IOTYPE=aio,jdk=java16_default,label=messaging-10/lastCompletedBuild/testReport/org.jboss.qa.hornetq.test.failover/ReplicatedColocatedClusterFailoverTestCase/testFailbackTransAckTopic/
It seems you just need a retry mechanism on the client for large messages... I would mark this as a wont-fix or works for me.. I have no substantial information to work here
Hi Clebert, thanks for your feedback. There is IndexOutOfBoundException thrown from consumer.receive() method which is unexpected and unchecked. Consumer can retry receive only if JMSException is thrown. In this case buffer for receiving large message was not allocated properly. From the log it appears that IndexOutOfBoundException is not related to failover/failback scenario. It was thrown when no kind of server failure was in progress. Subscriber is using transacted session and commits every 100th message. I could not replicate the issue but I suspect that certain sequence of large and small messages can lead to this exception. We should not close this bz. Thanks, Mirek Adding updated link to "raw" log from jenkins job: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-HornetQ/job/eap-60-hornetq-ha-failover-colocated-replicated-journal-rhel-ipv4/IOTYPE=aio,jdk=java16_default,label=messaging-10/33/consoleText
Updating url: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-HornetQ/job/eap-60-hornetq-ha-failover-colocated-replicated-journal-rhel-ipv4/35/IOTYPE=nio,jdk=java16_default,label=messaging-10/testReport/org.jboss.qa.hornetq.test.failover/ReplicatedColocatedClusterFailoverTestCase/testFailbackTransAckTopic/
Closing as I don't see IndexOutOfBoundException any more.