Hide Forgot
Customer impact: In HA scenario after the failover/failback the clients are not able to continue in sending/receiving messages, because the server stops responding them. There are no lost messages. This is not regression against previous EAP 6 version. Description of problem: In HA scenario with replicated journal I can see the exception [1]. After that the server stops responding to clients and warnings [2] are printed to the server log. The exception is thrown from the FileWrapperJournal.appendRollbackRecord. This method is claimed as unsupported. The same issue was hit in ActiveMQ Artemis and it was already fixed. See https://issues.apache.org/jira/browse/ARTEMIS-422. [1] 07:52:49,236 WARN [org.hornetq.core.server] (Old I/O client worker ([id: 0x2ce9cc57, /127.0.0.1:35838 => localhost/127.0.0.1:5445]), runningThread=Thread-1 (HornetQ-client-global-threads-45175226)) HQ222086: er ror handling packet ReplicationCommitMessage[type=96, channel=2, journalID=1, txAction='rollback'] for replication: HornetQException[errorType=UNSUPPORTED_PACKET message=null] at org.hornetq.core.journal.impl.FileWrapperJournal.appendRollbackRecord(FileWrapperJournal.java:209) [hornetq-journal-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.journal.impl.JournalBase.appendRollbackRecord(JournalBase.java:151) [hornetq-journal-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.journal.impl.FileWrapperJournal.appendRollbackRecord(FileWrapperJournal.java:35) [hornetq-journal-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.replication.ReplicationEndpoint.handleCommitRollback(ReplicationEndpoint.java:708) [hornetq-server-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.replication.ReplicationEndpoint.handlePacket(ReplicationEndpoint.java:192) [hornetq-server-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:652) [hornetq-core-client-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:615) [hornetq-core-client-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:574) [hornetq-core-client-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1695) [hornetq-core-client-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.remoting.impl.netty.HornetQChannelHandler.messageReceived(HornetQChannelHandler.java:72) [hornetq-core-client-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:281) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.decode(HornetQFrameDecoder2.java:169) [hornetq-core-client-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.messageReceived(HornetQFrameDecoder2.java:134) [hornetq-core-client-2.3.25.SP13-redhat-1.jar:2.3.25.SP13-redhat-1] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.socket.oio.OioWorker.process(OioWorker.java:71) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.socket.oio.AbstractOioWorker.run(AbstractOioWorker.java:73) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:51) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:175) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_101] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_101] [2] 07:53:19,456 WARN [org.hornetq.core.server] (Old I/O server worker (parentId: -1612310000, [id: 0x9fe61a10, /127.0.0.1:5445]), runningThread=Thread-4 (HornetQ-remoting-threads-HornetQServerImpl::serverUUID=86cf2f36-68f5-11e6-83c6-138d94a4e5f9-1300803308-1874445692)) HQ222059: Duplicate message detected - message will not be routed. Message information: ServerMessage[messageID=2147485193,userID=e4b2383b-68f5-11e6-9cda-fda33d19066a,priority=4, bodySize=41279,expiration=0, durable=true, address=jms.queue.testQueue,properties=TypedProperties[__HQ_CID=99ae459d-68f5-11e6-9cda-cb49fd50e973,count=870,counter=870,_HQ_DUPL_ID=516fc68d-a0ab-4a1c-b4ea-dcceb3717520]]@577021911 Actual results: In HA scenario after the failover/failback the clients are not able to continue in sending/receiving messages. Expected results: In HA scenario no messages are lost and clients are able to continue in sending/receiving messages even when one server is killed.