Hide Forgot
Created attachment 479081 [details] Issue reproducer Description of problem: Topology: One replicated queue set on the source broker. The replicated queue is set to replicate enqueue and dequeue events to backup queues on several backup brokers (load balancing) Dequeue event on replicated queue is not correctly replicated to backup queues. Dequeue event on replicated queue is correctly replicated to backup queues only in the case when all of the backup queues contain the same number of messages. Version-Release number of selected component (if applicable): python-qpid-0.7.946106-15.el5 qpid-cpp-client-0.7.946106-28.el5 qpid-cpp-client-devel-0.7.946106-28.el5 qpid-cpp-client-devel-docs-0.7.946106-28.el5 qpid-cpp-client-ssl-0.7.946106-28.el5 qpid-cpp-mrg-debuginfo-0.7.946106-28.el5 qpid-cpp-server-0.7.946106-28.el5 qpid-cpp-server-cluster-0.7.946106-28.el5 qpid-cpp-server-devel-0.7.946106-28.el5 qpid-cpp-server-ssl-0.7.946106-28.el5 qpid-cpp-server-store-0.7.946106-28.el5 qpid-cpp-server-xml-0.7.946106-28.el5 qpid-java-client-0.7.946106-15.el5 qpid-java-common-0.7.946106-15.el5 qpid-java-example-0.7.946106-15.el5 qpid-tools-0.7.946106-12.el5 How reproducible: 100% Steps to Reproduce: 1. set one replicated queue on source the broker 2. set the replicated queue to replicate enqueue and dequeue events to backup queues on several backup brokers 2. enqueue several messages on replicated queue 3. dequeue messages on replicated queue 5. list the content of backup queues or simply use attached issue reproducer, see additional info Actual results: Dequeue event is replicated only in the case that all of the backup queues contains the same number of messages Expected results: Dequeue event on replicated queue is correctly replicated to all of the backup queues Additional info: Issue reproducer takes exactly one argument - number of messages to enqueue content of backup queues is displayed # ./replication.sh 2 // messages on backup queues are not dequeued # ./replication.sh 3 // CORRECT # ./replication.sh 4 // messages on backup queues are not dequeued # ./replication.sh 2 Running replication test between localhost:54313 -> localhost:49930, localhost:42700, localhost:52701 reading messages from broker localhost:54313 Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'a8bd21af-5ec2-5c4d-95b3-b825452248d8:0'}, content='Backuped msg from A') Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'a8bd21af-5ec2-5c4d-95b3-b825452248d8:1'}, content='Backuped msg from A') reading messages from broker localhost:49930 Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'a8bd21af-5ec2-5c4d-95b3-b825452248d8:0'}, content='Backuped msg from A') reading messages from broker localhost:42700 Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'a8bd21af-5ec2-5c4d-95b3-b825452248d8:1'}, content='Backuped msg from A') reading messages from broker localhost:52701 # ./replication.sh 3 Running replication test between localhost:52081 -> localhost:42890, localhost:58247, localhost:55445 reading messages from broker localhost:52081 Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'907968c4-5640-1640-a2e2-28204c0dbd95:0'}, content='Backuped msg from A') Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'907968c4-5640-1640-a2e2-28204c0dbd95:1'}, content='Backuped msg from A') Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'907968c4-5640-1640-a2e2-28204c0dbd95:2'}, content='Backuped msg from A') reading messages from broker localhost:42890 reading messages from broker localhost:58247 reading messages from broker localhost:55445 # ./replication.sh 4 Running replication test between localhost:51132 -> localhost:48402, localhost:34184, localhost:51132 reading messages from broker localhost:51132 Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:0'}, content='Backuped msg from A') Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:1'}, content='Backuped msg from A') Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:2'}, content='Backuped msg from A') Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:3'}, content='Backuped msg from A') reading messages from broker localhost:48402 Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:0'}, content='Backuped msg from A') Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:3'}, content='Backuped msg from A') reading messages from broker localhost:34184 Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:1'}, content='Backuped msg from A') reading messages from broker localhost:55924 Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:2'}, content='Backuped msg from A')
The issue here is that the events from the replication queue are delivered to competing consumers, a situation the current replication protocol cannot handle.