Bug 677948

Summary: Dequeue event is not replicated
Product: Red Hat Enterprise MRG Reporter: Petr Matousek <pematous>
Component: qpid-cppAssignee: messaging-bugs <messaging-bugs>
Status: NEW --- QA Contact: MRG Quality Engineering <mrgqe-bugs>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 1.3CC: gsim, iboverma
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Issue reproducer none

Description Petr Matousek 2011-02-16 10:40:10 UTC
Created attachment 479081 [details]
Issue reproducer

Description of problem:

Topology:
One replicated queue set on the source broker.
The replicated queue is set to replicate enqueue and dequeue events to backup queues on several backup brokers (load balancing)

Dequeue event on replicated queue is not correctly replicated to backup queues.
Dequeue event on replicated queue is correctly replicated to backup queues only in the case when all of the backup queues contain the same number of messages.

Version-Release number of selected component (if applicable):
python-qpid-0.7.946106-15.el5
qpid-cpp-client-0.7.946106-28.el5
qpid-cpp-client-devel-0.7.946106-28.el5
qpid-cpp-client-devel-docs-0.7.946106-28.el5
qpid-cpp-client-ssl-0.7.946106-28.el5
qpid-cpp-mrg-debuginfo-0.7.946106-28.el5
qpid-cpp-server-0.7.946106-28.el5
qpid-cpp-server-cluster-0.7.946106-28.el5
qpid-cpp-server-devel-0.7.946106-28.el5
qpid-cpp-server-ssl-0.7.946106-28.el5
qpid-cpp-server-store-0.7.946106-28.el5
qpid-cpp-server-xml-0.7.946106-28.el5
qpid-java-client-0.7.946106-15.el5
qpid-java-common-0.7.946106-15.el5
qpid-java-example-0.7.946106-15.el5
qpid-tools-0.7.946106-12.el5

How reproducible:
100%

Steps to Reproduce:
1. set one replicated queue on source the broker
2. set the replicated queue to replicate enqueue and dequeue events to backup queues on several backup brokers
2. enqueue several messages on replicated queue
3. dequeue messages on replicated queue
5. list the content of backup queues

or simply use attached issue reproducer, see additional info

Actual results:
Dequeue event is replicated only in the case that all of the backup queues contains the same number of messages

Expected results:
Dequeue event on replicated queue is correctly replicated to all of the backup queues

Additional info:
Issue reproducer takes exactly one argument - number of messages to enqueue
content of backup queues is displayed

# ./replication.sh 2 // messages on backup queues are not dequeued
# ./replication.sh 3 // CORRECT
# ./replication.sh 4 // messages on backup queues are not dequeued

# ./replication.sh 2
Running replication test between localhost:54313 -> localhost:49930, localhost:42700, localhost:52701
reading messages from broker localhost:54313
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'a8bd21af-5ec2-5c4d-95b3-b825452248d8:0'}, content='Backuped msg from A')
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'a8bd21af-5ec2-5c4d-95b3-b825452248d8:1'}, content='Backuped msg from A')
reading messages from broker localhost:49930
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'a8bd21af-5ec2-5c4d-95b3-b825452248d8:0'}, content='Backuped msg from A')
reading messages from broker localhost:42700
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'a8bd21af-5ec2-5c4d-95b3-b825452248d8:1'}, content='Backuped msg from A')
reading messages from broker localhost:52701

# ./replication.sh 3
Running replication test between localhost:52081 -> localhost:42890, localhost:58247, localhost:55445
reading messages from broker localhost:52081
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'907968c4-5640-1640-a2e2-28204c0dbd95:0'}, content='Backuped msg from A')
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'907968c4-5640-1640-a2e2-28204c0dbd95:1'}, content='Backuped msg from A')
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'907968c4-5640-1640-a2e2-28204c0dbd95:2'}, content='Backuped msg from A')
reading messages from broker localhost:42890
reading messages from broker localhost:58247
reading messages from broker localhost:55445

# ./replication.sh 4
Running replication test between localhost:51132 -> localhost:48402, localhost:34184, localhost:51132
reading messages from broker localhost:51132
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:0'}, content='Backuped msg from A')
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:1'}, content='Backuped msg from A')
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:2'}, content='Backuped msg from A')
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:3'}, content='Backuped msg from A')
reading messages from broker localhost:48402
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:0'}, content='Backuped msg from A')
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:3'}, content='Backuped msg from A')
reading messages from broker localhost:34184
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:1'}, content='Backuped msg from A')
reading messages from broker localhost:55924
Message(properties={'x-amqp-0-10.routing-key': u'queue-a', u'spout-id': u'b76a13e7-df18-a644-ab51-64ffc8056085:2'}, content='Backuped msg from A')

Comment 1 Gordon Sim 2011-02-16 10:51:11 UTC
The issue here is that the events from the replication queue are delivered to competing consumers, a situation the current replication protocol cannot handle.