Bug 743099

Summary: mgsGroups: messages can be dequeued in different order than they were produced using one consumer
Product: Red Hat Enterprise MRG Reporter: Petr Matousek <pematous>
Component: qpid-cppAssignee: Ken Giusti <kgiusti>
Status: CLOSED NOTABUG QA Contact: MRG Quality Engineering <mrgqe-bugs>
Severity: unspecified Docs Contact:
Priority: medium    
Version: Development   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-01-13 16:50:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
test reproducer none

Description Petr Matousek 2011-10-03 20:16:33 UTC 
Description of problem:

N+1'th message in the group can be consumed by the same consumer even if the N'th message has not been acknowledged by the consumer.

Version-Release number of selected component (if applicable):
qpidd (qpidc) version 0.13
snv branch QPID-3346 (revision 1177617)

How reproducible:
100%

Steps to Reproduce:
1. run the attached test reproducer
  
Actual results:
other messages can be processed before the previously queued message is acknowledged even if they belong to the same group

Expected results:
N+1'th message in the group would be blocked until the N'th message has been accepted (or rejected) by a client.

Additional info:

there is just one consumer on already existing queue with qpid.group_header_key set, and the following messages queued:
message1 (group header set to "A")
message2 (group header set to "A")

a.) consumer consumed the first available message, the message
was not acknowledged:
  message1 was received
b.) consumer consumed all available messages:
  message2 was received
c.) session is closed
d.) now another session is established and the new consumer is
consuming all available messages:
  message1 is redelivered

This means that the second message in the queue was processed before
the first message was acknowledged. No message shall be received in step b. 

This is in conflict with the point 3 of the example described in the following document: https://docspace.corp.redhat.com/docs/DOC-69575
(Until the first message from "A" is accepted or rejected, all remaining messages in group "A" are blocked)

The behaviour is the same even if the receiver capacity is set to 1.
Comment 1 Petr Matousek 2011-10-03 20:19:04 UTC 
Created attachment 526124 [details]
test reproducer
Comment 2 Ken Giusti 2011-11-02 19:13:30 UTC 
Hi Petr,

The example above does violate the desired ordering rules, but it really isn't a bug in the broker.  The broker can only order the delivery of messages to consumers.  If the consumer acknowledges messages too early, or out of order, or releases some messages and acknowledges others, then strict sequenced consumption cannot be guaranteed.

This feature is a little weird in that it -requires- the consumers to follow certain consumption rules, but the broker really can't prevent them from violating them (and, in fact, some applications may need to behave this way).

I've updated the upstream documentation to be a bit clearer regarding how consumers need to behave - take a look in qpid/doc/book and try building the C++ broker guide.  There's a section on message groups in that document that is the "official" documentation - take a look at let me know if it needs more work.

Latest changes to that document were committed @ r1196764