Bug 786875 - msgGroups: releasing a message may cause consumption order discrepancies
Summary: msgGroups: releasing a message may cause consumption order discrepancies
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp
Version: Development
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: 2.1.2
: ---
Assignee: Ken Giusti
QA Contact: Petr Matousek
URL:
Whiteboard:
Depends On: 734115
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-02-02 16:14 UTC by Petr Matousek
Modified: 2012-12-07 17:46 UTC (History)
2 users (show)

Fixed In Version: qpid-cpp-mrg-0.14-5.el5, qpid-cpp-0.14-6.el6
Doc Type: Bug Fix
Doc Text:
Cause Fetching a message from a message group queue, then rolling back the fetch. Consequence The next fetch will incorrectly skip over the rolled back message. Fix A rollback will cause the consumer's position in the queue to reset. Result The next fetch will re-fetch the rolled back message if it has not already been consumed by another consumer.
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)
test_reproducer released message (1.87 KB, application/x-python)
2012-02-02 16:15 UTC, Petr Matousek
no flags Details
test_reproducer transactional (1.26 KB, application/x-python)
2012-02-02 16:16 UTC, Petr Matousek
no flags Details

Description Petr Matousek 2012-02-02 16:14:22 UTC
Description of problem:

When a consumer releases a message, the consumer gets in the next fetch() call  the next oldest message, either from the free group or from one of the groups it already owns. But calling the fetch() method once again causes the released message to be redelivered. This actually causes a failure in the message consumption order. 

This is also in conflict with MRG/M UG: "A subscriber might choose not to take ownership of an acquired message. In this case, the message is released. This allows the broker to re-deliver the message to any other available subscriber - this can include the subscriber that just released the message.

NOTE: the same behaviour can be seen using transaction as well. I suppose it is the same issue, but this needs to be tested as well.

in example:

queue: HEAD-> A0,B1,A2,A3

receiver_1 releases A0, then gets B1... then gets A0 on its next fetch.

OR

receiver_1 consumes A0 and do a rollback(), then gets B1... then gets A0 on its next fetch. 

Version-Release number of selected component (if applicable):
python-qpid-0.14-1.el5
python-qpid-qmf-0.14-2.el5
qpid-cpp-client-0.14-3.el5
qpid-cpp-client-devel-0.14-3.el5
qpid-cpp-client-devel-docs-0.14-3.el5
qpid-cpp-client-ssl-0.14-3.el5
qpid-cpp-server-0.14-3.el5
qpid-cpp-server-cluster-0.14-3.el5
qpid-cpp-server-devel-0.14-3.el5
qpid-cpp-server-ssl-0.14-3.el5
qpid-cpp-server-store-0.14-3.el5
qpid-cpp-server-xml-0.14-3.el5
qpid-java-client-0.14-1.el5
qpid-java-common-0.14-1.el5
qpid-java-example-0.14-1.el5
qpid-qmf-0.14-2.el5
qpid-qmf-devel-0.14-2.el5
qpid-tests-0.14-1.el5
qpid-tools-0.14-1.el5


How reproducible:
100%

Steps to Reproduce:
1. run the attached reproducer for "released messages" without an argument
(this demonstrates behaviour on common queue)
2. run the attached reproducer for "released messages" with an argument specified (this demonstrates behaviour on message group queue)
3. check results

AND

1. run the attached reproducer for "transactions" without an argument
(this demonstrates behaviour on common queue)
2. run the attached reproducer for "transactions" with an argument specified (this demonstrates behaviour on message group queue)
3. check results
  
Actual results:
releasing a message cause consumption order discrepancies

Expected results:
releasing a message do not cause consumption order discrepancies
released message can be immediately re-fetched by the same consumer

Additional info:

Reproducers output:

# ./test_released.py 
queueing message: Message('Message0')
queueing message: Message('Message1')
queueing message: Message('Message2')
queueing message: Message('Message3')

receiver_1: Message(properties={'x-amqp-0-10.routing-key': u'example'}, content='Message0')
receiver_1: Message(redelivered=True, properties={'x-amqp-0-10.routing-key': u'example'}, content='Message0')
receiver_1: Message(properties={'x-amqp-0-10.routing-key': u'example'}, content='Message1')
receiver_2: Message(properties={'x-amqp-0-10.routing-key': u'example'}, content='Message2')
receiver_2: Message(properties={'x-amqp-0-10.routing-key': u'example'}, content='Message3')

# ./test_released.py message_groups
queueing message: Message('Message0')
queueing message: Message('Message1')
queueing message: Message('Message2')
queueing message: Message('Message3')

receiver_1: Message(properties={u'GROUP-KEY': u'A', 'x-amqp-0-10.routing-key': u'example'}, content='Message0')
receiver_1: Message(properties={u'GROUP-KEY': u'B', 'x-amqp-0-10.routing-key': u'example'}, content='Message1')
receiver_1: Message(redelivered=True, properties={u'GROUP-KEY': u'A', 'x-amqp-0-10.routing-key': u'example'}, content='Message0')
receiver_2: Message(properties={u'GROUP-KEY': u'A', 'x-amqp-0-10.routing-key': u'example'}, content='Message2')
receiver_2: Message(properties={u'GROUP-KEY': u'A', 'x-amqp-0-10.routing-key': u'example'}, content='Message3')

# ./test_transactions.py 
queueing message: Message('None0')
queueing message: Message('None1')
queueing message: Message('None2')
queueing message: Message('None3')

receiver_1: Message(properties={'x-amqp-0-10.routing-key': u'example'}, content='None0')
receiver_1: Message(redelivered=True, properties={'x-amqp-0-10.routing-key': u'example'}, content='None0')
receiver_1: Message(properties={'x-amqp-0-10.routing-key': u'example'}, content='None1')
receiver_1: Message(properties={'x-amqp-0-10.routing-key': u'example'}, content='None2')
receiver_1: Message(properties={'x-amqp-0-10.routing-key': u'example'}, content='None3')

# ./test_transactions.py message_groups
queueing message: Message(properties={'GROUP-KEY': 'A'}, content='Message-A0')
queueing message: Message(properties={'GROUP-KEY': 'B'}, content='Message-B1')
queueing message: Message(properties={'GROUP-KEY': 'A'}, content='Message-A2')
queueing message: Message(properties={'GROUP-KEY': 'A'}, content='Message-A3')

receiver_1: Message(properties={u'GROUP-KEY': u'A', 'x-amqp-0-10.routing-key': u'example'}, content='Message-A0')
receiver_1: Message(properties={u'GROUP-KEY': u'B', 'x-amqp-0-10.routing-key': u'example'}, content='Message-B1')
receiver_1: Message(redelivered=True, properties={u'GROUP-KEY': u'A', 'x-amqp-0-10.routing-key': u'example'}, content='Message-A0')
receiver_1: Message(properties={u'GROUP-KEY': u'A', 'x-amqp-0-10.routing-key': u'example'}, content='Message-A2')
receiver_1: Message(properties={u'GROUP-KEY': u'A', 'x-amqp-0-10.routing-key': u'example'}, content='Message-A3')

Comment 1 Petr Matousek 2012-02-02 16:15:31 UTC
Created attachment 559077 [details]
test_reproducer released message

Comment 2 Petr Matousek 2012-02-02 16:16:15 UTC
Created attachment 559078 [details]
test_reproducer transactional

Comment 4 Petr Matousek 2012-02-06 17:04:08 UTC
Verified on RHEL5 x86_64, i386 - qpid-cpp-mrg-0.14-5 
Packages for RHEL6 not yet available.

Comment 5 Petr Matousek 2012-02-07 10:52:07 UTC
Retested on RHEL6 with qpid-cpp-0.14-5.el6 installed, this issue is *NOT* fixed

Comment 6 Petr Matousek 2012-03-01 18:53:05 UTC
This issue has been fixed.

Verified on RHEL5.7 and RHEL6.2, architectures: x86_64, i386 

packages installed:
qpid-cpp-0.14-7.el6
qpid-cpp-mrg-0.14-7.el5

Comment 7 Ken Giusti 2012-03-07 21:47:13 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause
    Fetching a message from a message group queue, then rolling back the fetch.
Consequence
    The next fetch will incorrectly skip over the rolled back message.
Fix
    A rollback will cause the consumer's position in the queue to reset.
Result
    The next fetch will re-fetch the rolled back message if it has not already been consumed by another consumer.


Note You need to log in before you can comment on or make changes to this bug.