Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 575177

Summary: Messages set with a TTL expire immediately when sent on qpid queues with LVQ ordering
Product: Red Hat Enterprise MRG Reporter: Mike Cressman <mcressma>
Component: qpid-cppAssignee: Carl Trieloff <cctrieloff>
Status: CLOSED ERRATA QA Contact: Jiri Kolar <jkolar>
Severity: high Docs Contact:
Priority: high    
Version: 1.2CC: jkolar, tao
Target Milestone: 1.3   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
MRG LVQ become unstable when msg TTLs were used and a message expired. For example, once a message with a key 'X' had expired, any subsequent messages sent to the LVQ with key 'X' were expired immediately, and were never made available to the client applications. Once the LVQ began exhibiting this behavior, it continued to act this way until the broker was restarted. With this update, LVQ no longer experiences problems when a message expires when using msg TTLs.
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-10-14 16:04:39 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Program to queue up messages
none
Make file none

Description Mike Cressman 2010-03-19 16:49:01 UTC
Created attachment 401290 [details]
Program to queue up messages

Description of problem:

MRG LVQ becomes unstable when msg TTLs are used and a message expires. Sometimes when messages expire, it seems to upset the LVQ mechanism. Eg.  Once a message with key 'X' has expired, any subsequent messages sent  to the LVQ with key 'X' are expired immediately, and never made  available to client applications. Once an LVQ begins exhibiting this  behaviour, This behaviour continues until the broker is restarted. At this point, another side effect is that the msgDepth and byteDepth properties on an LVQ do not always agree, i.e, when msgDepth is zero, byteDepth is not zero.


Version-Release number of selected component (if applicable):
MRG 1.2

How reproducible:
Always

Steps to Reproduce:

1. Create a queue with queue ordering as lvq or lvq-no-browse
2. Build and run attached producer for 1 minute
3. Stop the producer for a minute to allow messages to expire
4. Use qpid-tool to monitor the queue depth and wait for the message to be dequeued (usually takes about 10 minutes)
5. Once message has expired, restart the producer
6. Watch queue properties with qpid-tool

Actual Results:

The queue depth is always 0, but the byte depth is not. Occasionally, one will get the following error message from the producer:
  Unexpected exception: Attempted size underflow on dequeue  

Expected Results:

The queue depth should not be 0 (at least till the time expired messages are purged). Also, when queue depth is 0, byte depth should be 0.

Additional info:

The root cause seems to be that the lvq object in the Queue that holds mapping from key to messages (Queue::lvq) is not cleared when messages are expired (Queue::purgeExpired), which leads to incorrect accounting when the next message arrives in the queue with the same LVQ key as the message that expired.

Comment 1 Mike Cressman 2010-03-19 16:50:12 UTC
Created attachment 401292 [details]
Make file

Comment 2 Gordon Sim 2010-03-29 11:38:02 UTC
See also https://issues.apache.org/jira/browse/QPID-2454 ; fix committed as r928003.

Comment 3 Jiri Kolar 2010-06-24 15:40:20 UTC
Tested:
on 752581 staging is there
on 946106 not. It has been removed

validated on RHEL5.5/RHEL4  i386 / x86_64  

packages:

# rpm -qa | grep -E '(qpid|openais|rhm)' | sort -u

openais-0.80.6-16.el5_5.1
openais-debuginfo-0.80.6-16.el5_5.1
python-qpid-0.7.946106-1.el5
qpid-cpp-client-0.7.946106-2.el5
qpid-cpp-client-devel-0.7.946106-2.el5
qpid-cpp-client-devel-docs-0.7.946106-2.el5
qpid-cpp-client-ssl-0.7.946106-2.el5
qpid-cpp-mrg-debuginfo-0.7.946106-1.el5
qpid-cpp-server-0.7.946106-2.el5
qpid-cpp-server-cluster-0.7.946106-2.el5
qpid-cpp-server-devel-0.7.946106-2.el5
qpid-cpp-server-ssl-0.7.946106-2.el5
qpid-cpp-server-store-0.7.946106-2.el5
qpid-cpp-server-xml-0.7.946106-2.el5
qpid-java-client-0.7.946106-3.el5
qpid-java-common-0.7.946106-3.el5
qpid-tools-0.7.946106-4.el5
rhm-docs-0.7.946106-1.el5

->VERIFIED

Comment 4 Martin Prpič 2010-10-08 12:55:50 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
MRG LVQ become unstable when msg TTLs were used and a message expired. For example, once a message with a key 'X' had expired, any subsequent messages sent to the LVQ with key 'X' were expired immediately, and were never made available to the client applications. Once the LVQ began exhibiting this behavior, it continued to act this way until the broker was restarted. With this update, LVQ no longer experiences problems when a message expires when using msg TTLs.

Comment 6 errata-xmlrpc 2010-10-14 16:04:39 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2010-0773.html