Bug 606357
Summary: | Support for dropping messages first by priority, then by age within priority | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise MRG | Reporter: | Andy Goldstein <agoldste> | ||||
Component: | qpid-cpp | Assignee: | Gordon Sim <gsim> | ||||
Status: | CLOSED ERRATA | QA Contact: | Petra Svobodová <psvobodo> | ||||
Severity: | medium | Docs Contact: | |||||
Priority: | medium | ||||||
Version: | Development | CC: | agoldste, freznice, iboverma, jeder, jneedle, jross, jsarenik, mhusnain, psvobodo | ||||
Target Milestone: | 2.0 | Keywords: | FutureFeature | ||||
Target Release: | --- | ||||||
Hardware: | All | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | qpid-cpp-0.9.1079953 | Doc Type: | Enhancement | ||||
Doc Text: |
Cause:
A queue can be configured to remain within a given size by removing the oldest messages when needed to make space for newer messages. However this took no account of message priority.
Consequence:
In some cases messages that were more valuable would be removed while there were less valuable messages left on the queue.
Change:
If such a queue has also been configured to recognise message priority (i.e. qpid.priorities is specified) then it will discard lower priority messages first.
Result:
The application can configure the queue (and mark the messages) such that less useful/less important messages are discarded first.
Release Note Entry:
When the Messaging component runs out of space, it must remove older messages to make space for new incoming messages. The messages to be deleted first are selected using an algorithm that takes into account both the priority and the age of the message. This algorithm allows the oldest of the low priority messages to be considered expendable while the high priority messages are preserved.
|
Story Points: | --- | ||||
Clone Of: | Environment: | ||||||
Last Closed: | 2011-06-23 15:47:41 UTC | Type: | --- | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Description
Andy Goldstein
2010-06-21 13:30:56 UTC
For ring queues configured to recognise different priority levels (qpid.priorities is set), the lowest priority messages will be deleted first when the maximum size is reached. Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: Cause: A queue can be configured to remain within a given size by removing the oldest messages when needed to make space for newer messages. However this took no account of message priority. Consequence: In some cases messages that were more valuable would be removed while there were less valuable messages left on the queue. Change: If such a queue has also been configured to recognise message priority (i.e. qpid.priorities is specified) then it will discard lower priority messages first. Result: The application can configure the queue (and mark the messages) such that less useful/less important messages are discarded first. Created attachment 499601 [details] The behavior reproducer Queues can have defined their priority levels. A queue with declared "limit-policy" option value "ring" after its overfilling (if its limit is exceeded) removes a message with the lowest priority value or the latest message; this is expected behaviour. Queues with "limit-policy" value "ring-strict" have the same behaviour. Is it right? Should the "ring-strict" queues remove messages before their accepting? This seems to be in contrary to documentation Messaging_User_Guide chapter 3 (search for 'Similar to the ring policy, but will not remove messages that have not yet been accepted') Could you possibly double-check whether described ring-strict behavior is correct, please? Raising NEEDINFO. The testing steps: service qpidd restart ./bug606357_prior_mod &>log_ring service qpidd restart ./bug606357_prior_mod ring-strict &>log_ring_strict The difference with ring-strict is that if the message to be removed to make space for a new message has been delivered to a consumer but not yet acknowledged (and thus dequeued), it should not be replaced and the enqueue should fail. Your reproducer appears to use drain (in browse), so I don't see how it could trigger the ring-strict specific behaviour. Am I missing something? One way to see the ring-strict behaviour is to use the qpid-receive test program (currently only in the svn tree I think?) with options designed to prevent immediate acknowledgement: E.g. qpid-config add queue my-queue --limit-policy ring-strict --max-queue-count 5 for i in `seq 1 10`; do spout --content "msg-$i" my-queue; done then start (and leave running): qpid-receive --ack-frequency 10 --messages 10 --address my-queue -f and in another window: spout my-queue That last spout will fail for the ring-strict policy. However if you kill the qpid-receive program and try again it will work. Gordon, thank you for explanation; the "ring-strict" queues behave as you write. The issue is clearer for me now. Messages in queues are dropped first by priority, then by age; this is expected behaviour. Tested packages: [root@dhcp-26-233 ~]# rpm -qa | grep -E '(qpid|qmf|sesame)' | sort python-qpid-0.10-1.el5 python-qpid-qmf-0.10-6.el5 qpid-cpp-client-0.10-7.el5 qpid-cpp-client-devel-0.10-7.el5 qpid-cpp-client-devel-docs-0.10-7.el5 qpid-cpp-client-ssl-0.10-7.el5 qpid-cpp-mrg-debuginfo-0.10-4.el5 qpid-cpp-server-0.10-7.el5 qpid-cpp-server-cluster-0.10-7.el5 qpid-cpp-server-devel-0.10-7.el5 qpid-cpp-server-ssl-0.10-7.el5 qpid-cpp-server-store-0.10-7.el5 qpid-cpp-server-xml-0.10-7.el5 qpid-java-client-0.10-6.el5 qpid-java-common-0.10-6.el5 qpid-java-example-0.10-6.el5 qpid-qmf-0.10-6.el5 qpid-qmf-devel-0.10-6.el5 qpid-tools-0.10-5.el5 rh-qpid-cpp-tests-0.10-7.el5 sesame-0.10-1.el5 verified on RHEL 5.6 / 6.1rc i[36]86 / x86_64 --> VERIFIED Technical note updated. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -8,4 +8,8 @@ If such a queue has also been configured to recognise message priority (i.e. qpid.priorities is specified) then it will discard lower priority messages first. Result: -The application can configure the queue (and mark the messages) such that less useful/less important messages are discarded first.+The application can configure the queue (and mark the messages) such that less useful/less important messages are discarded first. + +Release Note Entry: + +When the Messaging component runs out of space, it must remove older messages to make space for new incoming messages. The messages to be deleted first are selected using an algorithm that takes into account both the priority and the age of the message. This algorithm allows the oldest of the low priority messages to be considered expendable while the high priority messages are preserved. Technical note can be viewed in the release notes for 2.0 at the documentation stage here: http://documentation-stage.bne.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/2.0/html-single/MRG_Release_Notes/index.html#tabl-MRG_Release_Notes-RHM_Update_Notes-RHM_Update_Notes An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHEA-2011-0890.html |