Bug 786522

Summary: --limit-policy=flow-to-disk is not applicable for non-durable queues
Product: Red Hat Enterprise MRG Reporter: Leonid Zhaldybin <lzhaldyb>
Component: Messaging_Installation_and_Configuration_GuideAssignee: Jared MORGAN <jmorgan>
Status: CLOSED CURRENTRELEASE QA Contact: Eric Sammons <esammons>
Severity: high Docs Contact:
Priority: medium    
Version: 2.1CC: esammons, jross, mmurray, pematous
Target Milestone: 2.3.4   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-01-22 15:27:58 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Leonid Zhaldybin 2012-02-01 17:09:38 UTC
Description of problem:
Messaging User Guide (staging version) states, that limit policy flow_to_disk "can apply to both persistent and non-persistent queues" (see http://documentation-stage.bne.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/2/html/Messaging_User_Guide/chap-Messaging_User_Guide-Queues.html "Controlling queue size"). Nevertheless, if the command 'qpid-config add queue --limit-policy=flow_to_disk' is called without the '--durable' option, then limit policy of the resulting queue is set to 'reject':

[root@lzhaldyb-rhel62x ~]# qpid-config add queue test --max-queue-count=100 --limit-policy=flow-to-disk --flow-stop-count=0 --flow-resume-count=0
[root@lzhaldyb-rhel62x ~]# echo $?
0

qpidd.log:
trace RECV [[::1]:5672-[::1]:48687]: Frame[BEbe; channel=1; {QueueDeclareBody: queue=test; arguments={qpid.flow_resume_count:F8:int64(0),qpid.flow_stop_count:F8:int64(0),qpid.max_count:F8:int64(100),qpid.policy_type:V2:12:str16(flow_to_disk)}; }]
info Queue "test": Policy created: type=reject; maxCount=100; maxSize=104857600

Although qmf shows the limit policy for such a queue as 'flow_to_disk':
[root@lzhaldyb-rhel62x ~]# qpid-config queues
Queue Name                           Attributes
============================================================ 
test                                 --max-queue-count=100 --limit-policy=flow-to-disk

it uses the 'reject' policy instead:
[root@lzhaldyb-rhel62x messaging]# ./spout -c 101 test
2012-02-01 17:45:18 warning Exception received from broker: resource-limit-exceeded: resource-limit-exceeded: resource-limit-exceeded: Policy exceeded on test, policy: size: max=104857600, current=0; count: max=100, current=100; type=reject (qpid/broker/QueuePolicy.cpp:87) [caused by 105 \x00:\x00]
2012-02-01 17:45:18 warning Exception received from broker: resource-limit-exceeded: resource-limit-exceeded: resource-limit-exceeded: Policy exceeded on test, policy: size: max=104857600, current=0; count: max=100, current=100; type=reject (qpid/broker/QueuePolicy.cpp:87) [caused by 1 \x00:\x00]
Failed to connect (reconnect disabled)

If the 'flow_on_disk' limit policy is in fact not applicable to non-durable queues, than this should be changed into documentation bug with request to alter chapter 3 of MIG accordingly.

Version-Release number of selected component (if applicable):
python-qpid-0.14-1.el6.noarch
python-qpid-qmf-0.14-3.el6.x86_64
qpid-cpp-client-0.14-1.el6.x86_64
qpid-cpp-client-devel-0.14-1.el6.x86_64
qpid-cpp-client-devel-docs-0.12-6.el6.noarch
qpid-cpp-server-0.14-1.el6.x86_64
qpid-cpp-server-devel-0.14-1.el6.x86_64
qpid-cpp-server-store-0.14-1.el6.x86_64
qpid-cpp-server-xml-0.14-1.el6.x86_64
qpid-java-client-0.14-1.el6.noarch
qpid-java-common-0.14-1.el6.noarch
qpid-java-example-0.14-1.el6.noarch
qpid-qmf-0.14-3.el6.x86_64
qpid-tools-0.14-1.el6.noarch


How reproducible:
always

Steps to Reproduce:
1. Try to create a non-durable queue with limit policy 'flow_to_disk' (f.e., by running "qpid-config add queue test --max-queue-count=100 --limit-policy=flow-to-disk --flow-stop-count=0 --flow-resume-count=0").
2. Try to send to the newly created queue an amount of messages which exceeds max-queue-count set by the previous command.
  
Actual results:
The limit policy 'reject' was used instead, the exception 'resource-limit-exceeded' received from the broker.


Expected results:
The queue was created with the given limit policy 'flow_to_disk', no exceptions from the broker, the messages exceeding max-queue-count limit are saved to disk.

Additional info:

Comment 2 Kim van der Riet 2013-05-01 15:03:59 UTC
The flow-to-disk mode only works on durable queues. The documentation quoted above is incorrect. Only durable queues have an active journal associated with them, and this is needed to write and recover flowed messages. When no store is present for a queue, there is no option available to a queue but to reject the message.

Note that a distinction must be made between the persistence of queues and of the messages on that queue. If a queue is non-persistent, then none of the messages will be persisted, no matter the persistence status of each message. However, if a queue is persistent, then the persistence status of the message determines whether a message will be persisted. For flow-to-disk messages, when the limit is reached, both persistent and non-persistent messages will be stored.

The above documentation link is stale. It is possible that the documentation is meant to say "can apply to both persistent and non-persistent _messages_". I recommend opening a documentation bug.

It is also possible that the queue logic that handles the policy should warn if flow-to-disk policies are used on non-persistent queues at the time the queue is declared.

I recommend changing this bug to CLOSED - NOTABUG.

Comment 3 Leonid Zhaldybin 2013-05-02 10:39:12 UTC
I'm changing this into a documentation bug.
The MCIG chapter "4.9.1. Controlling Queue Size" contains the description of flow-to-disk policy without stating that it is applicable to durable queues only. The information provided by Kim in comment 2 should be added to the documentation.

Comment 4 Joshua Wulf 2013-07-15 02:56:57 UTC
Added notes "effective only on durable queues" and "the messages stored to disk are not persistent unless the message is marked persistent."

http://deathstar1.usersys.redhat.com:3000/builds/7069-Messaging_Installation_and_Configuration_Guide/#Controlling_Queue_Size