Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 949014

Summary: Setting --ack value on Federation links forces a fixed finite credit limit.
Product: Red Hat Enterprise MRG Reporter: Ken Giusti <kgiusti>
Component: qpid-cppAssignee: Ken Giusti <kgiusti>
Status: CLOSED ERRATA QA Contact: Zdenek Kraus <zkraus>
Severity: high Docs Contact:
Priority: high    
Version: 2.3CC: jross, kgiusti, lzhaldyb, pmoravec, sauchter, zkraus
Target Milestone: 3.0Keywords: Improvement
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qpid-cpp-0.22-4.el6, qpid-cpp-0.22-4.el5 Doc Type: Enhancement
Doc Text:
This feature allows credit level configuration on Federation connections. By default, the credit level was fixed to unlimited, or forced to twice the --ack value (if specified). There was no other way to configure the credit level. The user can now better tune the performance of the Federation link by specifying custom credit levels. This feature is documented in the "qpid Route Options" section of the Messaging Installation and Configuration Guide.
Story Points: ---
Clone Of:
: 951296 961001 (view as bug list) Environment:
Last Closed: 2014-09-24 15:07:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 951296, 961001    
Attachments:
Description Flags
Pavel's traffic generator for the reproducer
none
Pavels reproducer script none

Description Ken Giusti 2013-04-05 16:06:39 UTC
Description of problem:


When a federation connection is created, by default a subscription is created that does not use ACKs (implicit accept), and allows infinite credit.

As an option, explicit acknowledgment can be turned on via the --ack N option provided by qpid-route.  This option configures the subscription to require implicit acknowledgments, and cause the session to synchronize every N messages.

However, when --ack is specified, the credit configuration is changed to be finite, and set to 2 * N messages.

Forcing use of finite credit when --acks are used should not be required.  For some applications, the backpressure enabled by setting finite credit is undesirable.  A good example would be when federation is used with asynchronous queue replication.  In that case, turning on acknowledgments could provide for better fail-over behavior, but setting finite credit makes no sense since the asynchronous events cannot be back pressured.

Instead, the user should be able to configure the credit limit separately from the acknowledgment setting.  This bug report proposes this approach:

1) modify qpid-route and QMF bridge creation methods to allow for a new credit option (e.g. --credit) that would configure the credit separately from acknowledgment.  It would take an unsigned integer argument in the range of 0 thru 0xFFFFFFFF-1 (0xFFFFFFF is reserved for internal use).  A setting of zero is interpreted as infinite credit.

2) change the existing configuration behavior to *no longer* automatically modify the credit setting when acknowledgment is configured.  

In summary, the change would separate the configuration of acks and credit.  Setting one would not affect the other.

Other details:

- this breaks the existing behavior for those that are using the --ack option.

- when setting both acks and credit, the value of credit must be > the configured ack value.  A setting of credit <= ack will be treated as a error.

Comment 1 Stuart Auchterlonie 2013-04-05 20:17:57 UTC
*** Bug 948981 has been marked as a duplicate of this bug. ***

Comment 2 Stuart Auchterlonie 2013-04-05 20:21:38 UTC
Created attachment 732007 [details]
Pavel's traffic generator for the reproducer

Comment 3 Stuart Auchterlonie 2013-04-05 20:23:40 UTC
Created attachment 732008 [details]
Pavels reproducer script

How to use Pavel's reproducer

Usage: 
./setup_asyncRepl.sh <clustering?> <replicating_queues_number> <producers_per_queue> <ack> <Nth_producer_durable>

I.e. when running:
./setup_asyncRepl.sh yes 9 1 10000 9

generates 1/9 msgs of traffic are durable messages, usually the federation link stops passing any traffic after passing few millions messages.

Comment 8 Zdenek Kraus 2013-06-19 12:07:37 UTC
This fix work fine, but I have one concerne with the setting the credit and QMF.

When you create a route without specifying the credit value (credit should be not applied (=unlimited)) the QMF reports the credit on the bridge object as 2^32 thus unlimited.

Object of type: org.apache.qpid.broker:bridge:_data(b71da3c2-cbac-950f-1954-2aeced85659e)
    Attribute   165
    ==============================================
    linkRef     178
    name        qpid.tcp:192.168.6.3:5672!dq!ex!
    channelId   2
    durable     False
    src         dq
    dest        ex
    key         
    srcIsQueue  True
    srcIsLocal  False
    tag         
    excludes    
    dynamic     False
    sync        1
    credit      4294967295



But the other case, when you specify the --credit=0, result should be the same (=unlimited), the QMF in this case reports a zero credit.

Object of type: org.apache.qpid.broker:bridge:_data(b71da3c2-cbac-950f-1954-2aeced85659e)
    Attribute   188
    ==============================================
    linkRef     201
    name        qpid.tcp:192.168.6.3:5672!dq!ex!
    channelId   3
    durable     False
    src         dq
    dest        ex
    key         
    srcIsQueue  True
    srcIsLocal  False
    tag         
    excludes    
    dynamic     False
    sync        1
    credit      0



My concern is, is there any special handling of the zero value? like disabling credit check to not degradate the performance? Or is the reporting wrong in the QMF?

Comment 9 Ken Giusti 2013-06-19 13:23:34 UTC
I think the reporting is incorrect.  Using zero when setting the credit limit is simply provided as a user interface convenience: it's a lot easier to specify '0' than '0xFFFFFFFF' on the command line (though both should result in setting unlimited credit).

Can you open a new BZ to correct the credit value that is advertised by QMF?

Comment 10 Zdenek Kraus 2013-06-24 13:19:16 UTC
As stated above (comment 8) this works fine, QMF issue was separated into Bug 975867.

It was tested on RHEL 5.9 and 6.4, i386 and x86_64, with packages:

python-qpid-0.22-3.el6.noarch
python-qpid-qmf-0.22-3.el6.x86_64
qpid-cpp-client-0.22-4.el6.x86_64
qpid-cpp-client-devel-0.22-4.el6.x86_64
qpid-cpp-client-devel-docs-0.22-4.el6.noarch
qpid-cpp-client-rdma-0.22-4.el6.x86_64
qpid-cpp-client-ssl-0.22-4.el6.x86_64
qpid-cpp-debuginfo-0.22-4.el6.x86_64
qpid-cpp-server-0.22-4.el6.x86_64
qpid-cpp-server-devel-0.22-4.el6.x86_64
qpid-cpp-server-ha-0.22-4.el6.x86_64
qpid-cpp-server-rdma-0.22-4.el6.x86_64
qpid-cpp-server-ssl-0.22-4.el6.x86_64
qpid-cpp-server-store-0.22-4.el6.x86_64
qpid-cpp-server-xml-0.22-4.el6.x86_64
qpid-proton-c-0.4-2.2.el6.x86_64
qpid-qmf-0.22-3.el6.x86_64
qpid-tools-0.22-3.el6.noarch

Comment 11 Jared MORGAN 2014-09-17 01:20:10 UTC
Added reference to the documentation done in BZ#961001 into the Release Notes.

Comment 12 errata-xmlrpc 2014-09-24 15:07:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1296.html