Red Hat Bugzilla – Bug 509892
byte credit calculation inconsistent for messages transfered to new joiner
Last modified: 2010-12-01 04:27:33 EST
Created attachment 350650 [details]
Description of problem:
For the case where messages in a queue are transfered to a new cluster member, subsequent credit calculations are inconsistent between nodes meaning that subscriptions using byte credit can cause inconsistent queue state and ultimately cause brokers in the cluster to exit with errors.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. start cluster node
2. send a message to a queue
e.g. qpid-config add queue test-queue
echo MyMessage | sender
3. add a new node to the cluster (e.g. on port 5673)
4. using attached test program, consume the message using an exact byte credit allocation from the new node
e.g. test --byte-credit 60 --message-credit 1 --port 5673
Client gets message but the first cluster node exits with:
2009-jul-06 13:28:34 critical 10.16.44.221:20781(READY/error) error 254 did not occur on 10.16.44.221:22731
2009-jul-06 13:28:34 error Error delivering frames: Aborted by local failure that did not occur on all replicas
2009-jul-06 13:28:34 notice 10.16.44.221:20781(LEFT/error) leaving cluster grs-mrg14-test-cluster
2009-jul-06 13:28:34 notice Shut down
No node exits, message gets delivered and removed from queue on both nodes.
From experiments it appears the the calculation for required byte credit on the newly added node is one byte less than on the first node.
Fixed in SVN r821830
The delivery-properties.exchange was not being set during the update process.
The reproducer uses the default exchange so there's a 1 byte differnece - the 0 count byte.
on 752581 bug appears
on 946106 does not. It has been fixed
validated on RHEL 5.5 i386 / x86_64 not on RHEL4 because lack of clustering
# rpm -qa | grep -E '(qpid|openais|rhm)' | sort -u
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
Previously, transferring messages in a queue to a new cluster member resulted in inconsistent credit calculations between nodes, which may have caused the brokers in the cluster to exit with errors. With this update, the "delivery-properties.exchange" is set during the update process, so that the credit calculations are consistent.
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.