Red Hat Bugzilla – Bug 500174
Queue purge unsafe on cluster
Last modified: 2015-11-15 19:07:09 EST
Description of problem:
If a new cluster member is added after broker objects (queues, exchanges etc) are added to the original cluster nodes, then the new member allocates different management ids to those objects.
This causes problems when e.g. a management method is invoked
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. start one cluster node
2. add a queue (e.g. qpid-config add queue test-queue)
3. start new node for the same cluster
4. send 5 messages to the queue (e.g. for m in A B C D E; do echo $m; done | ./src/tests/sender --send-eos 1 -p NODE_1)
5. connect to one node with qpid-tool and invoke purge on the queue specifying 3 messages should be removed (e.g. call <ID> purge 3)
6. exit qpid-tool and consume messages from the other node (i.e. not the one we just connected to using qpid-tool) (e.g. ./src/tests/receiver -p NODE_2)
All 5 message are received although three were purged off the other node. The other node also crashes with something like:
error Execution exception: invalid-argument: anonymous.04e9044c-211f-403e-863a-fbeb0704566b: confirmed < (4+0) but only sent < (3+0) (qpid/SessionState.cpp:151)
Purge should work consistently across cluster, leaving 2 messages on the queue in all nodes. Subsequent use of cluster should not cause crashes.
You can also just check using qpid-tool that the queues have different ids after step 3 above, which is the root of the problem.
(sender and receiver test programs used in examples above are from qpid svn)
Moving to XPlanner backlog.
Created attachment 344213 [details]
Patch for qpidd that disallows dangerous methods in clusters
Created attachment 344506 [details]
An improved patch for disallowing dangerous methods during cluster operation.
Queue.purge and other potentially dangerous methos disallowed for clusters in qpidd-0.5.752581-8
The issue has been workarounded by disabling queue purge in qpidd cluster configuration. Validated on RHEL 5.3 i386 / x86_64 on packages:
[root@intel-greencity-01 bz499872]# rpm -qa | egrep '(qpid|openais)' | sort -u
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.