Bug 723993

Summary: Broker crash while attempting to persist messages within last cluster node
Product: Red Hat Enterprise MRG Reporter: Jason Dillaman <jdillama>
Component: qpid-cppAssignee: Kim van der Riet <kim.vdriet>
Status: NEW --- QA Contact: MRG Quality Engineering <mrgqe-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 1.3CC: iboverma, jross, mcressma
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 698367, 803771    
Attachments:
Description Flags
reproducer none

Description Jason Dillaman 2011-07-21 17:09:00 UTC
Created attachment 514237 [details]
reproducer

Description of problem:
After dropping down to a single cluster member, the broker can crash while attempting to force persist messages.

Version-Release number of selected component (if applicable):
qpid-cpp-server-0.7.946106-32_ptc_hotfix_6_v3.el5

How reproducible:
100% using attached reproducer

Steps to Reproduce:
1. Create a broker cluster
2. Create a queue with ring policy, max count of 1, and persist last node policy
3. Enqueue a message in the queue
4. Reduce the cluster to a single member
  
Actual results:
Last standing broker crashes

Expected results:
Last standing broker does not crash

Additional info:
The logic within Queue::setLastNodeFailure() will iterate over all messages in the queue and attempt to re-enqueue any messages that need to be forced persistent.  However, it is possible that the queue policy that is run within Queue::enqueue will dequeue messages within the queue while the calling function is iterating over the message queue.  In the example of a queue that only permits a single message, the ring policy will dequeue the message that is being re-enqueued for persistence, invaliding the intrusive pointer.