Bug 1329582

Summary: consuming messages from priority ring queue on backup broker can trigger broker segfault
Product: Red Hat Enterprise MRG Reporter: Pavel Moravec <pmoravec>
Component: qpid-cppAssignee: messaging-bugs <messaging-bugs>
Status: CLOSED ERRATA QA Contact: Messaging QE <messaging-qe-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 3.2CC: gsim, jross, mcressma, tkratky, zkraus
Target Milestone: 3.2.2   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: qpid-cpp-0.34-10 Doc Type: Bug Fix
Doc Text:
Cause: A priority queue can result in messages being deleted from the queue even if they are acquired by a consumer. In this case the state of the message is changed, to indicate that the message may no longer be valid. However the code that releases acquired messages that have not been acknoweldged when the consumer session ends was not checking this state to ensure the messages to be released were still acquired. Consequence: An attempt was made to release a dequeued message, causing a segmentation fault. Fix: Before releasing the state is checked. Result: Releasing does not cause a segmentation fault, even if the message has already been dequeued.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-11 07:36:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Pavel Moravec 2016-04-22 09:50:18 UTC
Description of problem:
subscribing to a priority ring queue on backup broker and disconnecting with some acquired but not consumed messages, the broker segfaults in one scenario - see reproducer below.


Version-Release number of selected component (if applicable):
qpid-cpp-server-0.34-6.el6.x86_64
qpid-cpp-server-ha-0.34-6.el6.x86_64


How reproducible:
100%


Steps to Reproduce:

killall qpidd qpid-send qpid-receive

qpidd --ha-queue-replication true --auth=no --port 5672 --no-data-dir & 
qpidd --ha-queue-replication true --auth=no --port 6672 --no-data-dir &

qpid-config --broker localhost:5672 add queue q --max-queue-count 5 --limit-policy ring --argument qpid.priorities=9
qpid-config --broker localhost:6672 add queue q --max-queue-count 5 --limit-policy ring --argument qpid.priorities=9
qpid-ha replicate --broker localhost:6672 localhost:5672 q

qpid-receive --address q -f --broker localhost:6672 &
bckpreceivepid=$!

qpid-receive --address q -f --broker localhost:6672 -m2 &

qpid-send --address q --priority 4 --content-string m1
qpid-send --address q --priority 3 --content-string m2
qpid-send --address q --priority 2 --content-string m3
qpid-send --address q --priority 1 --content-string m4
qpid-send --address q --priority 4 --content-string m5
qpid-send --address q --priority 4 --content-string m6

kill $bckpreceivepid

sleep 1
ps aux | grep qpid


Actual results:
at the end, just broker listening on port 5672 is running. Broker listening on 6672 segfaulted with backtrace:

#0  qpid::broker::PriorityQueue::release (this=0x7faf00014dc0, cursor=<value optimized out>)
    at /usr/include/c++/4.4.7/bits/stl_vector.h:611
#1  0x00000030b0e30e7a in qpid::broker::Queue::release (this=0x7faf00013d78, position=..., markRedelivered=true)
    at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/Queue.cpp:349
#2  0x00000030b0ecbbf4 in operator() (this=0x16406b8) at /usr/include/c++/4.4.7/bits/stl_function.h:569
#3  for_each<std::reverse_iterator<std::_Deque_iterator<qpid::broker::DeliveryRecord, qpid::broker::DeliveryRecord&, qpid::broker::DeliveryRecord*> >, std::mem_fun_ref_t<void, qpid::broker::DeliveryRecord> > (this=0x16406b8)
    at /usr/include/c++/4.4.7/bits/stl_algo.h:4200
#4  qpid::broker::SemanticState::requeue (this=0x16406b8) at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/SemanticState.cpp:559
#5  0x00000030b0ed15a8 in qpid::broker::SemanticState::closed (this=0x16406b8)
    at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/SemanticState.cpp:106
#6  0x00000030b0ee9211 in qpid::broker::SessionState::~SessionState (this=0x16404f0, __in_chrg=<value optimized out>)
    at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/SessionState.cpp:107
#7  0x00000030b0ee97b9 in qpid::broker::SessionState::~SessionState (this=0x16404f0, __in_chrg=<value optimized out>)
    at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/SessionState.cpp:110
#8  0x00000030b0eeeb54 in ~auto_ptr (this=0x163bd20) at /usr/include/c++/4.4.7/backward/auto_ptr.h:168
#9  qpid::broker::SessionHandler::handleDetach (this=0x163bd20) at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/SessionHandler.cpp:110
#10 0x00000030b0e556bc in qpid::broker::amqp_0_10::Connection::closed (this=0x7faef80047c0)
    at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/amqp_0_10/Connection.cpp:374
#11 0x00000030affbaced in qpid::sys::AsynchIOHandler::disconnect (this=0x7faef8000f90)
    at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/AsynchIOHandler.cpp:201


Expected results:
no segfault


Additional info:

Comment 1 Pavel Moravec 2016-04-27 07:55:52 UTC
I can confirm the above reproducer does not cause segfault in 0.34-10.

Comment 5 errata-xmlrpc 2016-10-11 07:36:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2049.html