Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1103094

Summary: Memory leak and deadlock together when using qpid.auto_delete_timeout queues
Product: Red Hat Enterprise MRG Reporter: Pavel Moravec <pmoravec>
Component: qpid-cppAssignee: Gordon Sim <gsim>
Status: CLOSED ERRATA QA Contact: Leonid Zhaldybin <lzhaldyb>
Severity: urgent Docs Contact:
Priority: high    
Version: 2.5CC: esammons, gsim, jross, lzhaldyb, mcressma, pmoravec, rrajasek, sauchter
Target Milestone: 2.5.1Keywords: TestCaseProvided
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: qpid-cpp-0.18-25 Doc Type: Bug Fix
Doc Text:
It was discovered that a concurrent bind and delete on a queue could leave a binding in place, even though the queue was no longer accessible. Additionally, the queue lock was held when cancelling the auto-delete timer task. These factors caused memory usage to climb, and holding the lock while cancelling a timer could also result in a deadlock. Threads are now coordinated to ensure a concurrent bind and delete removes all bindings when a queue reaches the deleted state, and the timer task is cancelled outside the lock scope. This ensures no deadlocks or memory leaks when attempting to re-bind a queue that is being auto-deleted by a timer.
Story Points: ---
Clone Of:
: 1105221 (view as bug list) Environment:
Last Closed: 2014-06-30 10:26:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1105221    
Attachments:
Description Flags
qpidd log file from the startup till the point a client hangs
none
the output of 'strace -p $(pidof qpidd)
none
pstack trace of qpidd
none
pstack trace of the client
none
example of qpidd memory consumption growth over time none

Description Pavel Moravec 2014-05-30 08:37:17 UTC
Description of problem:
There is a memory leak in the broker when a client declares auto-delete queue with qpid.auto_delete_timeout and re-connects just at the time the broker is about to delete the queue.


Version-Release number of selected component (if applicable):
0.18-11, 0.18-20, 0.22-41


How reproducible:
100% within some time


Steps to Reproduce:
1) Run script:

queues=100
sleep=0
while true; do
	for i in $(seq 1 $queues); do
		qpid-receive -a "autoDelQueue_${i}; {create:always, node:{ x-declare:{auto-delete:True, arguments:{'qpid.auto_delete_timeout':1}}, x-bindings:[{exchange:'amq.fanout', queue:autoDelQueue_${i}}] }}" -b broker --print-content=no -m 10 &
	done
	wait
	sleep $sleep
	date
done

Modify parameters (esp. sleep) such that the loop is executed once every second.

2) Monitor qpid broker memory utilization over time.


Actual results:
Both VSZ and RSS of qpidd process grows.


Expected results:
No memory grow.


Additional info:
Optionally, one can generate some traffic by running:

qpid-send -a amq.fanout -m 100000 -b broker --send-rate=50

in parallel. If so, the broker will end up with coredump (manually generated) backtrace like:

Thread 4 (Thread 0x7f373424a7a0 (LWP 10281)):
#0  0x00007f3731fa23dc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f373378c09b in wait (this=0x8a1890) at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/posix/Condition.h:63
#2  wait (this=0x8a1890) at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/Monitor.h:41
#3  qpid::sys::TimerTask::cancel (this=0x8a1890) at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/Timer.cpp:94
#4  0x00007f3733c71345 in qpid::broker::Queue::consume (this=0xf05570, c=..., requestExclusive=false, arguments=..., connectionId=
    "qpid.10.34.1.121:5672-10.34.1.149:54185", userId="anonymous@QPID") at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/Queue.cpp:571
#5  0x00007f3733d0ec29 in qpid::broker::SemanticState::consume (this=0x7f37207ac168, tag="autoDelQueue_73", 
    queue=<value optimized out>, ackRequired=true, acquire=true, exclusive=false, resumeId="", resumeTtl=0, arguments=...)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/SemanticState.cpp:144
#6  0x00007f3733d21f8d in qpid::broker::SessionAdapter::MessageHandlerImpl::subscribe (this=0x7f37207ac358, queueName=
    "autoDelQueue_73", destination="autoDelQueue_73", acceptMode=0 '\000', acquireMode=<value optimized out>, exclusive=false, 
    resumeId="", resumeTtl=0, arguments=...) at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/SessionAdapter.cpp:415
#7  0x00007f37336e5668 in invoke<qpid::framing::AMQP_ServerOperations::MessageHandler> (this=<value optimized out>, body=...)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/framing/MessageSubscribeBody.h:120
#8  qpid::framing::AMQP_ServerOperations::MessageHandler::Invoker::visit (this=<value optimized out>, body=...)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/framing/ServerInvoker.cpp:554
#9  0x00007f37336e93e1 in qpid::framing::AMQP_ServerOperations::Invoker::visit (this=0x7fffeadd9140, body=...)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/framing/ServerInvoker.cpp:243
#10 0x00007f3733d296dc in invoke<qpid::broker::SessionAdapter> (this=<value optimized out>, method=0xec3e60)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/framing/Invoker.h:67
#11 qpid::broker::SessionState::handleCommand (this=<value optimized out>, method=0xec3e60)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/SessionState.cpp:175
#12 0x00007f3733d2a6c5 in qpid::broker::SessionState::handleIn (this=0x7f37207abfa0, frame=...)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/SessionState.cpp:267
#13 0x00007f373375a8d1 in qpid::amqp_0_10::SessionHandler::handleIn (this=0x7f3720200a20, f=...)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/amqp_0_10/SessionHandler.cpp:93
#14 0x00007f3733ca002b in operator() (this=0x7f37200d3608, frame=...) at /usr/src/debug/qpid-0.22/cpp/src/qpid/framing/Handler.h:42
#15 qpid::broker::ConnectionHandler::handle (this=0x7f37200d3608, frame=...)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/ConnectionHandler.cpp:94
#16 0x00007f3733c98ae8 in qpid::broker::amqp_0_10::Connection::received (this=0x7f37200d3420, frame=...)
---Type <return> to continue, or q <return> to quit---
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/amqp_0_10/Connection.cpp:198
#17 0x00007f3733c2d70d in qpid::amqp_0_10::Connection::decode (this=0x7f3720752730, buffer=<value optimized out>, 
    size=<value optimized out>) at /usr/src/debug/qpid-0.22/cpp/src/qpid/amqp_0_10/Connection.cpp:58
#18 0x00007f3733783980 in qpid::sys::AsynchIOHandler::readbuff (this=0x7f3728acdfd0, buff=0x7f3728a3e280)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/AsynchIOHandler.cpp:135
#19 0x00007f373370e524 in operator() (this=0x7f372873e390, h=...) at /usr/include/boost/function/function_template.hpp:1013
#20 qpid::sys::posix::AsynchIO::readable (this=0x7f372873e390, h=...)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/posix/AsynchIO.cpp:453
#21 0x00007f3733788033 in boost::function1<void, qpid::sys::DispatchHandle&>::operator() (this=<value optimized out>, 
    a0=<value optimized out>) at /usr/include/boost/function/function_template.hpp:1013
#22 0x00007f3733787181 in qpid::sys::DispatchHandle::processEvent (this=0x7f372873e398, type=qpid::sys::Poller::READABLE)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/DispatchHandle.cpp:280
#23 0x00007f3733731302 in process (this=0x70fa80) at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/Poller.h:131
#24 qpid::sys::Poller::run (this=0x70fa80) at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/epoll/EpollPoller.cpp:522
#25 0x00007f3733c489a2 in qpid::broker::Broker::run (this=<value optimized out>)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/Broker.cpp:450
#26 0x000000000040a9c4 in qpid::broker::QpiddDaemon::child (this=<value optimized out>)
    at /usr/src/debug/qpid-0.22/cpp/src/posix/QpiddBroker.cpp:149
#27 0x00007f3733bebcc3 in qpid::broker::Daemon::fork (this=0x7fffeaddb260)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/Daemon.cpp:91
#28 0x000000000040756a in qpid::broker::QpiddBroker::execute (this=<value optimized out>, options=<value optimized out>)
    at /usr/src/debug/qpid-0.22/cpp/src/posix/QpiddBroker.cpp:193
#29 0x000000000040cba4 in qpid::broker::run_broker (argc=6, argv=0x7fffeaddb898, hidden=<value optimized out>)
    at /usr/src/debug/qpid-0.22/cpp/src/qpidd.cpp:108
#30 0x00007f3731c14cdd in __libc_start_main () from /lib64/libc.so.6
#31 0x0000000000406bf9 in _start ()

Thread 1 (Thread 0x7f3731281700 (LWP 10282)):
#0  0x00007f3731fa4ff4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f3731fa0343 in _L_lock_892 () from /lib64/libpthread.so.0
#2  0x00007f3731fa0227 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00007f3733c0228c in lock (this=<value optimized out>, l=<value optimized out>)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/posix/Mutex.h:116
#4  qpid::sys::ScopedLock<qpid::sys::Mutex>::ScopedLock (this=<value optimized out>, l=<value optimized out>)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/Mutex.h:33
#5  0x00007f3733c7a0e6 in qpid::broker::Queue::remove (this=0xf05570, maxCount=0, p=..., f=..., type=qpid::broker::REPLICATOR, 
    triggerAutoDelete=false, maxTests=0) at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/Queue.cpp:746
#6  0x00007f3733c7acf2 in qpid::broker::Queue::destroyed (this=0xf05570)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/Queue.cpp:1151
#7  0x00007f3733c7b28b in qpid::broker::Queue::tryAutoDelete (this=0xf05570)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/broker/Queue.cpp:1319
#8  0x00007f373378b960 in qpid::sys::Timer::fire (this=<value optimized out>, t=...)
---Type <return> to continue, or q <return> to quit---
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/Timer.cpp:222
#9  0x00007f373378d05f in qpid::sys::Timer::run (this=0x729df0) at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/Timer.cpp:156
#10 0x00007f373372661a in qpid::sys::(anonymous namespace)::runRunnable (p=<value optimized out>)
    at /usr/src/debug/qpid-0.22/cpp/src/qpid/sys/posix/Thread.cpp:35
#11 0x00007f3731f9e7f1 in start_thread () from /lib64/libpthread.so.0
#12 0x00007f3731cdb70d in clone () from /lib64/libc.so.6


(btw isn't that a deadlock?)
(btw the referenced queue above did not exist at the time core was generated - at least per qpid-stat -q output)

Comment 1 Pavel Moravec 2014-05-30 10:08:51 UTC
Btw. this _is_ also a deadlock. When having even number of threads, one can end up with totally unresponsible broker (that consumes a lot of memory, in addition). Hence severity urgent.

Comment 2 Pavel Moravec 2014-05-30 11:00:19 UTC
Valgrind output (one has to increase qpid.auto_delete_timeout, in my case to 4s):

==16625== 5,481 (952 direct, 4,529 indirect) bytes in 1 blocks are definitely lost in loss record 183 of 193
==16625==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==16625==    by 0x3EE1E3571D: qpid::broker::QueueRegistry::declare(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, boost::shared_ptr<qpid::broker::Exchange>, qpid::framing::FieldTable const&, bool, std::string, std::string) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1D87E16: qpid::broker::Broker::createQueue(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, std::string const&, qpid::framing::FieldTable const&, std::string const&, std::string const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E5F282: qpid::broker::SessionAdapter::QueueHandlerImpl::declare(std::string const&, std::string const&, bool, bool, bool, bool, qpid::framing::FieldTable const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17B2C2E: qpid::framing::AMQP_ServerOperations::QueueHandler::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE17B4830: qpid::framing::AMQP_ServerOperations::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1E68E67: qpid::broker::SessionState::handleCommand(qpid::framing::AMQMethodBody*, qpid::framing::SequenceNumber const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E69348: qpid::broker::SessionState::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17F4590: qpid::amqp_0_10::SessionHandler::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1DACC92: qpid::broker::ConnectionHandler::handle(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1DA5697: qpid::broker::Connection::received(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1D7010C: qpid::amqp_0_10::Connection::decode(char const*, unsigned long) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625== 
==16625== 5,481 (24 direct, 5,457 indirect) bytes in 1 blocks are definitely lost in loss record 184 of 193
==16625==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==16625==    by 0x3EE1E35774: qpid::broker::QueueRegistry::declare(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, boost::shared_ptr<qpid::broker::Exchange>, qpid::framing::FieldTable const&, bool, std::string, std::string) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1D87E16: qpid::broker::Broker::createQueue(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, std::string const&, qpid::framing::FieldTable const&, std::string const&, std::string const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E5F282: qpid::broker::SessionAdapter::QueueHandlerImpl::declare(std::string const&, std::string const&, bool, bool, bool, bool, qpid::framing::FieldTable const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17B2C2E: qpid::framing::AMQP_ServerOperations::QueueHandler::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE17B4830: qpid::framing::AMQP_ServerOperations::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1E68E67: qpid::broker::SessionState::handleCommand(qpid::framing::AMQMethodBody*, qpid::framing::SequenceNumber const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E69348: qpid::broker::SessionState::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17F4590: qpid::amqp_0_10::SessionHandler::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1DACC92: qpid::broker::ConnectionHandler::handle(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1DA5697: qpid::broker::Connection::received(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1D7010C: qpid::amqp_0_10::Connection::decode(char const*, unsigned long) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625== 
==16625== 5,481 (48 direct, 5,433 indirect) bytes in 1 blocks are definitely lost in loss record 185 of 193
==16625==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==16625==    by 0x3EE1E079A0: qpid::broker::Queue::addObserver(boost::shared_ptr<qpid::broker::QueueObserver>) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E6C818: qpid::broker::ThresholdAlerts::observe(qpid::broker::Queue&, qpid::management::ManagementAgent&, unsigned long, unsigned long, long) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E6DEC0: qpid::broker::ThresholdAlerts::observe(qpid::broker::Queue&, qpid::management::ManagementAgent&, std::map<std::string, qpid::types::Variant, std::less<std::string>, std::allocator<std::pair<std::string const, qpid::types::Variant> > > const&, unsigned short) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E6E7D4: qpid::broker::ThresholdAlerts::observe(qpid::broker::Queue&, qpid::management::ManagementAgent&, qpid::framing::FieldTable const&, unsigned short) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E0C4B7: qpid::broker::Queue::configureImpl(qpid::framing::FieldTable const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E3588E: qpid::broker::QueueRegistry::declare(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, boost::shared_ptr<qpid::broker::Exchange>, qpid::framing::FieldTable const&, bool, std::string, std::string) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1D87E16: qpid::broker::Broker::createQueue(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, std::string const&, qpid::framing::FieldTable const&, std::string const&, std::string const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E5F282: qpid::broker::SessionAdapter::QueueHandlerImpl::declare(std::string const&, std::string const&, bool, bool, bool, bool, qpid::framing::FieldTable const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17B2C2E: qpid::framing::AMQP_ServerOperations::QueueHandler::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE17B4830: qpid::framing::AMQP_ServerOperations::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1E68E67: qpid::broker::SessionState::handleCommand(qpid::framing::AMQMethodBody*, qpid::framing::SequenceNumber const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625== 
==16625== 5,481 (48 direct, 5,433 indirect) bytes in 1 blocks are definitely lost in loss record 186 of 193
==16625==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==16625==    by 0x3EE1E079A0: qpid::broker::Queue::addObserver(boost::shared_ptr<qpid::broker::QueueObserver>) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E3D587: qpid::broker::QueueFlowLimit::observe(qpid::broker::Queue&, qpid::framing::FieldTable const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E0CDB6: qpid::broker::Queue::configureImpl(qpid::framing::FieldTable const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E3588E: qpid::broker::QueueRegistry::declare(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, boost::shared_ptr<qpid::broker::Exchange>, qpid::framing::FieldTable const&, bool, std::string, std::string) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1D87E16: qpid::broker::Broker::createQueue(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, std::string const&, qpid::framing::FieldTable const&, std::string const&, std::string const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E5F282: qpid::broker::SessionAdapter::QueueHandlerImpl::declare(std::string const&, std::string const&, bool, bool, bool, bool, qpid::framing::FieldTable const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17B2C2E: qpid::framing::AMQP_ServerOperations::QueueHandler::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE17B4830: qpid::framing::AMQP_ServerOperations::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1E68E67: qpid::broker::SessionState::handleCommand(qpid::framing::AMQMethodBody*, qpid::framing::SequenceNumber const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E69348: qpid::broker::SessionState::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17F4590: qpid::amqp_0_10::SessionHandler::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625== 
==16625== 5,516 (48 direct, 5,468 indirect) bytes in 1 blocks are definitely lost in loss record 187 of 193
==16625==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==16625==    by 0x3EE1D04B10: std::_Rb_tree<std::string, std::pair<std::string const, qpid::types::Variant>, std::_Select1st<std::pair<std::string const, qpid::types::Variant> >, std::less<std::string>, std::allocator<std::pair<std::string const, qpid::types::Variant> > >::_M_copy(std::_Rb_tree_node<std::pair<std::string const, qpid::types::Variant> > const*, std::_Rb_tree_node<std::pair<std::string const, qpid::types::Variant> >*) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E0CD44: qpid::broker::Queue::configureImpl(qpid::framing::FieldTable const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E3588E: qpid::broker::QueueRegistry::declare(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, boost::shared_ptr<qpid::broker::Exchange>, qpid::framing::FieldTable const&, bool, std::string, std::string) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1D87E16: qpid::broker::Broker::createQueue(std::string const&, bool, bool, qpid::broker::OwnershipToken const*, std::string const&, qpid::framing::FieldTable const&, std::string const&, std::string const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E5F282: qpid::broker::SessionAdapter::QueueHandlerImpl::declare(std::string const&, std::string const&, bool, bool, bool, bool, qpid::framing::FieldTable const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17B2C2E: qpid::framing::AMQP_ServerOperations::QueueHandler::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE17B4830: qpid::framing::AMQP_ServerOperations::Invoker::visit(qpid::framing::QueueDeclareBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1E68E67: qpid::broker::SessionState::handleCommand(qpid::framing::AMQMethodBody*, qpid::framing::SequenceNumber const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E69348: qpid::broker::SessionState::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17F4590: qpid::amqp_0_10::SessionHandler::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1DACC92: qpid::broker::ConnectionHandler::handle(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625== 
==16625== 18,114 (552 direct, 17,562 indirect) bytes in 3 blocks are definitely lost in loss record 193 of 193
==16625==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==16625==    by 0x3EE1E19202: qpid::broker::Queue::tryAutoDelete(qpid::broker::Broker&, boost::shared_ptr<qpid::broker::Queue>, std::string const&, std::string const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E48453: qpid::broker::SemanticState::cancel(boost::shared_ptr<qpid::broker::SemanticState::ConsumerImpl>) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E4D380: qpid::broker::SemanticState::cancel(std::string const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E5AF6D: qpid::broker::SessionAdapter::MessageHandlerImpl::cancel(std::string const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17B3272: qpid::framing::AMQP_ServerOperations::MessageHandler::Invoker::visit(qpid::framing::MessageCancelBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE17B64B0: qpid::framing::AMQP_ServerOperations::Invoker::visit(qpid::framing::MessageCancelBody const&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1E68E67: qpid::broker::SessionState::handleCommand(qpid::framing::AMQMethodBody*, qpid::framing::SequenceNumber const&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1E69348: qpid::broker::SessionState::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE17F4590: qpid::amqp_0_10::SessionHandler::handleIn(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidcommon.so.8.0.0)
==16625==    by 0x3EE1DACC92: qpid::broker::ConnectionHandler::handle(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)
==16625==    by 0x3EE1DA5697: qpid::broker::Connection::received(qpid::framing::AMQFrame&) (in /usr/lib64/libqpidbroker.so.8.0.0)

Comment 3 Gordon Sim 2014-06-03 19:20:30 UTC
The deadlock is addressed upstream by: https://svn.apache.org/r1599765

The memory growth is addressed upstream by: https://svn.apache.org/r1599766

Though in running the reproducer I do seem some initial memory growth even with this fix applied, the growth seems to level out rather than continuing to grow indefinitely.

Comment 9 Leonid Zhaldybin 2014-06-17 14:15:58 UTC
Created attachment 909607 [details]
qpidd log file from the startup till the point a client hangs

Comment 10 Leonid Zhaldybin 2014-06-17 14:16:42 UTC
Created attachment 909608 [details]
the output of 'strace -p $(pidof qpidd)

Comment 12 Leonid Zhaldybin 2014-06-17 15:25:37 UTC
Created attachment 909647 [details]
pstack trace of qpidd

Comment 13 Leonid Zhaldybin 2014-06-17 15:26:03 UTC
Created attachment 909648 [details]
pstack trace of the client

Comment 14 Leonid Zhaldybin 2014-06-17 15:27:07 UTC
Created attachment 909649 [details]
example of qpidd memory consumption growth over time

Comment 17 Leonid Zhaldybin 2014-06-23 11:12:24 UTC
Tested on the latest available packages on RHEL5 and RHEL6 (both i386 and x86_64). The reproducer script was running for a few days without triggering a deadlock. The broker does not appear to be leaking memory. This issue has been fixed.

Packages used for testing:

python-qpid-0.18-12
python-qpid-qmf-0.18-24
qpid-cpp-0.18-25
qpid-qmf-0.18-24
qpid-tools-0.18-10

-> VERIFIED

Comment 19 errata-xmlrpc 2014-06-30 10:26:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0804.html