Bug 471247 - Description of problem: qpidd crashing while running perftest
Description of problem: qpidd crashing while running perftest
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp (Show other bugs)
All Linux
urgent Severity urgent
: 1.1
: ---
Assigned To: Gordon Sim
Kim van der Riet
: 468821 468822 (view as bug list)
Depends On:
Blocks: 468428
  Show dependency treegraph
Reported: 2008-11-12 11:51 EST by Tom Tracy
Modified: 2009-02-04 10:35 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-02-04 10:35:28 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
core file (47 bytes, text/plain)
2008-11-12 11:51 EST, Tom Tracy
no flags Details
test to open lots of connections concurrently (2.01 KB, text/x-c++src)
2008-11-13 07:27 EST, Gordon Sim
no flags Details

  None (edit)
Description Tom Tracy 2008-11-12 11:51:01 EST
Created attachment 323348 [details]
core file

Description of problem: qpidd crashing while running perftest 
kernel RHEL5.2 GA  Intel X86_64

#0  0x000000369d830155 in raise () from /lib64/libc.so.6
#1  0x000000369d831bf0 in abort () from /lib64/libc.so.6
#2  0x000000369d86a38b in __libc_message () from /lib64/libc.so.6
#3  0x000000369d871634 in _int_free () from /lib64/libc.so.6
#4  0x000000369d874c5c in free () from /lib64/libc.so.6
#5  0x00000036ab69dc1a in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string () from /usr/lib64/libstdc++.so.6
#6  0x00002b74b57c669b in qpid::Url::~Url () from /usr/lib64/libqpidbroker.so.0
#7  0x00002b74b57bcf1d in qpid::broker::Broker::getKnownBrokersImpl () from /usr/lib64/libqpidbroker.so.0
#8  0x00002b74b57c1b59 in boost::detail::function::function_obj_invoker0<boost::_bi::bind_t<std::vector<qpid::Url, std::allocator<qpid::Url> >, boost::_mfi::mf0<std::vector<qpid::Url, std::allocator<qpid::Url> >, qpid::broker::Broker>, boost::_bi::list1<boost::_bi::value<qpid::broker::Broker*> > >, std::vector<qpid::Url, std::allocator<qpid::Url> > >::invoke () from /usr/lib64/libqpidbroker.so.0
#9  0x00002b74b57eb6f7 in boost::function0<std::vector<qpid::Url, std::allocator<qpid::Url> >, std::allocator<void> >::operator() ()
   from /usr/lib64/libqpidbroker.so.0
#10 0x00002b74b57ea057 in qpid::broker::ConnectionHandler::Handler::open () from /usr/lib64/libqpidbroker.so.0
#11 0x00002b74b5bd09a0 in qpid::framing::AMQP_ServerOperations::ConnectionHandler::Invoker::visit () from /usr/lib64/libqpidcommon.so.0
#12 0x00002b74b57eb8b4 in qpid::framing::invoke<qpid::framing::AMQP_ServerOperations::ConnectionHandler> () from /usr/lib64/libqpidbroker.so.0
#13 0x00002b74b57ea99f in qpid::broker::ConnectionHandler::handle () from /usr/lib64/libqpidbroker.so.0
#14 0x00002b74b57e4920 in qpid::broker::Connection::received () from /usr/lib64/libqpidbroker.so.0
#15 0x00002b74b57b7c36 in qpid::amqp_0_10::Connection::decode () from /usr/lib64/libqpidbroker.so.0
#16 0x00002b74b5c94513 in qpid::sys::AsynchIOHandler::readbuff () from /usr/lib64/libqpidcommon.so.0
#17 0x00002b74b5c43bba in boost::function2<void, qpid::sys::AsynchIO&, qpid::sys::AsynchIOBufferBase*, std::allocator<boost::function_base> >::operator() ()
   from /usr/lib64/libqpidcommon.so.0
#18 0x00002b74b5c40625 in qpid::sys::posix::AsynchIO::readable () from /usr/lib64/libqpidcommon.so.0
#19 0x00002b74b5c97219 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator() () from /usr/lib64/libqpidcommon.so.0
#20 0x00002b74b5c955a4 in qpid::sys::DispatchHandle::processEvent () from /usr/lib64/libqpidcommon.so.0
#21 0x00002b74b5c9505e in qpid::sys::Dispatcher::run () from /usr/lib64/libqpidcommon.so.0
#22 0x00002b74b5c4690a in qpid::sys::AbsTime::AbsTime () from /usr/lib64/libqpidcommon.so.0
#23 0x000000369e4062f7 in start_thread () from /lib64/libpthread.so.0
#24 0x000000369d8d1b6d in clone () from /lib64/libc.so.6
Comment 2 Gordon Sim 2008-11-13 07:27:22 EST
Created attachment 323444 [details]
test to open lots of connections concurrently

Also in https://svn.devel.redhat.com/repos/mrg-team/trunk/people/gordon/connection_test
Comment 3 Gordon Sim 2008-11-13 07:31:08 EST
The key to this is having connections opened concurrently. If using perftest to do that you need to run it as separate process for each subscriber etc. Attached is a simple test case that just starts multiple threads opening and closing connections in a loop. I found it to reliably reproduce this and BZ468822. I also believe that the root cause is the same as BZ468821 though I didn't observe that exact stack trace.
Comment 4 Gordon Sim 2008-11-13 07:33:12 EST
*** Bug 468822 has been marked as a duplicate of this bug. ***
Comment 5 Gordon Sim 2008-11-13 07:44:04 EST
Fixed by r713714.
Comment 7 Andrew Stitcher 2008-11-18 09:24:25 EST
*** Bug 468821 has been marked as a duplicate of this bug. ***
Comment 8 Frantisek Reznicek 2008-11-25 10:15:34 EST
Manual testing & qpid_test_broker_perftest_ecte_bz468428 test part B validate that issue has been fixed.
Validated on RHEL 4.7 / 5.2, i386 / x86_64 using packages:
qpidd-0.3.719671-1.el5, rhm-0.3.2804-4.el5
Comment 10 errata-xmlrpc 2009-02-04 10:35:28 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.