Bug 471247 - Description of problem: qpidd crashing while running perftest
Description of problem: qpidd crashing while running perftest
Status: CLOSED ERRATA
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp (Show other bugs)
1.1
All Linux
urgent Severity urgent
: 1.1
: ---
Assigned To: Gordon Sim
Kim van der Riet
:
: 468821 468822 (view as bug list)
Depends On:
Blocks: 468428
  Show dependency treegraph
 
Reported: 2008-11-12 11:51 EST by Tom Tracy
Modified: 2009-02-04 10:35 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-02-04 10:35:28 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
core file (47 bytes, text/plain)
2008-11-12 11:51 EST, Tom Tracy
no flags Details
test to open lots of connections concurrently (2.01 KB, text/x-c++src)
2008-11-13 07:27 EST, Gordon Sim
no flags Details

  None (edit)
Description Tom Tracy 2008-11-12 11:51:01 EST
Created attachment 323348 [details]
core file

Description of problem: qpidd crashing while running perftest 
kernel RHEL5.2 GA  Intel X86_64


#0  0x000000369d830155 in raise () from /lib64/libc.so.6
#1  0x000000369d831bf0 in abort () from /lib64/libc.so.6
#2  0x000000369d86a38b in __libc_message () from /lib64/libc.so.6
#3  0x000000369d871634 in _int_free () from /lib64/libc.so.6
#4  0x000000369d874c5c in free () from /lib64/libc.so.6
#5  0x00000036ab69dc1a in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string () from /usr/lib64/libstdc++.so.6
#6  0x00002b74b57c669b in qpid::Url::~Url () from /usr/lib64/libqpidbroker.so.0
#7  0x00002b74b57bcf1d in qpid::broker::Broker::getKnownBrokersImpl () from /usr/lib64/libqpidbroker.so.0
#8  0x00002b74b57c1b59 in boost::detail::function::function_obj_invoker0<boost::_bi::bind_t<std::vector<qpid::Url, std::allocator<qpid::Url> >, boost::_mfi::mf0<std::vector<qpid::Url, std::allocator<qpid::Url> >, qpid::broker::Broker>, boost::_bi::list1<boost::_bi::value<qpid::broker::Broker*> > >, std::vector<qpid::Url, std::allocator<qpid::Url> > >::invoke () from /usr/lib64/libqpidbroker.so.0
#9  0x00002b74b57eb6f7 in boost::function0<std::vector<qpid::Url, std::allocator<qpid::Url> >, std::allocator<void> >::operator() ()
   from /usr/lib64/libqpidbroker.so.0
#10 0x00002b74b57ea057 in qpid::broker::ConnectionHandler::Handler::open () from /usr/lib64/libqpidbroker.so.0
#11 0x00002b74b5bd09a0 in qpid::framing::AMQP_ServerOperations::ConnectionHandler::Invoker::visit () from /usr/lib64/libqpidcommon.so.0
#12 0x00002b74b57eb8b4 in qpid::framing::invoke<qpid::framing::AMQP_ServerOperations::ConnectionHandler> () from /usr/lib64/libqpidbroker.so.0
#13 0x00002b74b57ea99f in qpid::broker::ConnectionHandler::handle () from /usr/lib64/libqpidbroker.so.0
#14 0x00002b74b57e4920 in qpid::broker::Connection::received () from /usr/lib64/libqpidbroker.so.0
#15 0x00002b74b57b7c36 in qpid::amqp_0_10::Connection::decode () from /usr/lib64/libqpidbroker.so.0
#16 0x00002b74b5c94513 in qpid::sys::AsynchIOHandler::readbuff () from /usr/lib64/libqpidcommon.so.0
#17 0x00002b74b5c43bba in boost::function2<void, qpid::sys::AsynchIO&, qpid::sys::AsynchIOBufferBase*, std::allocator<boost::function_base> >::operator() ()
   from /usr/lib64/libqpidcommon.so.0
#18 0x00002b74b5c40625 in qpid::sys::posix::AsynchIO::readable () from /usr/lib64/libqpidcommon.so.0
#19 0x00002b74b5c97219 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator() () from /usr/lib64/libqpidcommon.so.0
#20 0x00002b74b5c955a4 in qpid::sys::DispatchHandle::processEvent () from /usr/lib64/libqpidcommon.so.0
#21 0x00002b74b5c9505e in qpid::sys::Dispatcher::run () from /usr/lib64/libqpidcommon.so.0
#22 0x00002b74b5c4690a in qpid::sys::AbsTime::AbsTime () from /usr/lib64/libqpidcommon.so.0
#23 0x000000369e4062f7 in start_thread () from /lib64/libpthread.so.0
#24 0x000000369d8d1b6d in clone () from /lib64/libc.so.6
Comment 2 Gordon Sim 2008-11-13 07:27:22 EST
Created attachment 323444 [details]
test to open lots of connections concurrently

Also in https://svn.devel.redhat.com/repos/mrg-team/trunk/people/gordon/connection_test
Comment 3 Gordon Sim 2008-11-13 07:31:08 EST
The key to this is having connections opened concurrently. If using perftest to do that you need to run it as separate process for each subscriber etc. Attached is a simple test case that just starts multiple threads opening and closing connections in a loop. I found it to reliably reproduce this and BZ468822. I also believe that the root cause is the same as BZ468821 though I didn't observe that exact stack trace.
Comment 4 Gordon Sim 2008-11-13 07:33:12 EST
*** Bug 468822 has been marked as a duplicate of this bug. ***
Comment 5 Gordon Sim 2008-11-13 07:44:04 EST
Fixed by r713714.
Comment 7 Andrew Stitcher 2008-11-18 09:24:25 EST
*** Bug 468821 has been marked as a duplicate of this bug. ***
Comment 8 Frantisek Reznicek 2008-11-25 10:15:34 EST
Manual testing & qpid_test_broker_perftest_ecte_bz468428 test part B validate that issue has been fixed.
Validated on RHEL 4.7 / 5.2, i386 / x86_64 using packages:
qpidd-0.3.719671-1.el5, rhm-0.3.2804-4.el5
->VERIFIED
Comment 10 errata-xmlrpc 2009-02-04 10:35:28 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-0035.html

Note You need to log in before you can comment on or make changes to this bug.