Attachment 638000 Details for Bug 872934 – clustered broker log (5673)

clustered broker log (5673)

broker2.log (text/plain), 12.59 KB, created by Petr Matousek on 2012-11-04 13:02:33 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Petr Matousek

Created: 2012-11-04 13:02:33 UTC

Size: 12.59 KB

patch

obsolete

>Core was generated by `qpidd -p 5673 --cluster-name pematous --data-dir=broker2'.
>Program terminated with signal 11, Segmentation fault.
>#0  0x009d3e83 in qpid::broker::Queue::deliver(boost::intrusive_ptr<qpid::broker::Message>) () from /usr/lib/libqpidbroker.so.8
>(gdb) info threads
>  14 Thread 0xb7f40720 (LWP 12618)  0x0036c410 in __kernel_vsyscall ()
>  13 Thread 12619  0x0036c410 in __kernel_vsyscall ()
>  12 Thread 12621  0x0036c410 in __kernel_vsyscall ()
>  11 Thread 12622  0x0036c410 in __kernel_vsyscall ()
>  10 Thread 12623  0x0036c410 in __kernel_vsyscall ()
>  9 Thread 12624  0x0036c410 in __kernel_vsyscall ()
>  8 Thread 12625  0x00d288d9 in start_thread () from /lib/libpthread.so.0
>  7 Thread 12629  0x00d288d9 in start_thread () from /lib/libpthread.so.0
>  6 Thread 12632  0x0036c410 in __kernel_vsyscall ()
>  5 Thread 12633  0x0036c410 in __kernel_vsyscall ()
>  4 Thread 12634  0x00d288d9 in start_thread () from /lib/libpthread.so.0
>  3 Thread 12636  0x0036c410 in __kernel_vsyscall ()
>  2 Thread 12638  0x0036c410 in __kernel_vsyscall ()
>* 1 Thread 0xb0d5db90 (LWP 12631)  0x009d3e83 in qpid::broker::Queue::deliver(boost::intrusive_ptr<qpid::broker::Message>) ()
>   from /usr/lib/libqpidbroker.so.8
>(gdb) thread apply all bt
>
>Thread 14 (Thread 0xb7f40720 (LWP 12618)):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00c93106 in epoll_wait () from /lib/libc.so.6
>#2  0x005e46da in qpid::sys::Poller::wait(qpid::sys::Duration) () from /usr/lib/libqpidcommon.so.8
>#3  0x005e5303 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x006f6754 in qpid::sys::Dispatcher::run() () from /usr/lib/libqpidcommon.so.8
>#5  0x009017cd in qpid::broker::Broker::run() () from /usr/lib/libqpidbroker.so.8
>#6  0x0804e7ec in ?? ()
>#7  0x0804cc28 in std::ios_base::Init::~Init() ()
>#8  0x0804e34a in ?? ()
>#9  0x00bd4e9c in __libc_start_main () from /lib/libc.so.6
>#10 0x0804c4b1 in std::ios_base::Init::~Init() ()
>
>Thread 13 (Thread 12619):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00d2cf32 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
>#2  0x00c9f1a4 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libc.so.6
>#3  0x006f9433 in qpid::sys::Timer::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#5  0x00d28852 in start_thread () from /lib/libpthread.so.0
>#6  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 12 (Thread 12621):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00d2cc05 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
>#2  0x00c9f15d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
>#3  0x006f90e7 in qpid::sys::Timer::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#5  0x00d28852 in start_thread () from /lib/libpthread.so.0
>#6  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 11 (Thread 12622):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00d2cc05 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
>#2  0x00c9f15d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
>#3  0x006f90e7 in qpid::sys::Timer::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#5  0x00d28852 in start_thread () from /lib/libpthread.so.0
>#6  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 10 (Thread 12623):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00c93106 in epoll_wait () from /lib/libc.so.6
>#2  0x005e46da in qpid::sys::Poller::wait(qpid::sys::Duration) () from /usr/lib/libqpidcommon.so.8
>#3  0x005e5303 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x006f6754 in qpid::sys::Dispatcher::run() () from /usr/lib/libqpidcommon.so.8
>#5  0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#6  0x00d28852 in start_thread () from /lib/libpthread.so.0
>#7  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 9 (Thread 12624):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00c93106 in epoll_wait () from /lib/libc.so.6
>#2  0x005e46da in qpid::sys::Poller::wait(qpid::sys::Duration) () from /usr/lib/libqpidcommon.so.8
>#3  0x005e5303 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x006f6754 in qpid::sys::Dispatcher::run() () from /usr/lib/libqpidcommon.so.8
>#5  0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#6  0x00d28852 in start_thread () from /lib/libpthread.so.0
>#7  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 8 (Thread 12625):
>#0  0x00d288d9 in start_thread () from /lib/libpthread.so.0
>#1  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 7 (Thread 12629):
>#0  0x00d288d9 in start_thread () from /lib/libpthread.so.0
>#1  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 6 (Thread 12632):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00c93106 in epoll_wait () from /lib/libc.so.6
>#2  0x005e46da in qpid::sys::Poller::wait(qpid::sys::Duration) () from /usr/lib/libqpidcommon.so.8
>#3  0x005e5303 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x006f6754 in qpid::sys::Dispatcher::run() () from /usr/lib/libqpidcommon.so.8
>#5  0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#6  0x00d28852 in start_thread () from /lib/libpthread.so.0
>#7  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 5 (Thread 12633):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00c93106 in epoll_wait () from /lib/libc.so.6
>#2  0x005e46da in qpid::sys::Poller::wait(qpid::sys::Duration) () from /usr/lib/libqpidcommon.so.8
>#3  0x005e5303 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x006f6754 in qpid::sys::Dispatcher::run() () from /usr/lib/libqpidcommon.so.8
>#5  0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#6  0x00d28852 in start_thread () from /lib/libpthread.so.0
>#7  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 4 (Thread 12634):
>#0  0x00d288d9 in start_thread () from /lib/libpthread.so.0
>#1  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 3 (Thread 12636):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00c93106 in epoll_wait () from /lib/libc.so.6
>#2  0x005e46da in qpid::sys::Poller::wait(qpid::sys::Duration) () from /usr/lib/libqpidcommon.so.8
>#3  0x005e5303 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x006f6754 in qpid::sys::Dispatcher::run() () from /usr/lib/libqpidcommon.so.8
>#5  0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#6  0x00d28852 in start_thread () from /lib/libpthread.so.0
>#7  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 2 (Thread 12638):
>#0  0x0036c410 in __kernel_vsyscall ()
>#1  0x00c93106 in epoll_wait () from /lib/libc.so.6
>#2  0x005e46da in qpid::sys::Poller::wait(qpid::sys::Duration) () from /usr/lib/libqpidcommon.so.8
>#3  0x005e5303 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
>#4  0x006f6754 in qpid::sys::Dispatcher::run() () from /usr/lib/libqpidcommon.so.8
>#5  0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#6  0x00d28852 in start_thread () from /lib/libpthread.so.0
>#7  0x00c92a8e in clone () from /lib/libc.so.6
>
>Thread 1 (Thread 0xb0d5db90 (LWP 12631)):
>#0  0x009d3e83 in qpid::broker::Queue::deliver(boost::intrusive_ptr<qpid::broker::Message>) () from /usr/lib/libqpidbroker.so.8
>#1  0x03900890 in qpid::cluster::Cluster::deliverToQueue(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, qpid::sys::ScopedLock<qpid::sys::Mutex>&) () from /usr/lib/qpid/daemon/cluster.so
>#2  0x03923525 in qpid::cluster::ClusterDispatcher::deliverToQueue(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /usr/lib/qpid/daemon/cluster.so
>#3  0x00612eee in qpid::framing::AMQP_AllOperations::ClusterHandler::Invoker::visit(qpid::framing::ClusterDeliverToQueueBody const&) ()
>   from /usr/lib/libqpidcommon.so.8
>#4  0x0064989b in qpid::framing::ClusterDeliverToQueueBody::accept(qpid::framing::MethodBodyConstVisitor&) const () from /usr/lib/libqpidcommon.so.8
>#5  0x03919b7b in qpid::framing::Invoker::Result qpid::framing::invoke<qpid::cluster::ClusterDispatcher>(qpid::cluster::ClusterDispatcher&, qpid::framing::AMQBody const&) () from /usr/lib/qpid/daemon/cluster.so
>#6  0x03911dbc in qpid::cluster::Cluster::processFrame(qpid::cluster::EventFrame const&, qpid::sys::ScopedLock<qpid::sys::Mutex>&) ()
>   from /usr/lib/qpid/daemon/cluster.so
>#7  0x03912759 in qpid::cluster::Cluster::deliveredFrame(qpid::cluster::EventFrame const&) () from /usr/lib/qpid/daemon/cluster.so
>#8  0x03912c3d in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::cluster::Cluster, qpid::cluster::EventFrame const&>, boost::_bi::list2<boost::_bi::value<qpid::cluster::Cluster*>, boost::arg<1> > >, void, qpid::cluster::EventFrame const&>::invoke(boost::detail::function::any_pointer, qpid::cluster::EventFrame const&) () from /usr/lib/qpid/daemon/cluster.so
>#9  0x0391ba54 in boost::function1<void, qpid::cluster::EventFrame const&, std::allocator<void> >::operator()(qpid::cluster::EventFrame const&) const ()
>   from /usr/lib/qpid/daemon/cluster.so
>#10 0x03924c0f in qpid::cluster::PollableQueue<qpid::cluster::EventFrame>::handleBatch(std::deque<qpid::cluster::EventFrame, std::allocator<qpid::cluster::EventFrame> > const&) () from /usr/lib/qpid/daemon/cluster.so
>#11 0x03912d99 in boost::detail::function::function_obj_invoker1<boost::_bi::bind_t<std::_Deque_iterator<qpid::cluster::EventFrame, qpid::cluster::EventFrame const&, qpid::cluster::EventFrame const*>, boost::_mfi::mf1<std::_Deque_iterator<qpid::cluster::EventFrame, qpid::cluster::EventFrame const&, qpid::cluster::EventFrame const*>, qpid::cluster::PollableQueue<qpid::cluster::EventFrame>, std::deque<qpid::cluster::EventFrame, std::allocator<qpid::cluster::EventFrame> > const&>, boost::_bi::list2<boost::_bi::value<qpid::cluster::PollableQueue<qpid::cluster::EventFrame>*>, boost::arg<1> > >, std::_Deque_iterator<qpid::cluster::EventFrame, qpid::cluster::EventFrame const&, qpid::cluster::EventFrame const*>, std::deque<qpid::cluster::EventFrame, std::allocator<qpid::cluster::EventFrame> > const&>::invoke(boost::detail::function::any_pointer, std::deque<qpid::cluster::EventFrame, std::allocator<qpid::cluster::EventFrame> > const&)
>    () from /usr/lib/qpid/daemon/cluster.so
>#12 0x0391bc9b in boost::function1<std::_Deque_iterator<qpid::cluster::EventFrame, qpid::cluster::EventFrame const&, qpid::cluster::EventFrame const*>, std::deque<qpid::cluster::EventFrame, std::allocator<qpid::cluster::EventFrame> > const&, std::allocator<void> >::operator()(std::deque<qpid::cluster::EventFrame, std::allocator<qpid::cluster::EventFrame> > const&) const () from /usr/lib/qpid/daemon/cluster.so
>#13 0x03922b06 in qpid::sys::PollableQueue<qpid::cluster::EventFrame>::process() () from /usr/lib/qpid/daemon/cluster.so
>#14 0x03922cf9 in qpid::sys::PollableQueue<qpid::cluster::EventFrame>::dispatch(qpid::sys::PollableCondition&) () from /usr/lib/qpid/daemon/cluster.so
>#15 0x03912e1d in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::PollableQueue<qpid::cluster::EventFrame>, qpid::sys::PollableCondition&>, boost::_bi::list2<boost::_bi::value<qpid::sys::PollableQueue<qpid::cluster::EventFrame>*>, boost::arg<1> > >, void, qpid::sys::PollableCondition&>::invoke(boost::detail::function::any_pointer, qpid::sys::PollableCondition&) () from /usr/lib/qpid/daemon/cluster.so
>#16 0x005e1da7 in boost::function1<void, qpid::sys::PollableCondition&, std::allocator<boost::function_base> >::operator()(qpid::sys::PollableCondition&) const () from /usr/lib/libqpidcommon.so.8
>#17 0x005df9a7 in qpid::sys::PollableConditionPrivate::dispatch(qpid::sys::DispatchHandle&) () from /usr/lib/libqpidcommon.so.8
>#18 0x005e1c0d in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::PollableConditionPrivate, qpid::sys::DispatchHandle&>, boost::_bi::list2<boost::_bi::value<qpid::sys::PollableConditionPrivate*>, boost::arg<1> > >, void, qpid::sys::DispatchHandle&>::invoke(boost::detail::function::any_pointer, qpid::sys::DispatchHandle&) () from /usr/lib/libqpidcommon.so.8
>#19 0x006f3514 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator()(qpid::sys::DispatchHandle&) const ()
>   from /usr/lib/libqpidcommon.so.8
>#20 0x006f2ddf in qpid::sys::DispatchHandle::processEvent(qpid::sys::Poller::EventType) () from /usr/lib/libqpidcommon.so.8
>#21 0x005e5321 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
>#22 0x006f6754 in qpid::sys::Dispatcher::run() () from /usr/lib/libqpidcommon.so.8
>#23 0x005db601 in ?? () from /usr/lib/libqpidcommon.so.8
>#24 0x00d28852 in start_thread () from /lib/libpthread.so.0
>#25 0x00c92a8e in clone () from /lib/libc.so.6

Actions: View

Attachments on bug 872934: 637988 | 638000 | 638007 | 638008 | 638009 | 638010