Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 580138

Summary:	c++ client library crash in qpid::client::TCPConnector::writebuff()
Product:	Red Hat Enterprise MRG	Reporter:	Frantisek Reznicek <freznice>
Component:	qpid-cpp	Assignee:	Andrew Stitcher <astitcher>
Status:	CLOSED DUPLICATE	QA Contact:	MRG Quality Engineering <mrgqe-bugs>
Severity:	high	Docs Contact:
Priority:	urgent
Version:	Development	CC:	aconway, esammons, gsim
Target Milestone:	1.3
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2010-04-07 19:30:15 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Frantisek Reznicek 2010-04-07 14:58:54 UTC

Description of problem:

qpid c++ clients quite often segfault in qpid::client::TCPConnector::writebuff().

There are already two observations with 4 segfaults showing two different paths to qpid::client::TCPConnector::writebuff() where sigsegv occurs.

a] first crash detected on rhel5.5 x86_64 (hard to reproduce)
(gdb)   2 Thread 3559  0x00002ac38e711b35 in pthread_join ()
   from /lib64/libpthread.so.0
* 1 Thread 3560  0x00002ac38d60a4e5 in qpid::client::TCPConnector::writebuff (
    this=0x733c640) at qpid/client/TCPConnector.cpp:229
(gdb)
Thread 2 (Thread 3559):
#0  0x00002ac38e711b35 in pthread_join () from /lib64/libpthread.so.0
#1  0x00002ac38d9b290d in qpid::sys::Thread::join (this=<value optimized out>)
    at qpid/sys/posix/Thread.cpp:58
#2  0x00002ac38d5bcc76 in ~IOThread () at qpid/client/ConnectionImpl.cpp:130
#3  __tcf_2 () at qpid/client/ConnectionImpl.cpp:136
#4  0x00002ac38e3e568e in __cxa_finalize () from /lib64/libc.so.6
#5  0x00002ac38d59a3b6 in __do_global_dtors_aux ()
   from /usr/lib64/libqpidclient.so.2
#6  0x0000000000000000 in ?? ()

Thread 1 (Thread 3560):
#0  0x00002ac38d60a4e5 in qpid::client::TCPConnector::writebuff (
    this=0x733c640) at qpid/client/TCPConnector.cpp:229
#1  0x00002ac38d9b06af in boost::function1<void, qpid::sys::AsynchIO&, std::allocator<boost::function_base> >::operator() (this=0xaaaaaaaa, a0=...)
    at /usr/include/boost/function/function_template.hpp:576
#2  0x00002ac38d9ae08f in qpid::sys::posix::AsynchIO::writeable (
    this=0x7344450, h=...) at qpid/sys/posix/AsynchIO.cpp:536
#3  0x00002ac38da95047 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator() (this=0xaaaaaaaa, a0=...)
    at /usr/include/boost/function/function_template.hpp:576
#4  0x00002ac38da8ff97 in qpid::sys::DispatchHandle::processEvent (
    this=0x7344458, type=READ_WRITABLE) at qpid/sys/DispatchHandle.cpp:285
#5  0x00002ac38d9bc52f in process (this=0x733c1a0) at qpid/sys/Poller.h:123
#6  qpid::sys::Poller::run (this=0x733c1a0)
    at qpid/sys/epoll/EpollPoller.cpp:483
#7  0x00002ac38d9b270a in qpid::sys::(anonymous namespace)::runRunnable (
    p=0x733b870) at qpid/sys/posix/Thread.cpp:35
#8  0x00002ac38e71073d in start_thread () from /lib64/libpthread.so.0
#9  0x00002ac38e485d1d in clone () from /lib64/libc.so.6
(gdb) quit

b] another crash from rhel 5.5 i386 (reproducible more rapidly)
(gdb)   2 Thread 2787  0x00c9a402 in __kernel_vsyscall ()
* 1 Thread 2788  0x00548686 in qpid::client::TCPConnector::writebuff (
    this=0x8cec620) at qpid/client/TCPConnector.cpp:229
(gdb)
Thread 2 (Thread 2787):
#0  0x00c9a402 in __kernel_vsyscall ()
#1  0x005dda77 in pthread_join () from /lib/libpthread.so.0
#2  0x001edec2 in qpid::sys::Thread::join (this=0x8cec610)
    at qpid/sys/posix/Thread.cpp:58
#3  0x004f20f7 in ~IOThread () at qpid/client/ConnectionImpl.cpp:130
#4  __tcf_2 () at qpid/client/ConnectionImpl.cpp:136
#5  0x007d0059 in __cxa_finalize () from /lib/libc.so.6
#6  0x004ccf04 in __do_global_dtors_aux () from /usr/lib/libqpidclient.so.2
#7  0x0059983c in _fini () from /usr/lib/libqpidclient.so.2
#8  0x0094180e in _dl_fini () from /lib/ld-linux.so.2
#9  0x007cfda9 in exit () from /lib/libc.so.6
#10 0x007b9ea4 in __libc_start_main () from /lib/libc.so.6
#11 0x080510c1 in _start ()

Thread 1 (Thread 2788):
#0  0x00548686 in qpid::client::TCPConnector::writebuff (this=0x8cec620)
    at qpid/client/TCPConnector.cpp:229
#1  0x0054d6ad in operator() (function_obj_ptr=..., a0=...)
    at /usr/include/boost/bind/mem_fn_template.hpp:149
#2  operator()<boost::_mfi::mf1<void, qpid::client::TCPConnector, qpid::sys::AsynchIO&>, boost::_bi::list1<qpid::sys::AsynchIO&> > (function_obj_ptr=...,
    a0=...) at /usr/include/boost/bind.hpp:287
#3  operator()<qpid::sys::AsynchIO> (function_obj_ptr=..., a0=...)
    at /usr/include/boost/bind/bind_template.hpp:32
#4  boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::client::TCPConnector, qpid::sys::AsynchIO&>, boost::_bi::list2<boost::_bi::value<qpid::client::TCPConnector*>, boost::arg<1> > >, void, qpid::sys::AsynchIO&>::invoke (function_obj_ptr=..., a0=...)
    at /usr/include/boost/function/function_template.hpp:136
#5  0x001eb0c7 in boost::function1<void, qpid::sys::AsynchIO&, std::allocator<boost::function_base> >::operator() (this=0x8cee0f8, a0=...)
    at /usr/include/boost/function/function_template.hpp:576
#6  0x001e84e6 in qpid::sys::posix::AsynchIO::writeable (this=0x8cee018, h=...)
    at qpid/sys/posix/AsynchIO.cpp:536
#7  0x001e9efd in operator() (function_obj_ptr=..., a0=...)
    at /usr/include/boost/bind/mem_fn_template.hpp:149
#8  operator()<boost::_mfi::mf1<void, qpid::sys::posix::AsynchIO, qpid::sys::DispatchHandle&>, boost::_bi::list1<qpid::sys::DispatchHandle&> > (
    function_obj_ptr=..., a0=...) at /usr/include/boost/bind.hpp:287
#9  operator()<qpid::sys::DispatchHandle> (function_obj_ptr=..., a0=...)
    at /usr/include/boost/bind/bind_template.hpp:32
#10 boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::posix::AsynchIO, qpid::sys::DispatchHandle&>, boost::_bi::list2<boost::_bi::value<qpid::sys::posix::AsynchIO*>, boost::arg<1> > >, void, qpid::sys::DispatchHandle&>::invoke (function_obj_ptr=..., a0=...)
    at /usr/include/boost/function/function_template.hpp:136
#11 0x002ed014 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator() (this=0x8cee030, a0=...)
    at /usr/include/boost/function/function_template.hpp:576
#12 0x002e8cb9 in qpid::sys::DispatchHandle::processEvent (this=0x8cee01c,
    type=READ_WRITABLE) at qpid/sys/DispatchHandle.cpp:285
#13 0x001f8f94 in process (this=0x8cec3d0) at qpid/sys/Poller.h:123
#14 qpid::sys::Poller::run (this=0x8cec3d0)
    at qpid/sys/epoll/EpollPoller.cpp:483
#15 0x001edc61 in qpid::sys::(anonymous namespace)::runRunnable (p=0x8cec3d0)
    at qpid/sys/posix/Thread.cpp:35
#16 0x005dc832 in start_thread () from /lib/libpthread.so.0
#17 0x00875e0e in clone () from /lib/libc.so.6
(gdb) quit



Version-Release number of selected component (if applicable):
qpid-cpp-client-0.7.929717-1.el5

How reproducible:
50-70%

Steps to Reproduce:
1. see Additional info for details
  
Actual results:
C++ client crashes.

Expected results:
C++ client should not crash.

Additional info:

Steps done:
rpm -qa | grep -E '(qpid|qmf)' | sort -u
rm -f /tmp/qpidd.log
tail  /etc/qpidd.conf
tail /etc/sasl2/qpidd.conf
sasldblistusers2 -f /var/lib/qpidd/qpidd.sasldb
service qpidd restart
sleep 2
netstat -nlp | grep qpidd

perftest --count 10 --mechanism PLAIN -b localhost --user guest --password guest >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b localhost --user guest --password guesty >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b localhost --user guesty --password guest >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b localhost --user guesty --password guesty >/dev/null ; echo $?

perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guest --password guest >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guest --password guesty >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guesty --password guest >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guesty --password guesty >/dev/null ; echo $?

perftest --count 10 --mechanism PLAIN -b $(hostname) --user guest --password guest >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b $(hostname) --user guest --password guesty >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b $(hostname) --user guesty --password guest >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b $(hostname) --user guesty --password guesty >/dev/null ; echo $?

ip=$(ip r l | grep src | awk '{print $NF}')
perftest --count 10 --mechanism PLAIN -b ${ip} --user guest --password guest >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b ${ip} --user guest --password guesty >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b ${ip} --user guesty --password guest >/dev/null ; echo $?
perftest --count 10 --mechanism PLAIN -b ${ip} --user guesty --password guesty >/dev/null ; echo $?

Comment 1 Frantisek Reznicek 2010-04-07 14:59:53 UTC

Details on first type of crash (rhel 5.5 x86_64):


[root@dhcp-30-90 ~]# uname -a
Linux dhcp-30-90.brq.redhat.com 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
[root@dhcp-30-90 ~]# ulimit -c unlimited
[root@dhcp-30-90 ~]# rpm -qa | grep -E '(qpid|qmf)' | sort -u
python-qmf-0.7.930108-1.el5
python-qpid-0.7.930108-1.el5
qmf-0.7.929717-1.el5
qmf-devel-0.7.929717-1.el5
qpid-cpp-client-0.7.929717-1.el5
qpid-cpp-client-devel-0.7.929717-1.el5
qpid-cpp-client-devel-docs-0.7.929717-1.el5
qpid-cpp-client-rdma-0.7.929717-1.el5
qpid-cpp-client-ssl-0.7.929717-1.el5
qpid-cpp-mrg-debuginfo-0.7.916826-2.el5
qpid-cpp-server-0.7.929717-1.el5
qpid-cpp-server-cluster-0.7.929717-1.el5
qpid-cpp-server-devel-0.7.929717-1.el5
qpid-cpp-server-rdma-0.7.929717-1.el5
qpid-cpp-server-ssl-0.7.929717-1.el5
qpid-cpp-server-store-0.7.929717-1.el5
qpid-cpp-server-xml-0.7.929717-1.el5
qpid-dotnet-0.4.738274-2.el5
qpid-java-client-0.7.918215-1.el5
qpid-java-common-0.7.918215-1.el5
qpid-tests-0.7.917717-4.el5
qpid-tools-0.7.930108-1.el5
rh-qpid-cpp-tests-0.7.929717-1.el5
ruby-qpid-0.7.904654-1.el5
[root@dhcp-30-90 ~]# rm -f /tmp/qpidd.log
[root@dhcp-30-90 ~]# tail  /etc/qpidd.conf
# under the License.
#
# Configuration file for qpidd. Entries are of the form:
#   name=value
#
# (Note: no spaces on either side of '='). Using default settings:
# "qpidd --help" or "man qpidd" for more details.
cluster-mechanism=ANONYMOUS
log-to-file=/tmp/qpidd.log

[root@dhcp-30-90 ~]# tail /etc/sasl2/qpidd.conf
# NOTE: The sasldb file must be readable by the user running the qpidd
# daemon, and should be readable only by that user.
#
pwcheck_method: auxprop
auxprop_plugin: sasldb
sasldb_path: /var/lib/qpidd/qpidd.sasldb

#following line stops spurious 'sql_select option missing' errors when
#cyrus-sql-sasl plugin is installed
sql_select: dummy select
[root@dhcp-30-90 ~]# sasldblistusers2 -f /var/lib/qpidd/qpidd.sasldb
guest@QPID: userPassword
[root@dhcp-30-90 ~]# service qpidd restart
Stopping Qpid AMQP daemon:                                 [  OK  ]
Starting Qpid AMQP daemon:                                 [  OK  ]
[root@dhcp-30-90 ~]# sleep 2
[root@dhcp-30-90 ~]# netstat -nlp | grep qpidd
tcp        0      0 0.0.0.0:5672                0.0.0.0:*                   LISTEN      3539/qpidd
[root@dhcp-30-90 ~]#
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b localhost --user guest --password guest >/dev/null ; echo $?
0
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b localhost --user guest --password guesty >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b localhost --user guesty --password guest >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
Segmentation fault (core dumped)
139
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b localhost --user guesty --password guesty >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]#
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guest --password guest >/dev/null ; echo $?
0
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guest --password guesty >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guesty --password guest >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guesty --password guesty >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]#
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b $(hostname) --user guest --password guest >/dev/null ; echo $?
0
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b $(hostname) --user guest --password guesty >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b $(hostname) --user guesty --password guest >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b $(hostname) --user guesty --password guesty >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]#
[root@dhcp-30-90 ~]# ip=$(ip r l | grep src | awk '{print $NF}')
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b ${ip} --user guest --password guest >/dev/null ; echo $?
0
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b ${ip} --user guest --password guesty >/dev/null ; echo $?
2010-04-07 16:25:28 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b ${ip} --user guesty --password guest >/dev/null ; echo $?
2010-04-07 16:25:29 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]# perftest --count 10 --mechanism PLAIN -b ${ip} --user guesty --password guesty >/dev/null ; echo $?
2010-04-07 16:25:29 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-30-90 ~]# file core.3559
core.3559: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV), SVR4-style, from 'perftest'



GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-23.el5)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/perftest...Reading symbols from /usr/lib/debug/usr/bin/perftest.debug...done.
done.
[New Thread 3559]
Reading symbols from /usr/lib64/libqpidclient.so.2...Reading symbols from /usr/lib/debug/usr/lib64/libqpidclient.so.2.0.0.debug...don
e.
done.
Loaded symbols for /usr/lib64/libqpidclient.so.2
Reading symbols from /usr/lib64/libqpidcommon.so.2...Reading symbols from /usr/lib/debug/usr/lib64/libqpidcommon.so.2.0.0.debug...don
e.
done.
Loaded symbols for /usr/lib64/libqpidcommon.so.2
...
Loaded symbols for /usr/lib64/sasl2/libplain.so.2
Core was generated by `perftest --count 10 --mechanism PLAIN -b localhost --user guesty --password gue'.
Program terminated with signal 11, Segmentation fault.
#0  0x00002ac38d60a4e5 in qpid::client::TCPConnector::writebuff (
    this=0x733c640) at qpid/client/TCPConnector.cpp:229
229         if (codec->canEncode()) {
(gdb) rax            0xaaaaaaaa 2863311530
rbx            0x7344500        120866048
rcx            0x2ac38d60a4a0   47019378910368
rdx            0x733c650        120833616
rsi            0x7344450        120865872
rdi            0x733b870        120830064
rbp            0x733c640        0x733c640
rsp            0x41ddc920       0x41ddc920
r8             0x41ddd940       1105058112
r9             0x733b6c8        120829640
r10            0x0      0
r11            0x2ac38d60a4a0   47019378910368
r12            0x733b870        120830064
r13            0x7344560        120866144
r14            0x0      0
r15            0x7344458        120865880
rip            0x2ac38d60a4e5   0x2ac38d60a4e5 <qpid::client::TCPConnector::writebuff(qpid::sys::AsynchIO&)+69>
eflags         0x10202  [ IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x63     99
gs             0x0      0
st0            0        (raw 0x00000000000000000000)
st1            0        (raw 0x00000000000000000000)
st2            0        (raw 0x00000000000000000000)
st3            0        (raw 0x00000000000000000000)
st4            0        (raw 0x00000000000000000000)
st5            0        (raw 0x00000000000000000000)
st6            0        (raw 0x00000000000000000000)
st7            0        (raw 0x00000000000000000000)
fctrl          0x37f    895
fstat          0x0      0
ftag           0xffff   65535
fiseg          0x0      0
fioff          0x0      0
foseg          0x0      0
fooff          0x0      0
fop            0x0      0
xmm0           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm2           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm3           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm4           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm7           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm8           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm9           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm10          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm11          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm12          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm13          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm14          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm15          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
mxcsr          0x1f80   [ IM DM ZM OM UM PM ]
(gdb) Using memory regions provided by the target.
There are no memory regions defined.
(gdb)   2 Thread 3559  0x00002ac38e711b35 in pthread_join ()
   from /lib64/libpthread.so.0
* 1 Thread 3560  0x00002ac38d60a4e5 in qpid::client::TCPConnector::writebuff (
    this=0x733c640) at qpid/client/TCPConnector.cpp:229
(gdb)
Thread 2 (Thread 3559):
#0  0x00002ac38e711b35 in pthread_join () from /lib64/libpthread.so.0
#1  0x00002ac38d9b290d in qpid::sys::Thread::join (this=<value optimized out>)
    at qpid/sys/posix/Thread.cpp:58
#2  0x00002ac38d5bcc76 in ~IOThread () at qpid/client/ConnectionImpl.cpp:130
#3  __tcf_2 () at qpid/client/ConnectionImpl.cpp:136
#4  0x00002ac38e3e568e in __cxa_finalize () from /lib64/libc.so.6
#5  0x00002ac38d59a3b6 in __do_global_dtors_aux ()
   from /usr/lib64/libqpidclient.so.2
#6  0x0000000000000000 in ?? ()

Thread 1 (Thread 3560):
#0  0x00002ac38d60a4e5 in qpid::client::TCPConnector::writebuff (
    this=0x733c640) at qpid/client/TCPConnector.cpp:229
#1  0x00002ac38d9b06af in boost::function1<void, qpid::sys::AsynchIO&, std::allocator<boost::function_base> >::operator() (this=0xaaaaaaaa, a0=...)
    at /usr/include/boost/function/function_template.hpp:576
#2  0x00002ac38d9ae08f in qpid::sys::posix::AsynchIO::writeable (
    this=0x7344450, h=...) at qpid/sys/posix/AsynchIO.cpp:536
#3  0x00002ac38da95047 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator() (this=0xaaaaaaaa, a0=...)
    at /usr/include/boost/function/function_template.hpp:576
#4  0x00002ac38da8ff97 in qpid::sys::DispatchHandle::processEvent (
    this=0x7344458, type=READ_WRITABLE) at qpid/sys/DispatchHandle.cpp:285
#5  0x00002ac38d9bc52f in process (this=0x733c1a0) at qpid/sys/Poller.h:123
#6  qpid::sys::Poller::run (this=0x733c1a0)
    at qpid/sys/epoll/EpollPoller.cpp:483
#7  0x00002ac38d9b270a in qpid::sys::(anonymous namespace)::runRunnable (
    p=0x733b870) at qpid/sys/posix/Thread.cpp:35
#8  0x00002ac38e71073d in start_thread () from /lib64/libpthread.so.0
#9  0x00002ac38e485d1d in clone () from /lib64/libc.so.6
(gdb) quit

Comment 2 Frantisek Reznicek 2010-04-07 15:00:53 UTC

ad b] Details on second type of crash (rhel 5.5 i386):

[root@dhcp-31-88 ~]# ulimit -c unlimited
[root@dhcp-31-88 ~]# rpm -qa | grep -E '(qpid|qmf)' | sort -u
python-qmf-0.7.930108-1.el5
python-qpid-0.7.930108-1.el5
qmf-0.7.908272-1.el5
qmf-0.7.929717-1.el5
qmf-devel-0.7.929717-1.el5
qpid-cpp-client-0.7.929717-1.el5
qpid-cpp-client-devel-0.7.929717-1.el5
qpid-cpp-client-devel-docs-0.7.929717-1.el5
qpid-cpp-client-rdma-0.7.929717-1.el5
qpid-cpp-client-ssl-0.7.929717-1.el5
qpid-cpp-mrg-debuginfo-0.7.916826-2.el5
qpid-cpp-server-0.7.929717-1.el5
qpid-cpp-server-cluster-0.7.929717-1.el5
qpid-cpp-server-devel-0.7.929717-1.el5
qpid-cpp-server-rdma-0.7.929717-1.el5
qpid-cpp-server-ssl-0.7.929717-1.el5
qpid-cpp-server-store-0.7.929717-1.el5
qpid-cpp-server-xml-0.7.929717-1.el5
qpid-dotnet-0.4.738274-2.el5
qpid-java-client-0.7.918215-1.el5
qpid-java-common-0.7.918215-1.el5
qpid-tests-0.7.917717-4.el5
qpid-tools-0.7.930108-1.el5
rh-qpid-cpp-tests-0.7.929717-1.el5
ruby-qpid-0.7.904654-1.el5
[root@dhcp-31-88 ~]# rm -f /tmp/qpidd.log
[root@dhcp-31-88 ~]# tail  /etc/qpidd.conf
# specific language governing permissions and limitations
# under the License.
#
# Configuration file for qpidd. Entries are of the form:
#   name=value
#
# (Note: no spaces on either side of '='). Using default settings:
# "qpidd --help" or "man qpidd" for more details.
cluster-mechanism=ANONYMOUS
log-to-file=/tmp/qpidd.log
[root@dhcp-31-88 ~]# tail /etc/sasl2/qpidd.conf
# NOTE: The sasldb file must be readable by the user running the qpidd
# daemon, and should be readable only by that user.
#
pwcheck_method: auxprop
auxprop_plugin: sasldb
sasldb_path: /var/lib/qpidd/qpidd.sasldb

#following line stops spurious 'sql_select option missing' errors when
#cyrus-sql-sasl plugin is installed
sql_select: dummy select
[root@dhcp-31-88 ~]# sasldblistusers2 -f /var/lib/qpidd/qpidd.sasldb
guest@QPID: userPassword
[root@dhcp-31-88 ~]# service qpidd restart
Stopping Qpid AMQP daemon:                                 [  OK  ]
Starting Qpid AMQP daemon:                                 [  OK  ]
[root@dhcp-31-88 ~]# sleep 2
[root@dhcp-31-88 ~]# netstat -nlp | grep qpidd
tcp        0      0 0.0.0.0:5672                0.0.0.0:*                   LISTEN      2748/qpidd
[root@dhcp-31-88 ~]#
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b localhost --user guest --password guest >/dev/null ; echo $?
0
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b localhost --user guest --password guesty >/dev/null ; echo $?
2010-04-07 16:25:34 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b localhost --user guesty --password guest >/dev/null ; echo $?
2010-04-07 16:25:34 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b localhost --user guesty --password guesty >/dev/null ; echo $?
2010-04-07 16:25:35 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-31-88 ~]#
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guest --password guest >/dev/null ; echo $?
0
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guest --password guesty >/dev/null ; echo $?
2010-04-07 16:25:36 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guesty --password guest >/dev/null ; echo $?
2010-04-07 16:25:36 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b 127.0.0.1 --user guesty --password guesty >/dev/null ; echo $?
2010-04-07 16:25:36 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-31-88 ~]#
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b $(hostname) --user guest --password guest >/dev/null ; echo $?
0
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b $(hostname) --user guest --password guesty >/dev/null ; echo $?
2010-04-07 16:25:37 warning Broker closed connection: 320, connection-forced: Authentication failed
Segmentation fault (core dumped)
139
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b $(hostname) --user guesty --password guest >/dev/null ; echo $?
2010-04-07 16:25:38 warning Broker closed connection: 320, connection-forced: Authentication failed
Segmentation fault (core dumped)
139
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b $(hostname) --user guesty --password guesty >/dev/null ; echo $?
2010-04-07 16:25:38 warning Broker closed connection: 320, connection-forced: Authentication failed
Segmentation fault (core dumped)
139
[root@dhcp-31-88 ~]#
[root@dhcp-31-88 ~]# ip=$(ip r l | grep src | awk '{print $NF}')
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b ${ip} --user guest --password guest >/dev/null ; echo $?
0
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b ${ip} --user guest --password guesty >/dev/null ; echo $?
2010-04-07 16:25:39 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b ${ip} --user guesty --password guest >/dev/null ; echo $?
2010-04-07 16:25:40 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-31-88 ~]# perftest --count 10 --mechanism PLAIN -b ${ip} --user guesty --password guesty >/dev/null ; echo $?
2010-04-07 16:25:40 warning Broker closed connection: 320, connection-forced: Authentication failed
1
[root@dhcp-31-88 ~]# uname -a
Linux dhcp-31-88.brq.redhat.com 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:43 EDT 2010 i686 i686 i386 GNU/Linux
[root@dhcp-31-88 ~]# 


GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-23.el5)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/perftest...Reading symbols from /usr/lib/debug/usr/bin/perftest.debug...done.
done.
[New Thread 2787]

warning: .dynamic section for "/usr/lib/libstdc++.so.6" is not at the expected address

warning: difference appears to be caused by prelink, adjusting expectations
Reading symbols from /usr/lib/libqpidclient.so.2...Reading symbols from /usr/lib/debug/usr/lib/libqpidclient.so.2.0.0.debug...done.
done.
Loaded symbols for /usr/lib/libqpidclient.so.2
Reading symbols from /usr/lib/libqpidcommon.so.2...Reading symbols from /usr/lib/debug/usr/lib/libqpidcommon.so.2.0.0.debug...done.
done.
...
Loaded symbols for /usr/lib/sasl2/liblogin.so.2
Core was generated by `perftest --count 10 --mechanism PLAIN -b dhcp-31-88.brq.redhat.com --user guest'.
Program terminated with signal 11, Segmentation fault.
#0  0x00548686 in qpid::client::TCPConnector::writebuff (this=0x8cec620)
    at qpid/client/TCPConnector.cpp:229
229         if (codec->canEncode()) {
(gdb) eax            0x0        0
ecx            0x548650 5539408
edx            0x8cec620        147768864
ebx            0x5d2b14 6105876
esp            0xb7f97dd0       0xb7f97dd0
ebp            0xb7f97df8       0xb7f97df8
esi            0x8cece58        147770968
edi            0x8cec628        147768872
eip            0x548686 0x548686 <qpid::client::TCPConnector::writebuff(qpid::sys::AsynchIO&)+54>
eflags         0x206    [ PF IF ]
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0x33     51
st0            0        (raw 0x00000000000000000000)
st1            0        (raw 0x00000000000000000000)
st2            0        (raw 0x00000000000000000000)
st3            0        (raw 0x00000000000000000000)
st4            0        (raw 0x00000000000000000000)
st5            0        (raw 0x00000000000000000000)
st6            0        (raw 0x00000000000000000000)
st7            0        (raw 0x00000000000000000000)
fctrl          0x0      0
fstat          0x0      0
ftag           0x0      0
fiseg          0x0      0
fioff          0x0      0
foseg          0x0      0
fooff          0x0      0
fop            0x0      0
xmm0           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm2           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm3           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm4           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm7           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
mxcsr          0x0      [ ]
mm0            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm1            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm2            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm3            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm4            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm5            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm6            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm7            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
(gdb) Using memory regions provided by the target.
There are no memory regions defined.
(gdb)   2 Thread 2787  0x00c9a402 in __kernel_vsyscall ()
* 1 Thread 2788  0x00548686 in qpid::client::TCPConnector::writebuff (
    this=0x8cec620) at qpid/client/TCPConnector.cpp:229
(gdb)
Thread 2 (Thread 2787):
#0  0x00c9a402 in __kernel_vsyscall ()
#1  0x005dda77 in pthread_join () from /lib/libpthread.so.0
#2  0x001edec2 in qpid::sys::Thread::join (this=0x8cec610)
    at qpid/sys/posix/Thread.cpp:58
#3  0x004f20f7 in ~IOThread () at qpid/client/ConnectionImpl.cpp:130
#4  __tcf_2 () at qpid/client/ConnectionImpl.cpp:136
#5  0x007d0059 in __cxa_finalize () from /lib/libc.so.6
#6  0x004ccf04 in __do_global_dtors_aux () from /usr/lib/libqpidclient.so.2
#7  0x0059983c in _fini () from /usr/lib/libqpidclient.so.2
#8  0x0094180e in _dl_fini () from /lib/ld-linux.so.2
#9  0x007cfda9 in exit () from /lib/libc.so.6
#10 0x007b9ea4 in __libc_start_main () from /lib/libc.so.6
#11 0x080510c1 in _start ()

Thread 1 (Thread 2788):
#0  0x00548686 in qpid::client::TCPConnector::writebuff (this=0x8cec620)
    at qpid/client/TCPConnector.cpp:229
#1  0x0054d6ad in operator() (function_obj_ptr=..., a0=...)
    at /usr/include/boost/bind/mem_fn_template.hpp:149
#2  operator()<boost::_mfi::mf1<void, qpid::client::TCPConnector, qpid::sys::AsynchIO&>, boost::_bi::list1<qpid::sys::AsynchIO&> > (function_obj_ptr=...,
    a0=...) at /usr/include/boost/bind.hpp:287
#3  operator()<qpid::sys::AsynchIO> (function_obj_ptr=..., a0=...)
    at /usr/include/boost/bind/bind_template.hpp:32
#4  boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::client::TCPConnector, qpid::sys::AsynchIO&>, boost::_bi::list2<boost::_bi::value<qpid::client::TCPConnector*>, boost::arg<1> > >, void, qpid::sys::AsynchIO&>::invoke (function_obj_ptr=..., a0=...)
    at /usr/include/boost/function/function_template.hpp:136
#5  0x001eb0c7 in boost::function1<void, qpid::sys::AsynchIO&, std::allocator<boost::function_base> >::operator() (this=0x8cee0f8, a0=...)
    at /usr/include/boost/function/function_template.hpp:576
#6  0x001e84e6 in qpid::sys::posix::AsynchIO::writeable (this=0x8cee018, h=...)
    at qpid/sys/posix/AsynchIO.cpp:536
#7  0x001e9efd in operator() (function_obj_ptr=..., a0=...)
    at /usr/include/boost/bind/mem_fn_template.hpp:149
#8  operator()<boost::_mfi::mf1<void, qpid::sys::posix::AsynchIO, qpid::sys::DispatchHandle&>, boost::_bi::list1<qpid::sys::DispatchHandle&> > (
    function_obj_ptr=..., a0=...) at /usr/include/boost/bind.hpp:287
#9  operator()<qpid::sys::DispatchHandle> (function_obj_ptr=..., a0=...)
    at /usr/include/boost/bind/bind_template.hpp:32
#10 boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::posix::AsynchIO, qpid::sys::DispatchHandle&>, boost::_bi::list2<boost::_bi::value<qpid::sys::posix::AsynchIO*>, boost::arg<1> > >, void, qpid::sys::DispatchHandle&>::invoke (function_obj_ptr=..., a0=...)
    at /usr/include/boost/function/function_template.hpp:136
#11 0x002ed014 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator() (this=0x8cee030, a0=...)
    at /usr/include/boost/function/function_template.hpp:576
#12 0x002e8cb9 in qpid::sys::DispatchHandle::processEvent (this=0x8cee01c,
    type=READ_WRITABLE) at qpid/sys/DispatchHandle.cpp:285
#13 0x001f8f94 in process (this=0x8cec3d0) at qpid/sys/Poller.h:123
#14 qpid::sys::Poller::run (this=0x8cec3d0)
    at qpid/sys/epoll/EpollPoller.cpp:483
#15 0x001edc61 in qpid::sys::(anonymous namespace)::runRunnable (p=0x8cec3d0)
    at qpid/sys/posix/Thread.cpp:35
#16 0x005dc832 in start_thread () from /lib/libpthread.so.0
#17 0x00875e0e in clone () from /lib/libc.so.6
(gdb) quit

Comment 3 Frantisek Reznicek 2010-04-07 15:02:57 UTC

This bug appears to me as side-effect of the bug 574463 fix.

Comment 4 Gordon Sim 2010-04-07 15:33:40 UTC

The changes made to fix bug 574463 only affected the new client API whereas these are triggered using perftest, so I don't believe it is related to that change.

Comment 5 Alan Conway 2010-04-07 19:30:15 UTC


*** This bug has been marked as a duplicate of bug 568831 ***