Bug 499878 - modified ping example causes segfault in c++ qmf-console code
modified ping example causes segfault in c++ qmf-console code
Status: NEW
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-qmf (Show other bugs)
1.1.1
All Linux
medium Severity medium
: ---
: ---
Assigned To: Ted Ross
MRG Quality Engineering
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-05-08 13:40 EDT by Gordon Sim
Modified: 2015-11-15 20:18 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
ping client (4.15 KB, text/x-c++src)
2009-05-11 07:49 EDT, Gordon Sim
no flags Details
bz499872_fail_rhel53_x86_64_090514.tar.bz2 - issue reproducer (11.98 KB, application/x-tbz)
2009-05-14 03:54 EDT, Frantisek Reznicek
no flags Details

  None (edit)
Description Gordon Sim 2009-05-08 13:40:47 EDT
When running a modified version of the ping (only change is to loop forever) example I noticed a couple of occasions when it core dumped.


Ping Broker: localhost:5672... Result: code=0 text=OK seq=971
Ping Broker: localhost:5672... Result: code=0 text=OK seq=972
Ping Broker: localhost:5672... Result: code=0 text=OK seq=973
Ping Broker: localhost:5672... Result: code=0 text=OK seq=974
Ping Broker: localhost:5672... Result: code=0 text=OK seq=975
Ping Broker: localhost:5672... Result: code=0 text=OK seq=976
Ping Broker: localhost:5672... Result: code=4097 text=Method call 
timed out
Ping Broker: localhost:5672... Segmentation fault (core dumped)

Backtrace:
#0  0x00002b9aebdf2065 in qpid::console::Object::handleMethodResp ()
   from /usr/lib64/libqmfconsole.so.0
#1  0x00002b9aebde495f in qpid::console::Broker::received ()
   from /usr/lib64/libqmfconsole.so.0
#2  0x0000003d44ca74a8 in qpid::client::SubscriptionImpl::received ()
   from /usr/lib64/libqpidclient.so.0
#3  0x0000003d44c79695 in qpid::client::Dispatcher::run ()
   from /usr/lib64/libqpidclient.so.0
#4  0x00002b9aebde58db in qpid::console::Broker::ConnectionThread::run ()
   from /usr/lib64/libqmfconsole.so.0

Above was from latest packages and I'm not entirely sure what caused it. 

Later I saw another from an svn checkout. This is more reproducible and looks like it may be related. On restarting the broker connected to while ping is running I saw:

Ping Broker: localhost:5672... lt-ping: qpid/framing/Buffer.cpp:141: uint32_t qpid::framing::Buffer::getLong(): Assertion `position <= size' failed.
Aborted (core dumped)

and got the following backtrace:

#0  0x45b8b410 in __kernel_vsyscall ()
#1  0x45bd1069 in raise () from /lib/libc.so.6
#2  0x45bd2671 in abort () from /lib/libc.so.6
#3  0x45bca9d9 in __assert_fail () from /lib/libc.so.6
#4  0x002bb4dd in qpid::framing::Buffer::getLong (this=Variable "this" is not available.
) at qpid/framing/Buffer.cpp:141
#5  0x00f0c49f in qpid::console::ValueFactory::newValue (typeCode=3, buffer=@0xb7f01814) at qpid/console/Value.cpp:126
#6  0x00efc004 in qpid::console::SchemaArgument::decodeValue (this=0x9453328, buffer=@0xb7f01814)
    at qpid/console/Schema.cpp:57
#7  0x00ef6621 in qpid::console::Object::handleMethodResp (this=0x9458048, buffer=@0xb7f01814, sequence=1668)
    at qpid/console/Object.cpp:221
#8  0x00f02475 in qpid::console::SessionManager::handleMethodResp (this=0xbfe1b820, broker=0x92378f8, buffer=@0xb7f01814,
    sequence=1668) at qpid/console/SessionManager.cpp:345
#9  0x00eed8da in qpid::console::Broker::received (this=0x92378f8, msg=@0xb7f02190) at qpid/console/Broker.cpp:107
#10 0x009ac1ad in qpid::client::SubscriptionImpl::received (this=0x943c658, m=@0xb7f02190)
    at qpid/client/SubscriptionImpl.cpp:132
#11 0x0097d61e in qpid::client::Dispatcher::run (this=0x923a534) at qpid/client/Dispatcher.cpp:81
#12 0x009af468 in qpid::client::SubscriptionManagerImpl::run (this=0x923a510) at qpid/client/SubscriptionManagerImpl.cpp:98
#13 0x009ad4c4 in qpid::client::SubscriptionManager::run (this=0x9456d80) at qpid/client/SubscriptionManager.cpp:60
#14 0x00eea246 in qpid::console::Broker::ConnectionThread::run (this=0x92379cc) at qpid/console/Broker.cpp:190
#15 0x00284461 in qpid::sys::(anonymous namespace)::runRunnable (p=0x92379cc) at qpid/sys/posix/Thread.cpp:35
#16 0x45f0640b in start_thread () from /lib/libpthread.so.0
#17 0x45c71b7e in clone () from /lib/libc.so.6


Note that restarting the broker doesn't seem to cause this reliably with the latest 1.1.1 packaged lib and example. For the packaged version the ping process fails to get back to normal state after the broker is restarted - it does seem to reconnect (as restarting again prints a disconnected message):

Ping Broker: localhost:5672... Result: code=4097 text=Method call timed out
Ping Broker: localhost:5672... Result: code=4097 text=Method call timed out
Ping Broker: localhost:5672... 2009-may-08 13:31:42 warning Connection closed
Result: code=4097 text=Method call timed out
Ping Broker: localhost:5672... Result: code=4097 text=Method call timed out
Ping Broker: localhost:5672... Result: code=4097 text=Method call timed out

Assume this latter issue is something already fixed?
Comment 1 Gordon Sim 2009-05-11 07:49:59 EDT
Created attachment 343434 [details]
ping client
Comment 2 Frantisek Reznicek 2009-05-14 03:53:32 EDT
I was able to trigger one more occurence of this bug using my bug 499872 reproducer (bz499872_fail_rhel53_x86_64_090514.tar.bz2)

There is three node cluster on ports 5672, 10000 and 10001 running and ping client connected to broker on port 5672. After approx. 11 hours ping client crashed, See backtrace below:
 
[root@hp-xw6600-02 bz499872]# service openais restart
Stopping OpenAIS daemon (aisexec): [  OK  ]
Starting OpenAIS daemon (aisexec): [  OK  ]
[root@hp-xw6600-02 bz499872]# ./run.sh
client compile, ecode:0
client[s] ready
starting brokers in the cluster:...done
tcp        0      0 0.0.0.0:5672                0.0.0.0:*                   LISTEN      3158/qpidd
tcp        0      0 0.0.0.0:10001               0.0.0.0:*                   LISTEN      3168/qpidd
tcp        0      0 0.0.0.0:10002               0.0.0.0:*                   LISTEN      3180/qpidd
broker[s] running
running core test (ping), ecode:./run.sh: line 105:  3207 Segmentation fault      (core dumped) ./${client_app} > ./${client_app}.log 2> ./${client_app}.stderr
139
test broker[s]:3158 3168 3180 done
stopping brokers...
Test Summary:
TEST FAILED !!! (1 failures)
[root@hp-xw6600-02 bz499872]# ll
total 64724
-rw-r--r-- 1 root root        0 May 13 10:43 compilation.log
-rw------- 1 root root 34684928 May 13 21:49 core.3207
drwxr-xr-x 3 root root     4096 May 13 10:43 data_0
drwxr-xr-x 3 root root     4096 May 13 10:43 data_1
drwxr-xr-x 3 root root     4096 May 13 10:43 data_2
-rwxr-xr-x 1 root root   579364 May 13 10:43 ping
-rw-rw-r-- 1  502  502     4289 May 11 10:33 ping.cpp
-rw-rw-r-- 1  502  502     4289 May 11 10:33 ping.cpp~
lrwxrwxrwx 1  502  502       13 May 13 05:20 ping.cpp.bz -> ping.cpp.orig
-rw-rw-r-- 1  502  502     4289 May 11 10:33 ping.cpp.orig
-rw-r--r-- 1 root root 64397371 May 13 21:49 ping.log
-rw-r--r-- 1 root root      447 May 13 10:43 ping.stderr
-rw-r--r-- 1 root root     3481 May 13 21:49 qpidd_0.log
-rw-r--r-- 1 root root     3356 May 13 21:49 qpidd_1.log
-rw-r--r-- 1 root root     3085 May 13 21:49 qpidd_2.log
-rwxrwxr-x 1  502  502     3866 May 11 10:47 run.sh
-rwxrwxr-x 1  502  502     3852 May 11 10:40 run.sh~
[root@hp-xw6600-02 bz499872]# rpm -q qpidc-perftest
qpidc-perftest-0.5.752581-5.el5
[root@hp-xw6600-02 bz499872]# rpm -q qpidc-debuginfo
qpidc-debuginfo-0.5.752581-5.el5
[root@hp-xw6600-02 bz499872]# gdb `which qpidd` core.3207 <<< "thread apply all bt" > core.3207.ctxt

warning: core file may not match specified executable file.
[root@hp-xw6600-02 bz499872]# ls
compilation.log  core.3207.ctxt  data_1  ping      ping.cpp~    ping.cpp.orig  ping.stderr  qpidd_1.log  run.sh
core.3207        data_0          data_2  ping.cpp  ping.cpp.bz  ping.log       qpidd_0.log  qpidd_2.log  run.sh~
[root@hp-xw6600-02 bz499872]# !nets
-bash: !nets: event not found
[root@hp-xw6600-02 bz499872]# netstat -nlp | grep qpidd
[root@hp-xw6600-02 bz499872]# vi qpidd_0.log
[root@hp-xw6600-02 bz499872]# grep error *.log
[root@hp-xw6600-02 bz499872]# gdb ./ping core.3207 <<< "thread apply all bt" > core.3207.ctxt
[root@hp-xw6600-02 bz499872]# vi core.3207.ctxt
[root@hp-xw6600-02 bz499872]# vi ping.stderr
[root@hp-xw6600-02 bz499872]# vi ping.log
[root@hp-xw6600-02 bz499872]# vi ping.log
[root@hp-xw6600-02 bz499872]# cat core.3207.ctxt
...
Core was generated by `./ping'.
Program terminated with signal 11, Segmentation fault.
[New process 3208]
[New process 3210]
[New process 3209]
[New process 3207]
#0  qpid::console::Object::handleMethodResp (this=0xf4bad10, buffer=@0x41ed13b0, sequence=<value optimized out>)
    at /usr/include/c++/4.1.2/bits/stl_iterator.h:659
659             : _M_current(__i.base()) { }
(gdb)
Thread 4 (process 3207):
#0  0x000000353d2c56db in write () from /lib64/libc.so.6
#1  0x000000353d26ba63 in _IO_new_file_write () from /lib64/libc.so.6
#2  0x000000353d26b976 in _IO_new_do_write () from /lib64/libc.so.6
#3  0x000000353d26bfa7 in _IO_new_file_sync () from /lib64/libc.so.6
#4  0x000000353d261150 in fflush () from /lib64/libc.so.6
#5  0x000000353e28f3fb in std::ostream::flush () from /usr/lib64/libstdc++.so.6
#6  0x0000000000406433 in main_int () at ping.cpp:109
#7  0x00000000004065ac in main (argc=1, argv=0x7fff739fd338) at ping.cpp:126

Thread 3 (process 3209):
#0  0x000000353d2d3498 in epoll_wait () from /lib64/libc.so.6
#1  0x00002b6e377599ad in qpid::sys::Poller::wait (this=0xf295b30, timeout=<value optimized out>)
    at qpid/sys/epoll/EpollPoller.cpp:432
#2  0x00002b6e3775a787 in qpid::sys::Poller::run (this=0xf295b30) at qpid/sys/epoll/EpollPoller.cpp:398
#3  0x00002b6e37124edb in qpid::client::TCPConnector::run (this=0xf2954b0) at qpid/client/Connector.cpp:393
#4  0x00002b6e3775176a in runRunnable (p=0x7) at qpid/sys/posix/Thread.cpp:35
#5  0x000000353de06367 in start_thread () from /lib64/libpthread.so.0
#6  0x000000353d2d30ad in clone () from /lib64/libc.so.6

Thread 2 (process 3210):
#0  0x000000353de0a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00002b6e37137bfe in qpid::client::Dispatcher::run (this=0xf4ad850) at qpid/sys/posix/Condition.h:63
#2  0x00002b6e37141bdf in qpid::client::FailoverListener::run (this=<value optimized out>) at qpid/client/FailoverListener.cpp:70
#3  0x00002b6e3775176a in runRunnable (p=0xf4aaa7c) at qpid/sys/posix/Thread.cpp:35
#4  0x000000353de06367 in start_thread () from /lib64/libpthread.so.0
#5  0x000000353d2d30ad in clone () from /lib64/libc.so.6

Thread 1 (process 3208):
#0  qpid::console::Object::handleMethodResp (this=0xf4bad10, buffer=@0x41ed13b0, sequence=<value optimized out>)
    at /usr/include/c++/4.1.2/bits/stl_iterator.h:659
#1  0x00002b6e373ac95f in qpid::console::Broker::received (this=0xf28e100, msg=<value optimized out>)
    at qpid/console/Broker.cpp:107
#2  0x00002b6e371664a8 in qpid::client::SubscriptionImpl::received (this=0xf4ae4a0, m=@0x41ed2690)
    at qpid/client/SubscriptionImpl.cpp:124
#3  0x00002b6e37138695 in qpid::client::Dispatcher::run (this=0xf4adc40) at qpid/client/Dispatcher.cpp:80
#4  0x00002b6e373ad8db in qpid::console::Broker::ConnectionThread::run (this=0xf28e240) at qpid/console/Broker.cpp:190
#5  0x00002b6e3775176a in runRunnable (p=0x353d5519c0) at qpid/sys/posix/Thread.cpp:35
#6  0x000000353de06367 in start_thread () from /lib64/libpthread.so.0
#7  0x000000353d2d30ad in clone () from /lib64/libc.so.6
(gdb) quit


[root@hp-xw6600-02 bz499872]# cat ping.log
Ping Broker: localhost:5672... Result: code=0 text=OK seq=0
Ping Broker: localhost:5672... Result: code=0 text=OK seq=1
Ping Broker: localhost:5672... Result: code=0 text=OK seq=2
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3
Ping Broker: localhost:5672... Result: code=0 text=OK seq=4
Ping Broker: localhost:5672... Result: code=0 text=OK seq=5
Ping Broker: localhost:5672... Result: code=0 text=OK seq=6
Ping Broker: localhost:5672... Result: code=0 text=OK seq=7
Ping Broker: localhost:5672... Result: code=0 text=OK seq=8
Ping Broker: localhost:5672... Result: code=0 text=OK seq=9
Ping Broker: localhost:5672... Result: code=0 text=OK seq=10
Ping Broker: localhost:5672... Result: code=0 text=OK seq=11
Ping Broker: localhost:5672... Result: code=0 text=OK seq=12
Ping Broker: localhost:5672... Result: code=0 text=OK seq=13
Ping Broker: localhost:5672... Result: code=0 text=OK seq=14
Ping Broker: localhost:5672... Result: code=0 text=OK seq=15
...
Ping Broker: localhost:5672... Result: code=0 text=OK seq=988747
Ping Broker: localhost:5672... Result: code=0 text=OK seq=988748
Ping Broker: localhost:5672... Result: code=0 text=OK seq=988749
Ping Broker: localhost:5672... Result: code=4097 text=Method call timed out
Ping Broker: localhost:5672... Result: code=0 text=OK seq=988750
Ping Broker: localhost:5672... Result: code=0 text=OK seq=988751
Ping Broker: localhost:5672... Result: code=0 text=OK seq=988752
...
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992416
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992417
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992418
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992419
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992420
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992421
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992422
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992423
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992424
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992425
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992426
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992427
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992428
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992429
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992430
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992431
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992432
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992433
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992434
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992435
Ping Broker: localhost:5672... Result: code=0 text=OK seq=992436


[root@hp-xw6600-02 mrg1.1.2]# rpm -qa | egrep '(qpid|rhm|opena)' | sort -u
openais-0.80.3-22.el5_3.4
openais-debuginfo-0.80.3-22.el5_3.4
openais-devel-0.80.3-22.el5_3.4
python-qpid-0.5.752581-1.el5
qpidc-0.5.752581-5.el5
qpidc-debuginfo-0.5.752581-5.el5
qpidc-devel-0.5.752581-5.el5
qpidc-perftest-0.5.752581-5.el5
qpidc-rdma-0.5.752581-5.el5
qpidc-ssl-0.5.752581-5.el5
qpidd-0.5.752581-5.el5
qpidd-acl-0.5.752581-5.el5
qpidd-cluster-0.5.752581-5.el5
qpidd-devel-0.5.752581-5.el5
qpid-dotnet-0.4.738274-2.el5
qpidd-rdma-0.5.752581-5.el5
qpidd-ssl-0.5.752581-5.el5
qpidd-xml-0.5.752581-5.el5
qpid-java-client-0.5.751061-2.el5
qpid-java-common-0.5.751061-2.el5
rhm-0.5.3206-1.el5
rhm-docs-0.5.756148-1.el5
Comment 3 Frantisek Reznicek 2009-05-14 03:54:35 EDT
Created attachment 343923 [details]
bz499872_fail_rhel53_x86_64_090514.tar.bz2 - issue reproducer
Comment 4 Frantisek Reznicek 2009-06-01 03:01:09 EDT
One more occurence of the problem on RHEL 5.3 x86_64:
[root@intel-greencity-01 bz499872]# rpm -qa | egrep '(qpid|openais)' | sort -u
openais-0.80.3-22.el5_3.7
openais-debuginfo-0.80.3-22.el5_3.7
openais-devel-0.80.3-22.el5_3.7
python-qpid-0.5.752581-1.el5
qpidc-0.5.752581-10.el5
qpidc-debuginfo-0.5.752581-10.el5
qpidc-devel-0.5.752581-10.el5
qpidc-perftest-0.5.752581-10.el5
qpidc-rdma-0.5.752581-10.el5
qpidc-ssl-0.5.752581-10.el5
qpidd-0.5.752581-10.el5
qpidd-acl-0.5.752581-10.el5
qpidd-cluster-0.5.752581-10.el5
qpidd-devel-0.5.752581-10.el5
qpid-dotnet-0.4.738274-2.el5
qpidd-rdma-0.5.752581-10.el5
qpidd-ssl-0.5.752581-10.el5
qpidd-xml-0.5.752581-10.el5
qpid-java-client-0.5.751061-4.el5
qpid-java-common-0.5.751061-4.el5

...................................................................................................................................................................................................................................................................................................................................................../run.sh: line 109: 22867 Segmentation fault      (core dumped) ./${client_app} > ./${client_app}.log 2> ./${client_app}.stderr
.Number qpidd processes:3 vs 3 expected.
ping process pid:
test broker[s]:22772 22797 22824 done
stopping brokers...
egrep: qpidd*.ecode: No such file or directory
ERROR: brokers did not exited cleanly
Test Summary:
TEST FAILED !!! (1 failures)
You have new mail in /var/spool/mail/root
[root@intel-greencity-01 bz499872]# ls
compilation.log  data_0  ping       ping.cpp.bz    ping.log       qpidd_0.log    qpidd_2.ecode  run.sh~   run.sh.w_restarts
core.22867       data_1  ping.cpp   ping.cpp.orig  ping.stderr    qpidd_1.ecode  qpidd_2.log    run.sh.0
cores.ctxt       data_2  ping.cpp~  ping.ecode     qpidd_0.ecode  qpidd_1.log    run.sh         run.sh.1
You have new mail in /var/spool/mail/root
[root@intel-greencity-01 bz499872]# file core.22867
core.22867: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV), SVR4-style, from 'ping'
[root@intel-greencity-01 bz499872]# gdb ./ping core.22867
GNU gdb Fedora (6.8-27.el5)
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu"...

warning: .dynamic section for "/usr/lib64/libqpidclient.so.0" is not at the expected address
...
Loaded symbols for /usr/lib64/sasl2/libanonymous.so.2
Core was generated by `./ping'.
Program terminated with signal 11, Segmentation fault.
[New process 22868]
[New process 22876]
[New process 22869]
[New process 22867]
#0  qpid::console::Object::handleMethodResp (this=0xf097340, buffer=@0x421b13b0, sequence=<value optimized out>)
    at /usr/include/c++/4.1.2/bits/stl_iterator.h:659
659             : _M_current(__i.base()) { }
(gdb) thread apply all bt

Thread 4 (process 22867):
#0  0x000000370040a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00002b68a423932d in qpid::sys::Waitable::wait (this=<value optimized out>) at qpid/sys/posix/Condition.h:63
#2  0x00002b68a4232a50 in qpid::client::SessionImpl::waitForCompletionImpl (this=<value optimized out>, id=<value optimized out>)
    at qpid/client/SessionImpl.cpp:171
#3  0x00002b68a4232aa0 in qpid::client::SessionImpl::waitForCompletion (this=<value optimized out>, id=<value optimized out>)
    at qpid/client/SessionImpl.cpp:164
#4  0x00002b68a421d9c8 in qpid::client::Future::wait (this=<value optimized out>, session=<value optimized out>)
    at qpid/client/Future.cpp:30
#5  0x00002b68a41de3c0 in qpid::client::no_keyword::Session_0_10::messageTransfer (this=<value optimized out>,
    destination=<value optimized out>, acceptMode=<value optimized out>, acquireMode=<value optimized out>,
    content=<value optimized out>, sync=<value optimized out>) at qpid/client/Completion.h:61
#6  0x00002b68a4487364 in qpid::console::Broker::ConnectionThread::sendBuffer (this=0xee77280, buf=<value optimized out>,
    length=<value optimized out>, exchange=@0x7fff06911d60, routingKey=@0x7fff06911d50) at gen/qpid/client/Session_0_10.h:305
#7  0x00002b68a44953c4 in qpid::console::Object::invokeMethod (this=0xf097340, name=
        {static npos = 18446744073709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x7fff069221d0 "H\r"}}, args=@0x7fff06922110, result=@0x7fff069220d0) at qpid/console/Object.cpp:189
#8  0x00000000004062aa in main_int () at ping.cpp:98
#9  0x00000000004065ac in main (argc=1, argv=0x7fff06922358) at ping.cpp:126

Thread 3 (process 22869):
#0  0x00000036ffcd3498 in epoll_wait () from /lib64/libc.so.6
#1  0x00002b68a4832c9d in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>)
    at qpid/sys/epoll/EpollPoller.cpp:432
#2  0x00002b68a4833a77 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:398
#3  0x00002b68a41fdf1b in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:393
#4  0x00002b68a482aa5a in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35
#5  0x0000003700406367 in start_thread () from /lib64/libpthread.so.0
#6  0x00000036ffcd30ad in clone () from /lib64/libc.so.6

Thread 2 (process 22876):
#0  0x000000370040a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00002b68a4210c3e in qpid::client::Dispatcher::run (this=<value optimized out>) at qpid/sys/posix/Condition.h:63
#2  0x00002b68a421ac1f in qpid::client::FailoverListener::run (this=<value optimized out>) at qpid/client/FailoverListener.cpp:70
#3  0x00002b68a482aa5a in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35
---Type <return> to continue, or q <return> to quit---
#4  0x0000003700406367 in start_thread () from /lib64/libpthread.so.0
#5  0x00000036ffcd30ad in clone () from /lib64/libc.so.6

Thread 1 (process 22868):
#0  qpid::console::Object::handleMethodResp (this=0xf097340, buffer=@0x421b13b0, sequence=<value optimized out>)
    at /usr/include/c++/4.1.2/bits/stl_iterator.h:659
#1  0x00002b68a4485aff in qpid::console::Broker::received (this=0xee77140, msg=<value optimized out>) at qpid/console/Broker.cpp:107
#2  0x00002b68a423f4f8 in qpid::client::SubscriptionImpl::received (this=<value optimized out>, m=<value optimized out>)
    at qpid/client/SubscriptionImpl.cpp:124
#3  0x00002b68a42116d5 in qpid::client::Dispatcher::run (this=<value optimized out>) at qpid/client/Dispatcher.cpp:80
#4  0x00002b68a4486a7b in qpid::console::Broker::ConnectionThread::run (this=0xee77280) at qpid/console/Broker.cpp:190
#5  0x00002b68a482aa5a in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35
#6  0x0000003700406367 in start_thread () from /lib64/libpthread.so.0
#7  0x00000036ffcd30ad in clone () from /lib64/libc.so.6
(gdb) quit
[root@intel-greencity-01 bz499872]# uname -a
Linux intel-greencity-01.rhts.bos.redhat.com 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux
[root@intel-greencity-01 bz499872]# tail -12 ping.log
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477566
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477567
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477568
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477569
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477570
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477571
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477572
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477573
Ping Broker: localhost:5672... Result: code=4097 text=Method call timed out
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477574
Ping Broker: localhost:5672... Result: code=0 text=OK seq=3477575
Ping Broker: localhost:5672...

Note You need to log in before you can comment on or make changes to this bug.