Bug 501668
| Summary: | perftest --unique-data yes --qt 1 --durable yes --npubs 2 --nsubs 1 -tx 0 --async-commit no hangs when run against standalone broker | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise MRG | Reporter: | Frantisek Reznicek <freznice> | ||||
| Component: | qpid-cpp | Assignee: | Gordon Sim <gsim> | ||||
| Status: | CLOSED NOTABUG | QA Contact: | MRG Quality Engineering <mrgqe-bugs> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | 1.1.1 | CC: | esammons, gsim, mcressma | ||||
| Target Milestone: | 1.3 | ||||||
| Target Release: | --- | ||||||
| Hardware: | All | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2010-04-30 10:08:57 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Frantisek Reznicek
2009-05-20 09:04:02 UTC
The issue is still present on 1.1.6 packages on both RHEL 4.7 (4.8) / 5.3. The issue causes multiple perftest related automated test to hang which causes test report to FAIL. See RHEL 5.3 x86_64 data: [root@dell-per905-01 ~]# ps auxw| grep perftest root 6089 0.0 0.0 58884 548 ? S 05:03 0:00 tee /mnt/tests/distribution/MRG/Messaging/qpid_test_perftest_topic/qpid_test_perftest_topic.log root 10571 0.4 0.4 376248 36608 ? Ssl 05:10 1:48 /usr/sbin/qpidd --auth no --daemon --port 0 --log-enable info+ --log-to-file qpidd.log --data-dir /mnt/tests/distribution/MRG/Messaging/qpid_test_perftest_topic/rhts_qpidd/broker.568 --no-module-dir root 10594 0.0 0.1 615612 10052 ? Sl 05:10 0:04 /usr/bin/perftest --port 48048 --mode topic --iterations 23 --count 25000 --size 256 --nsubs 5 --summary root 10996 0.0 0.0 61144 748 pts/0 S+ 11:59 0:00 grep perftest [root@dell-per905-01 ~]# uptime 11:59:30 up 7:17, 1 user, load average: 0.03, 0.05, 0.00 [root@dell-per905-01 ~]# strace -p 10594 Process 10594 attached - interrupt to quit futex(0x2aaaac037bac, FUTEX_WAIT_PRIVATE, 11, NULL <unfinished ...> Process 10594 detached [root@dell-per905-01 ~]# strace -p 10571 Process 10571 attached - interrupt to quit rt_sigprocmask(SIG_SETMASK, ~[KILL STOP RTMIN RT_1], NULL, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], ~[KILL STOP RTMIN RT_1], 8) = 0 epoll_wait(6, <unfinished ...> Process 10571 detached [root@dell-per905-01 ~]# pstack 10594 Thread 14 (Thread 0x416fd940 (LWP 10596)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339d4682ab in qpid::client::TCPConnector::run () #4 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x42803940 (LWP 10597)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x000000339d489d8a in qpid::client::LocalQueue::get () #2 0x000000339d48a569 in qpid::client::LocalQueue::get () #3 0x000000339d48a829 in qpid::client::LocalQueue::pop () #4 0x000000000041a60d in ?? () #5 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x43204940 (LWP 10598)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339d4682ab in qpid::client::TCPConnector::run () #4 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x43c05940 (LWP 10599)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x000000339d489d8a in qpid::client::LocalQueue::get () #2 0x000000339d48a569 in qpid::client::LocalQueue::get () #3 0x000000339d48a829 in qpid::client::LocalQueue::pop () #4 0x0000000000418f79 in ?? () #5 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x44606940 (LWP 10600)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339d4682ab in qpid::client::TCPConnector::run () #4 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x45007940 (LWP 10601)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x000000339d489d8a in qpid::client::LocalQueue::get () #2 0x000000339d48a569 in qpid::client::LocalQueue::get () #3 0x000000339d48a829 in qpid::client::LocalQueue::pop () #4 0x0000000000418f79 in ?? () #5 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x45a08940 (LWP 10602)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339d4682ab in qpid::client::TCPConnector::run () #4 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x46409940 (LWP 10603)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x000000339d489d8a in qpid::client::LocalQueue::get () #2 0x000000339d48a569 in qpid::client::LocalQueue::get () #3 0x000000339d48a829 in qpid::client::LocalQueue::pop () #4 0x0000000000418f79 in ?? () #5 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x46e0a940 (LWP 10604)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339d4682ab in qpid::client::TCPConnector::run () #4 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x4780b940 (LWP 10605)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x000000339d489d8a in qpid::client::LocalQueue::get () #2 0x000000339d48a569 in qpid::client::LocalQueue::get () #3 0x000000339d48a829 in qpid::client::LocalQueue::pop () #4 0x0000000000418f79 in ?? () #5 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x40a38940 (LWP 10606)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339d4682ab in qpid::client::TCPConnector::run () #4 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x4820c940 (LWP 10607)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x000000339d489d8a in qpid::client::LocalQueue::get () #2 0x000000339d48a569 in qpid::client::LocalQueue::get () #3 0x000000339d48a829 in qpid::client::LocalQueue::pop () #4 0x0000000000418f79 in ?? () #5 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x48c0d940 (LWP 10608)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339d4682ab in qpid::client::TCPConnector::run () #4 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x2acc8746c7d0 (LWP 10594)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () #1 0x000000339d489d8a in qpid::client::LocalQueue::get () #2 0x000000339d48a569 in qpid::client::LocalQueue::get () #3 0x000000339d48a829 in qpid::client::LocalQueue::pop () #4 0x0000000000416f86 in ?? () #5 0x000000000041787a in ?? () #6 0x000000000040d695 in qpid::framing::TransferContent::getData () #7 0x000000339981d974 in __libc_start_main () from /lib64/libc.so.6 #8 0x000000000040b889 in qpid::framing::TransferContent::getData () #9 0x00007fff236548d8 in ?? () #10 0x0000000000000000 in ?? () [root@dell-per905-01 ~]# pstack 10571 Thread 20 (Thread 0x420ba940 (LWP 10572)): #0 0x000000339a00ab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () #1 0x000000339d592139 in qpid::broker::Timer::run () #2 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #3 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #4 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x42abb940 (LWP 10573)): #0 0x000000339a00ab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () #1 0x000000339d592139 in qpid::broker::Timer::run () #2 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #3 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #4 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x434bc940 (LWP 10574)): #0 0x000000339a00ab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () #1 0x000000339d592139 in qpid::broker::Timer::run () #2 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #3 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #4 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x43ebd940 (LWP 10575)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x448be940 (LWP 10576)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x452bf940 (LWP 10577)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x45cc0940 (LWP 10578)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x466c1940 (LWP 10579)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x470c2940 (LWP 10580)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x47ac3940 (LWP 10581)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x484c4940 (LWP 10582)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x48ec5940 (LWP 10583)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x498c6940 (LWP 10584)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x4a2c7940 (LWP 10585)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x4acc8940 (LWP 10586)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x4b6c9940 (LWP 10587)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x4c0ca940 (LWP 10588)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x4cacb940 (LWP 10589)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x4d4cc940 (LWP 10590)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339c573cea in ?? () from /usr/lib64/libqpidcommon.so.0 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x2b7ea998a7a0 (LWP 10571)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait () #2 0x000000339c57dc87 in qpid::sys::Poller::run () #3 0x000000339d4c9b9e in qpid::broker::Broker::run () #4 0x000000000040917b in ?? () #5 0x000000339d50631e in qpid::broker::Daemon::fork () #6 0x0000000000406a95 in ?? () #7 0x00000000004054a8 in __cxa_pure_virtual () #8 0x000000339981d974 in __libc_start_main () from /lib64/libc.so.6 #9 0x0000000000404f29 in __cxa_pure_virtual () #10 0x00007fff01136368 in ?? () #11 0x0000000000000000 in ?? () [root@dell-per905-01 ~]# gdb GNU gdb Fedora (6.8-27.el5) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". (gdb) attach 10594 Attaching to process 10594 Reading symbols from /usr/bin/perftest...Reading symbols from /usr/lib/debug/usr/bin/perftest.debug...done. done. Reading symbols from /usr/lib64/libqpidclient.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libqpidclient.so.0.1.0.debug...done. done. Loaded symbols for /usr/lib64/libqpidclient.so.0 Reading symbols from /usr/lib64/libqpidcommon.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libqpidcommon.so.0.1.0.debug...done. done. Loaded symbols for /usr/lib64/libqpidcommon.so.0 Reading symbols from /usr/lib64/libboost_program_options.so.2...done. Loaded symbols for /usr/lib64/libboost_program_options.so.2 Reading symbols from /usr/lib64/libboost_filesystem.so.2...done. Loaded symbols for /usr/lib64/libboost_filesystem.so.2 Reading symbols from /lib64/libdl.so.2...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/librt.so.1...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /usr/lib64/libsasl2.so.2...done. Loaded symbols for /usr/lib64/libsasl2.so.2 Reading symbols from /lib64/libuuid.so.1...done. Loaded symbols for /lib64/libuuid.so.1 Reading symbols from /usr/lib64/libstdc++.so.6...done. Loaded symbols for /usr/lib64/libstdc++.so.6 Reading symbols from /lib64/libm.so.6...done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/libgcc_s.so.1...done. Loaded symbols for /lib64/libgcc_s.so.1 Reading symbols from /lib64/libc.so.6...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libpthread.so.0...done. [Thread debugging using libthread_db enabled] [New Thread 0x2acc8746c7d0 (LWP 10594)] [New Thread 0x48c0d940 (LWP 10608)] [New Thread 0x4820c940 (LWP 10607)] [New Thread 0x40a38940 (LWP 10606)] [New Thread 0x4780b940 (LWP 10605)] [New Thread 0x46e0a940 (LWP 10604)] [New Thread 0x46409940 (LWP 10603)] [New Thread 0x45a08940 (LWP 10602)] [New Thread 0x45007940 (LWP 10601)] [New Thread 0x44606940 (LWP 10600)] [New Thread 0x43c05940 (LWP 10599)] [New Thread 0x43204940 (LWP 10598)] [New Thread 0x42803940 (LWP 10597)] [New Thread 0x416fd940 (LWP 10596)] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libresolv.so.2...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/libcrypt.so.1...done. Loaded symbols for /lib64/libcrypt.so.1 Reading symbols from /usr/lib64/qpid/client/sslconnector.so...Reading symbols from /usr/lib/debug/usr/lib64/qpid/client/sslconnector.so.debug...done. done. Loaded symbols for /usr/lib64/qpid/client/sslconnector.so Reading symbols from /usr/lib64/libsslcommon.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libsslcommon.so.0.1.0.debug...done. done. Loaded symbols for /usr/lib64/libsslcommon.so.0 Reading symbols from /usr/lib64/libnss3.so...done. Loaded symbols for /usr/lib64/libnss3.so Reading symbols from /usr/lib64/libssl3.so...done. Loaded symbols for /usr/lib64/libssl3.so Reading symbols from /usr/lib64/libnspr4.so...done. Loaded symbols for /usr/lib64/libnspr4.so Reading symbols from /usr/lib64/libnssutil3.so...done. Loaded symbols for /usr/lib64/libnssutil3.so Reading symbols from /usr/lib64/libplc4.so...done. Loaded symbols for /usr/lib64/libplc4.so Reading symbols from /usr/lib64/libplds4.so...done. Loaded symbols for /usr/lib64/libplds4.so Reading symbols from /usr/lib64/qpid/client/rdmaconnector.so...Reading symbols from /usr/lib/debug/usr/lib64/qpid/client/rdmaconnector.so.debug...done. done. Loaded symbols for /usr/lib64/qpid/client/rdmaconnector.so Reading symbols from /usr/lib64/librdmawrap.so.0...Reading symbols from /usr/lib/debug/usr/lib64/librdmawrap.so.0.1.0.debug...done. done. Loaded symbols for /usr/lib64/librdmawrap.so.0 Reading symbols from /usr/lib64/librdmacm.so.1...done. Loaded symbols for /usr/lib64/librdmacm.so.1 Reading symbols from /usr/lib64/libibverbs.so.1...done. Loaded symbols for /usr/lib64/libibverbs.so.1 Reading symbols from /lib64/libnss_files.so.2...done. Loaded symbols for /lib64/libnss_files.so.2 Reading symbols from /usr/lib64/sasl2/libanonymous.so.2...done. Loaded symbols for /usr/lib64/sasl2/libanonymous.so.2 Reading symbols from /usr/lib64/sasl2/libplain.so.2...done. Loaded symbols for /usr/lib64/sasl2/libplain.so.2 Reading symbols from /usr/lib64/sasl2/liblogin.so.2...done. Loaded symbols for /usr/lib64/sasl2/liblogin.so.2 Reading symbols from /usr/lib64/sasl2/libsasldb.so.2...done. Loaded symbols for /usr/lib64/sasl2/libsasldb.so.2 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 (gdb) info threads 14 Thread 0x416fd940 (LWP 10596) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 13 Thread 0x42803940 (LWP 10597) 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 12 Thread 0x43204940 (LWP 10598) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 11 Thread 0x43c05940 (LWP 10599) 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 10 Thread 0x44606940 (LWP 10600) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 9 Thread 0x45007940 (LWP 10601) 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 8 Thread 0x45a08940 (LWP 10602) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 7 Thread 0x46409940 (LWP 10603) 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 6 Thread 0x46e0a940 (LWP 10604) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 5 Thread 0x4780b940 (LWP 10605) 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 4 Thread 0x40a38940 (LWP 10606) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 3 Thread 0x4820c940 (LWP 10607) 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 2 Thread 0x48c0d940 (LWP 10608) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 * 1 Thread 0x2acc8746c7d0 (LWP 10594) 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 (gdb) thread apply all bt Thread 14 (Thread 0x416fd940 (LWP 10596)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339d4682ab in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:396 #4 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x42803940 (LWP 10597)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d489d8a in qpid::client::LocalQueue::get (this=<value optimized out>, result=<value optimized out>, timeout=<value optimized out>) at qpid/sys/posix/Condition.h:63 #2 0x000000339d48a569 in qpid::client::LocalQueue::get (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:40 #3 0x000000339d48a829 in qpid::client::LocalQueue::pop (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:36 #4 0x000000000041a60d in PublishThread::run (this=0x2aaaac0008c0) at perftest.cpp:501 #5 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x43204940 (LWP 10598)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339d4682ab in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:396 #4 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x43c05940 (LWP 10599)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d489d8a in qpid::client::LocalQueue::get (this=<value optimized out>, result=<value optimized out>, timeout=<value optimized out>) at qpid/sys/posix/Condition.h:63 #2 0x000000339d48a569 in qpid::client::LocalQueue::get (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:40 #3 0x000000339d48a829 in qpid::client::LocalQueue::pop (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:36 #4 0x0000000000418f79 in SubscribeThread::run (this=0x12a2b2a0) at perftest.cpp:602 ---Type <return> to continue, or q <return> to quit--- #5 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x44606940 (LWP 10600)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339d4682ab in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:396 #4 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x45007940 (LWP 10601)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d489d8a in qpid::client::LocalQueue::get (this=<value optimized out>, result=<value optimized out>, timeout=<value optimized out>) at qpid/sys/posix/Condition.h:63 #2 0x000000339d48a569 in qpid::client::LocalQueue::get (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:40 #3 0x000000339d48a829 in qpid::client::LocalQueue::pop (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:36 #4 0x0000000000418f79 in SubscribeThread::run (this=0x2aaaac0244f0) at perftest.cpp:602 #5 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x45a08940 (LWP 10602)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339d4682ab in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:396 #4 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x46409940 (LWP 10603)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d489d8a in qpid::client::LocalQueue::get (this=<value optimized out>, result=<value optimized out>, timeout=<value optimized out>) at qpid/sys/posix/Condition.h:63 #2 0x000000339d48a569 in qpid::client::LocalQueue::get (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:40 ---Type <return> to continue, or q <return> to quit--- #3 0x000000339d48a829 in qpid::client::LocalQueue::pop (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:36 #4 0x0000000000418f79 in SubscribeThread::run (this=0x2aaaac026640) at perftest.cpp:602 #5 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x46e0a940 (LWP 10604)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339d4682ab in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:396 #4 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x4780b940 (LWP 10605)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d489d8a in qpid::client::LocalQueue::get (this=<value optimized out>, result=<value optimized out>, timeout=<value optimized out>) at qpid/sys/posix/Condition.h:63 #2 0x000000339d48a569 in qpid::client::LocalQueue::get (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:40 #3 0x000000339d48a829 in qpid::client::LocalQueue::pop (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:36 #4 0x0000000000418f79 in SubscribeThread::run (this=0x2aaaac039890) at perftest.cpp:602 #5 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x40a38940 (LWP 10606)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339d4682ab in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:396 #4 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x4820c940 (LWP 10607)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d489d8a in qpid::client::LocalQueue::get (this=<value optimized out>, result=<value optimized out>, ---Type <return> to continue, or q <return> to quit--- timeout=<value optimized out>) at qpid/sys/posix/Condition.h:63 #2 0x000000339d48a569 in qpid::client::LocalQueue::get (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:40 #3 0x000000339d48a829 in qpid::client::LocalQueue::pop (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:36 #4 0x0000000000418f79 in SubscribeThread::run (this=0x12c35bf0) at perftest.cpp:602 #5 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #6 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #7 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x48c0d940 (LWP 10608)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339d4682ab in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:396 #4 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #5 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #6 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x2acc8746c7d0 (LWP 10594)): #0 0x000000339a00a899 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d489d8a in qpid::client::LocalQueue::get (this=<value optimized out>, result=<value optimized out>, timeout=<value optimized out>) at qpid/sys/posix/Condition.h:63 #2 0x000000339d48a569 in qpid::client::LocalQueue::get (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:40 #3 0x000000339d48a829 in qpid::client::LocalQueue::pop (this=<value optimized out>, timeout=<value optimized out>) at qpid/client/LocalQueue.cpp:36 #4 0x0000000000416f86 in Controller::process (this=<value optimized out>, n=5, lq= {queue = {px = 0x7fff236541b0, pn = {pi_ = 0x416f86}}, subscription = {<qpid::client::Handle<qpid::client::SubscriptionImpl>> = {impl = 0x3300000217}, <No data fields>}}, queue= {static npos = 18446744073709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x7fff236542d0 "x\t�\022"}}, msgFn= {<boost::function1<void,const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&,std::allocator<void> >> = {<boost::function_base> = {manager = 0x7fff23654190, functor = {obj_ptr = 0x7fff236541b0, const_obj_ptr = 0x7fff236541b0, func_ptr = 0x7fff236541b0, data = "�"}}, static args = <optimized out>, static arity = <optimized out>, invoker = 0x416f86 <Controller::process(unsigned long, qpid::client::LocalQueue, std::string, boost::function<void ()(std::string const&), std::allocator<void> >)+214>}, <No data fields>}) at perftest.cpp:368 #5 0x000000000041787a in Controller::run (this=0x7fff23654610) at perftest.cpp:409 #6 0x000000000040d695 in main (argc=-1340140960, argv=<value optimized out>) at perftest.cpp:699 (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from program: /usr/bin/perftest, process 10594 [root@dell-per905-01 ~]# gdb GNU gdb Fedora (6.8-27.el5) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". (gdb) attach 10571 Attaching to process 10571 Reading symbols from /usr/sbin/qpidd...Reading symbols from /usr/lib/debug/usr/sbin/qpidd.debug...done. done. Reading symbols from /usr/lib64/libqpidbroker.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libqpidbroker.so.0.1.0.debug...done. done. Loaded symbols for /usr/lib64/libqpidbroker.so.0 Reading symbols from /usr/lib64/libqpidcommon.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libqpidcommon.so.0.1.0.debug...done. done. Loaded symbols for /usr/lib64/libqpidcommon.so.0 Reading symbols from /usr/lib64/libboost_program_options.so.2...done. Loaded symbols for /usr/lib64/libboost_program_options.so.2 Reading symbols from /usr/lib64/libboost_filesystem.so.2...done. Loaded symbols for /usr/lib64/libboost_filesystem.so.2 Reading symbols from /lib64/libuuid.so.1...done. Loaded symbols for /lib64/libuuid.so.1 Reading symbols from /lib64/libdl.so.2...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/librt.so.1...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /usr/lib64/libsasl2.so.2...done. Loaded symbols for /usr/lib64/libsasl2.so.2 Reading symbols from /usr/lib64/libstdc++.so.6...done. Loaded symbols for /usr/lib64/libstdc++.so.6 Reading symbols from /lib64/libm.so.6...done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/libgcc_s.so.1...done. Loaded symbols for /lib64/libgcc_s.so.1 Reading symbols from /lib64/libc.so.6...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libpthread.so.0...done. [Thread debugging using libthread_db enabled] [New Thread 0x2b7ea998a7a0 (LWP 10571)] [New Thread 0x4d4cc940 (LWP 10590)] [New Thread 0x4cacb940 (LWP 10589)] [New Thread 0x4c0ca940 (LWP 10588)] [New Thread 0x4b6c9940 (LWP 10587)] [New Thread 0x4acc8940 (LWP 10586)] [New Thread 0x4a2c7940 (LWP 10585)] [New Thread 0x498c6940 (LWP 10584)] [New Thread 0x48ec5940 (LWP 10583)] [New Thread 0x484c4940 (LWP 10582)] [New Thread 0x47ac3940 (LWP 10581)] [New Thread 0x470c2940 (LWP 10580)] [New Thread 0x466c1940 (LWP 10579)] [New Thread 0x45cc0940 (LWP 10578)] [New Thread 0x452bf940 (LWP 10577)] [New Thread 0x448be940 (LWP 10576)] [New Thread 0x43ebd940 (LWP 10575)] [New Thread 0x434bc940 (LWP 10574)] [New Thread 0x42abb940 (LWP 10573)] [New Thread 0x420ba940 (LWP 10572)] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libresolv.so.2...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/libcrypt.so.1...done. Loaded symbols for /lib64/libcrypt.so.1 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 (gdb) info threads 20 Thread 0x420ba940 (LWP 10572) 0x000000339a00ab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 19 Thread 0x42abb940 (LWP 10573) 0x000000339a00ab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 18 Thread 0x434bc940 (LWP 10574) 0x000000339a00ab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 17 Thread 0x43ebd940 (LWP 10575) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 16 Thread 0x448be940 (LWP 10576) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 15 Thread 0x452bf940 (LWP 10577) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 14 Thread 0x45cc0940 (LWP 10578) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 13 Thread 0x466c1940 (LWP 10579) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 12 Thread 0x470c2940 (LWP 10580) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 11 Thread 0x47ac3940 (LWP 10581) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 10 Thread 0x484c4940 (LWP 10582) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 9 Thread 0x48ec5940 (LWP 10583) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 8 Thread 0x498c6940 (LWP 10584) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 7 Thread 0x4a2c7940 (LWP 10585) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 6 Thread 0x4acc8940 (LWP 10586) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 5 Thread 0x4b6c9940 (LWP 10587) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 4 Thread 0x4c0ca940 (LWP 10588) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 3 Thread 0x4cacb940 (LWP 10589) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 2 Thread 0x4d4cc940 (LWP 10590) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 * 1 Thread 0x2b7ea998a7a0 (LWP 10571) 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 (gdb) thread apply all bt Thread 20 (Thread 0x420ba940 (LWP 10572)): #0 0x000000339a00ab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d592139 in qpid::broker::Timer::run (this=<value optimized out>) at qpid/sys/posix/Condition.h:69 #2 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #3 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #4 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x42abb940 (LWP 10573)): #0 0x000000339a00ab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d592139 in qpid::broker::Timer::run (this=<value optimized out>) at qpid/sys/posix/Condition.h:69 #2 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #3 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #4 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x434bc940 (LWP 10574)): #0 0x000000339a00ab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000339d592139 in qpid::broker::Timer::run (this=<value optimized out>) at qpid/sys/posix/Condition.h:69 #2 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #3 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #4 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x43ebd940 (LWP 10575)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x448be940 (LWP 10576)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x452bf940 (LWP 10577)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) ---Type <return> to continue, or q <return> to quit--- at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x45cc0940 (LWP 10578)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x466c1940 (LWP 10579)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x470c2940 (LWP 10580)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x47ac3940 (LWP 10581)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x484c4940 (LWP 10582)): ---Type <return> to continue, or q <return> to quit--- #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x48ec5940 (LWP 10583)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x498c6940 (LWP 10584)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x4a2c7940 (LWP 10585)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x4acc8940 (LWP 10586)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- Thread 5 (Thread 0x4b6c9940 (LWP 10587)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x4c0ca940 (LWP 10588)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x4cacb940 (LWP 10589)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x4d4cc940 (LWP 10590)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339c573cea in runRunnable (p=<value optimized out>) at qpid/sys/posix/Thread.cpp:35 #4 0x000000339a006367 in start_thread () from /lib64/libpthread.so.0 #5 0x00000033998d30ad in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x2b7ea998a7a0 (LWP 10571)): #0 0x00000033998d3498 in epoll_wait () from /lib64/libc.so.6 #1 0x000000339c57d0dd in qpid::sys::Poller::wait (this=<value optimized out>, timeout=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:439 #2 0x000000339c57dc87 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405 #3 0x000000339d4c9b9e in qpid::broker::Broker::run (this=<value optimized out>) at qpid/broker/Broker.cpp:319 ---Type <return> to continue, or q <return> to quit--- #4 0x000000000040917b in QpiddDaemon::child (this=0x7fff01135d80) at posix/QpiddBroker.cpp:128 #5 0x000000339d50631e in qpid::broker::Daemon::fork (this=<value optimized out>) at qpid/broker/Daemon.cpp:85 #6 0x0000000000406a95 in QpiddBroker::execute (this=<value optimized out>, options=<value optimized out>) at posix/QpiddBroker.cpp:159 #7 0x00000000004054a8 in main (argc=13, argv=0x7fff01136378) at qpidd.cpp:77 (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from program: /usr/sbin/qpidd, process 10571 [02:52:06] Machine and user info: [02:52:06] os: Red Hat Enterprise Linux Server release 5.3 (Tikanga) [02:52:06] who: [02:52:06] uname: Linux dell-per905-01.rhts.bos.redhat.com 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux [02:52:06] uptime: 02:52:06 up 16 min, 0 users, load average: 0.29, 0.47, 0.44 [02:52:06] whoami: root (USER:, LOGNAME:, id:root, HOME:/) [02:52:06] date: 2009-08-13 02:52:06 1250146326 [02:52:06] pwd: /mnt/tests/distribution/MRG/Messaging/qpid_common [02:52:06] df: /dev/mapper/VolGroup00-LogVol00 57G 2.2G 51G 5% / [02:52:06] get_cpu_info():CPU information: processor : 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 vendor_id : AuthenticAMD model name : Quad-Core AMD Opteron(tm) Processor 8354 cpu MHz : 1100.000 cpu cores : 4 bogomips : 4390.41 4388.54 4388.59 4388.36 4388.22 4387.99 4387.09 4388.49 4388.43 4388.75 4388.39 4388.65 4388.60 4388.75 4388.04 4389.15 [02:52:06] Memory info: total used free shared buffers cached Mem: 8238776 996364 7242412 0 60688 754956 -/+ buffers/cache: 180720 8058056 Swap: 10289144 0 10289144 python-qpid-0.5.752581-3.el5 qpidc-0.5.752581-25.el5 qpidc-debuginfo-0.5.752581-25.el5 qpidc-devel-0.5.752581-25.el5 qpidc-perftest-0.5.752581-25.el5 qpidc-rdma-0.5.752581-25.el5 qpidc-ssl-0.5.752581-25.el5 qpidd-0.5.752581-25.el5 qpidd-acl-0.5.752581-25.el5 qpidd-cluster-0.5.752581-25.el5 qpidd-devel-0.5.752581-25.el5 qpid-dotnet-0.4.738274-2.el5 qpidd-rdma-0.5.752581-25.el5 qpidd-ssl-0.5.752581-25.el5 qpidd-xml-0.5.752581-25.el5 qpid-java-client-0.5.751061-8.el5 qpid-java-common-0.5.751061-8.el5 Another data analysis from RHEL 4.7 (similar one from 4.8 got as well):
[root@hp-bl460c-02 ~]# ps auxw | grep perftest
root 21114 0.0 0.0 49932 476 ? S 09:15 0:00 tee /mnt/tests/distribution/MRG/Messaging/qpid_test_qpidd-perftest_performance/qpid_test_qpidd-perftest_performance.log
root 26687 0.1 9.6 317600 198784 ? Sl 09:22 0:11 /usr/sbin/qpidd --data-dir /mnt/tests/distribution/MRG/Messaging/qpid_test_qpidd-perftest_performance/rhts_qpidd/20090814_091505/b.6678 --port 43356 --auth no --log-enable info+ --num-jfiles 32 --jfile-size-pgs 64 --wcache-page-size 64 --tpl-num-jfiles 32 --tpl-jfile-size-pgs 64 --tpl-wcache-page-size 64 --mgmt-enable 0
root 26749 0.0 0.0 2540 332 ? S 09:22 0:00 /usr/bin/time -f %e -o time.transcript.log perftest --iterations 1 --summary --unique-data yes --log-enable info+ -p 43356 --mode shared --qt 1 --durable yes --npubs 2 --nsubs 1 --count 200000 --size 1024 --tx 0 --async-commit no
root 26750 0.0 0.4 134872 9904 ? Sl 09:22 0:04 perftest --iterations 1 --summary --unique-data yes --log-enable info+ -p 43356 --mode shared --qt 1 --durable yes --npubs 2 --nsubs 1 --count 200000 --size 1024 --tx 0 --async-commit no
root 30325 0.0 0.0 51104 676 pts/0 S+ 11:44 0:00 grep perftest
[root@hp-bl460c-02 ~]# pstree -p 26750
perftest(26750)
[root@hp-bl460c-02 ~]# pstree -p 26687
qpidd(26687)
[root@hp-bl460c-02 ~]# strace -p 26750
Process 26750 attached - interrupt to quit
futex(0x2a97101b4c, FUTEX_WAIT, 1, NULL <unfinished ...>
Process 26750 detached
[root@hp-bl460c-02 ~]# strace -p 26687
Process 26687 attached - interrupt to quit
rt_sigprocmask(SIG_SETMASK, ~[KILL STOP RTMIN RT_1], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], ~[KILL STOP RTMIN RT_1], 8) = 0
epoll_wait(6, <unfinished ...>
Process 26687 detached
[root@hp-bl460c-02 ~]# pstack 26750
Thread 4 (Thread 1126189408 (LWP 26780)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait ()
#2 0x00000037b00c6537 in qpid::sys::Poller::run ()
#3 0x00000037b038d733 in qpid::client::TCPConnector::run ()
#4 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable ()
#5 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#6 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 3 (Thread 1136679264 (LWP 26781)):
#0 0x00000037af708b3a in pthread_cond_wait@@GLIBC_2.3.2 ()
#1 0x00000037b03a7432 in qpid::sys::BlockingQueue<boost::shared_ptr<qpid::framing::FrameSet> >::pop () from /usr/lib64/libqpidclient.so.0
#2 0x00000037b03b660a in qpid::client::LocalQueue::get ()
#3 0x00000037b03b8147 in qpid::client::LocalQueue::get ()
#4 0x00000037b03ba259 in qpid::client::LocalQueue::pop ()
#5 0x0000000000423bcd in qpid::optValue<bool> ()
#6 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable ()
#7 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#8 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 2 (Thread 1147169120 (LWP 26782)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait ()
#2 0x00000037b00c6537 in qpid::sys::Poller::run ()
#3 0x00000037b038d733 in qpid::client::TCPConnector::run ()
#4 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable ()
#5 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#6 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 1 (Thread 182894103040 (LWP 26750)):
#0 0x00000037af708b3a in pthread_cond_wait@@GLIBC_2.3.2 ()
#1 0x00000037b03a7432 in qpid::sys::BlockingQueue<boost::shared_ptr<qpid::framing::FrameSet> >::pop () from /usr/lib64/libqpidclient.so.0
#2 0x00000037b03b660a in qpid::client::LocalQueue::get ()
#3 0x00000037b03b8147 in qpid::client::LocalQueue::get ()
#4 0x00000037b03ba259 in qpid::client::LocalQueue::pop ()
#5 0x0000000000414cf1 in qpid::client::Message::~Message$delete ()
#6 0x000000000041647a in qpid::client::Message::~Message$delete ()
#7 0x000000000040a3d6 in ?? ()
#8 0x00000037af21c40b in __libc_start_main () from /lib64/tls/libc.so.6
#9 0x00000000004098ea in ?? ()
#10 0x0000007fbffff138 in ?? ()
#11 0x000000000000001c in ?? ()
#12 0x000000000000001c in ?? ()
#13 0x0000007fbffff570 in ?? ()
#14 0x0000007fbffff579 in ?? ()
#15 0x0000007fbffff586 in ?? ()
#16 0x0000007fbffff588 in ?? ()
#17 0x0000007fbffff592 in ?? ()
#18 0x0000007fbffff5a0 in ?? ()
#19 0x0000007fbffff5a4 in ?? ()
#20 0x0000007fbffff5b1 in ?? ()
#21 0x0000007fbffff5b7 in ?? ()
#22 0x0000007fbffff5ba in ?? ()
#23 0x0000007fbffff5c0 in ?? ()
#24 0x0000007fbffff5c7 in ?? ()
#25 0x0000007fbffff5ce in ?? ()
#26 0x0000007fbffff5d3 in ?? ()
#27 0x0000007fbffff5d5 in ?? ()
#28 0x0000007fbffff5df in ?? ()
#29 0x0000007fbffff5e3 in ?? ()
#30 0x0000007fbffff5eb in ?? ()
#31 0x0000007fbffff5ed in ?? ()
#32 0x0000007fbffff5f5 in ?? ()
#33 0x0000007fbffff5f7 in ?? ()
#34 0x0000007fbffff5ff in ?? ()
#35 0x0000007fbffff606 in ?? ()
#36 0x0000007fbffff60d in ?? ()
#37 0x0000007fbffff612 in ?? ()
#38 0x0000007fbffff617 in ?? ()
#39 0x0000007fbffff619 in ?? ()
#40 0x0000007fbffff628 in ?? ()
#41 0x0000000000000000 in ?? ()
[root@hp-bl460c-02 ~]# pstack 26687
Thread 8 (Thread 1084229984 (LWP 26692)):
#0 0x00000037af708d2f in pthread_cond_timedwait@@GLIBC_2.3.2 ()
#1 0x00000037b050338e in qpid::broker::Timer::run ()
#2 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable ()
#3 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#4 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 7 (Thread 1094719840 (LWP 26693)):
#0 0x00000037af708d2f in pthread_cond_timedwait@@GLIBC_2.3.2 ()
#1 0x00000037b050338e in qpid::broker::Timer::run ()
#2 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable ()
#3 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#4 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 6 (Thread 1105209696 (LWP 26702)):
#0 0x00000037af70af8b in __lll_mutex_lock_wait ()
#1 0x0000000000567698 in ?? ()
#2 0x0000002a962a5158 in vtable for mrg::msgstore::InactivityFireEvent ()
#3 0x00000037af707d54 in pthread_mutex_lock () from /lib64/tls/libpthread.so.0
#4 0x0000000000000000 in ?? ()
Thread 5 (Thread 1115699552 (LWP 26703)):
#0 0x00000037af70af8b in __lll_mutex_lock_wait ()
#1 0x00000037b047e9f0 in qpid::broker::IncompleteMessageList::process ()
#2 0x0000002a962a5350 in vtable for mrg::msgstore::JournalImpl ()
#3 0x00000037af708edb in pthread_cond_signal@@GLIBC_2.3.2 ()
#4 0x00000037b047ea2c in qpid::broker::IncompleteMessageList::enqueueComplete
#5 0x00000037b0480071 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::broker::IncompleteMessageList, boost::intrusive_ptr<qpid::broker::Message> const&>, boost::_bi::list2<boost::_bi::value<qpid::broker::IncompleteMessageList*>, boost::arg<1> > >, void, boost::intrusive_ptr<qpid::broker::Message> const&>::invoke ()
#6 0x00000037b047f3a7 in boost::function1<void, boost::intrusive_ptr<qpid::broker::Message> const&, std::allocator<void> >::operator() ()
#7 0x00000037b049e4c9 in qpid::broker::Message::allEnqueuesComplete ()
#8 0x00000037b043a829 in qpid::broker::PersistableMessage::enqueueComplete ()
#9 0x0000002a960e3e55 in mrg::msgstore::JournalImpl::wr_aio_cb ()
#10 0x0000002a9616c0e3 in mrg::journal::wmgr::get_events ()
#11 0x0000002a96167c3f in mrg::journal::wmgr::write_flush ()
#12 0x0000002a9616a24a in mrg::journal::wmgr::dequeue ()
#13 0x0000002a9613e1f5 in mrg::journal::jcntl::dequeue_data_record ()
#14 0x0000002a960eae2a in mrg::msgstore::JournalImpl::dequeue_data_record ()
#15 0x0000002a9610149d in mrg::msgstore::MessageStoreImpl::async_dequeue ()
#16 0x0000002a9610a662 in mrg::msgstore::MessageStoreImpl::dequeue ()
#17 0x00000037b04a64e0 in qpid::broker::MessageStoreModule::dequeue ()
#18 0x00000037b0428428 in qpid::broker::Queue::dequeue ()
#19 0x00000037b04d25f6 in qpid::broker::SemanticState::ConsumerImpl::deliver ()
#20 0x00000037b042ea87 in qpid::broker::Queue::dispatch ()
#21 0x00000037b04c9281 in qpid::broker::SemanticState::ConsumerImpl::doOutput
#22 0x00000037b0115783 in qpid::sys::AggregateOutput::doOutput ()
#23 0x00000037b04d2770 in qpid::broker::SemanticState::doOutput ()
#24 0x00000037b0115783 in qpid::sys::AggregateOutput::doOutput ()
#25 0x00000037b0444769 in qpid::broker::Connection::doOutput ()
#26 0x00000037b03f6752 in qpid::amqp_0_10::Connection::canEncode ()
#27 0x00000037b04c4afa in qpid::broker::SecureConnection::canEncode ()
#28 0x00000037b011748b in qpid::sys::AsynchIOHandler::idle ()
#29 0x00000037b0526e71 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::AsynchIOHandler, qpid::sys::AsynchIO&>, boost::_bi::list2<boost::_bi::value<qpid::sys::AsynchIOHandler*>, boost::arg<1> > >, void, qpid::sys::AsynchIO&>::invoke ()
#30 0x00000037b00b5477 in boost::function1<void, qpid::sys::AsynchIO&, std::allocator<boost::function_base> >::operator() () from /usr/lib64/libqpidcommon.so.0
#31 0x00000037b00b3d63 in qpid::sys::posix::AsynchIO::writeable ()
#32 0x00000037b00b8041 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::posix::AsynchIO, qpid::sys::DispatchHandle&>, boost::_bi::list2<boost::_bi::value<qpid::sys::posix::AsynchIO*>, boost::arg<1> > >, void, qpid::sys::DispatchHandle&>::invoke ()
#33 0x00000037b011e9e7 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator() ()
#34 0x00000037b011d4e6 in qpid::sys::DispatchHandle::processEvent ()
#35 0x00000037b00c656c in qpid::sys::Poller::run ()
#36 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable ()
#37 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#38 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 4 (Thread 1126189408 (LWP 26704)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait ()
#2 0x00000037b00c6537 in qpid::sys::Poller::run ()
#3 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable ()
#4 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#5 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 3 (Thread 1136679264 (LWP 26705)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait ()
#2 0x00000037b00c6537 in qpid::sys::Poller::run ()
#3 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable ()
#4 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#5 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 2 (Thread 1147169120 (LWP 26706)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait ()
#2 0x00000037b00c6537 in qpid::sys::Poller::run ()
#3 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable ()
#4 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#5 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 1 (Thread 182894103008 (LWP 26687)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait ()
#2 0x00000037b00c6537 in qpid::sys::Poller::run ()
#3 0x00000037b0400848 in qpid::broker::Broker::run ()
#4 0x000000000040b842 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string<char*> ()
#5 0x00000000004085c8 in ?? ()
#6 0x00000037af21c40b in __libc_start_main () from /lib64/tls/libc.so.6
#7 0x000000000040549a in ?? ()
#8 0x0000007fbffff0e8 in ?? ()
#9 0x000000000000001c in ?? ()
#10 0x0000000000000017 in ?? ()
#11 0x0000007fbffff4f7 in ?? ()
#12 0x0000007fbffff507 in ?? ()
#13 0x0000007fbffff512 in ?? ()
#14 0x0000007fbffff57f in ?? ()
#15 0x0000007fbffff586 in ?? ()
#16 0x0000007fbffff58c in ?? ()
#17 0x0000007fbffff593 in ?? ()
#18 0x0000007fbffff596 in ?? ()
#19 0x0000007fbffff5a3 in ?? ()
#20 0x0000007fbffff5a9 in ?? ()
#21 0x0000007fbffff5b6 in ?? ()
#22 0x0000007fbffff5b9 in ?? ()
#23 0x0000007fbffff5ca in ?? ()
#24 0x0000007fbffff5cd in ?? ()
#25 0x0000007fbffff5e0 in ?? ()
#26 0x0000007fbffff5e3 in ?? ()
#27 0x0000007fbffff5f4 in ?? ()
#28 0x0000007fbffff5f7 in ?? ()
#29 0x0000007fbffff60c in ?? ()
#30 0x0000007fbffff60f in ?? ()
#31 0x0000007fbffff626 in ?? ()
#32 0x0000007fbffff629 in ?? ()
#33 0x0000007fbffff637 in ?? ()
#34 0x0000000000000000 in ?? ()
[root@hp-bl460c-02 ~]# gdb
GNU gdb Red Hat Linux (6.3.0.0-1.159.el4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
(gdb) attach 26750
Attaching to process 26750
Reading symbols from /usr/bin/perftest...Reading symbols from /usr/lib/debug/usr/bin/perftest.debug...done.
Using host libthread_db library "/lib64/tls/libthread_db.so.1".
done.
Reading symbols from /usr/lib64/libqpidclient.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libqpidclient.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib64/libqpidclient.so.0
Reading symbols from /usr/lib64/libqpidcommon.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libqpidcommon.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib64/libqpidcommon.so.0
Reading symbols from /usr/lib64/libboost_program_options.so.1...done.
Loaded symbols for /usr/lib64/libboost_program_options.so.1
Reading symbols from /usr/lib64/libboost_filesystem.so.1...done.
Loaded symbols for /usr/lib64/libboost_filesystem.so.1
Reading symbols from /lib64/tls/librt.so.1...done.
Loaded symbols for /lib64/tls/librt.so.1
Reading symbols from /usr/lib64/libsasl2.so.2...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libresolv.so.2...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /lib64/libuuid.so.1...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /usr/lib64/libstdc++.so.6...done.
Loaded symbols for /usr/lib64/libstdc++.so.6
Reading symbols from /lib64/tls/libm.so.6...done.
Loaded symbols for /lib64/tls/libm.so.6
Reading symbols from /lib64/libgcc_s.so.1...done.
Loaded symbols for /lib64/libgcc_s.so.1
Reading symbols from /lib64/tls/libc.so.6...done.
Loaded symbols for /lib64/tls/libc.so.6
Reading symbols from /lib64/tls/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 182894103040 (LWP 26750)]
[New Thread 1147169120 (LWP 26782)]
[New Thread 1136679264 (LWP 26781)]
[New Thread 1126189408 (LWP 26780)]
Loaded symbols for /lib64/tls/libpthread.so.0
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /usr/lib64/qpid/client/sslconnector.so...Reading symbols from /usr/lib/debug/usr/lib64/qpid/client/sslconnector.so.debug...done.
done.
Loaded symbols for /usr/lib64/qpid/client/sslconnector.so
Reading symbols from /usr/lib64/libsslcommon.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libsslcommon.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib64/libsslcommon.so.0
Reading symbols from /usr/lib64/libnss3.so...done.
Loaded symbols for /usr/lib64/libnss3.so
Reading symbols from /usr/lib64/libssl3.so...done.
Loaded symbols for /usr/lib64/libssl3.so
Reading symbols from /usr/lib64/libnspr4.so...done.
Loaded symbols for /usr/lib64/libnspr4.so
Reading symbols from /usr/lib64/libnssutil3.so...done.
Loaded symbols for /usr/lib64/libnssutil3.so
Reading symbols from /usr/lib64/libplc4.so...done.
Loaded symbols for /usr/lib64/libplc4.so
Reading symbols from /usr/lib64/libplds4.so...done.
Loaded symbols for /usr/lib64/libplds4.so
Reading symbols from /lib64/libnss_files.so.2...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /usr/lib64/sasl2/libcrammd5.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libcrammd5.so.2
Reading symbols from /usr/lib64/sasl2/libdigestmd5.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libdigestmd5.so.2
Reading symbols from /lib64/libcrypto.so.4...done.
Loaded symbols for /lib64/libcrypto.so.4
Reading symbols from /usr/lib64/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib64/libgssapi_krb5.so.2
Reading symbols from /usr/lib64/libkrb5.so.3...done.
Loaded symbols for /usr/lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /usr/lib64/libk5crypto.so.3...done.
Loaded symbols for /usr/lib64/libk5crypto.so.3
Reading symbols from /usr/lib64/libz.so.1...done.
Loaded symbols for /usr/lib64/libz.so.1
Reading symbols from /usr/lib64/sasl2/libsasldb.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libsasldb.so.2
Reading symbols from /usr/lib64/sasl2/libgssapiv2.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libgssapiv2.so.2
Reading symbols from /usr/lib64/sasl2/libanonymous.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libanonymous.so.2
Reading symbols from /usr/lib64/sasl2/liblogin.so.2...done.
Loaded symbols for /usr/lib64/sasl2/liblogin.so.2
Reading symbols from /usr/lib64/sasl2/libplain.so.2...done.
Loaded symbols for /usr/lib64/sasl2/libplain.so.2
0x00000037af708b3a in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
(gdb) info threads
4 Thread 1126189408 (LWP 26780) 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
3 Thread 1136679264 (LWP 26781) 0x00000037af708b3a in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
2 Thread 1147169120 (LWP 26782) 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
1 Thread 182894103040 (LWP 26750) 0x00000037af708b3a in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
(gdb) thread apply all bt
Thread 4 (Thread 1126189408 (LWP 26780)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait (this=0xa940b0, timeout=Variable "timeout" is not available.
) at qpid/sys/epoll/EpollPoller.cpp:439
#2 0x00000037b00c6537 in qpid::sys::Poller::run (this=0xa940b0) at qpid/sys/epoll/EpollPoller.cpp:405
#3 0x00000037b038d733 in qpid::client::TCPConnector::run (this=0xa94110) at qpid/client/Connector.cpp:396
#4 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable (p=0xc) at qpid/sys/posix/Thread.cpp:35
#5 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#6 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 3 (Thread 1136679264 (LWP 26781)):
#0 0x00000037af708b3a in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
#1 0x00000037b03a7432 in qpid::sys::BlockingQueue<boost::shared_ptr<qpid::framing::FrameSet> >::pop (this=0xcba810,
result=@0x43c04760, timeout=Variable "timeout" is not available.
) at ./qpid/sys/posix/Condition.h:63
#2 0x00000037b03b660a in qpid::client::LocalQueue::get (this=0x43c04ef0, result=@0x43c04a10, timeout={nanosecs = 110491})
at /usr/include/boost/shared_ptr.hpp:252
#3 0x00000037b03b8147 in qpid::client::LocalQueue::get (this=0x43c04ef0, timeout={nanosecs = 9223372036854775807})
at qpid/client/LocalQueue.cpp:40
#4 0x00000037b03ba259 in qpid::client::LocalQueue::pop (this=0x0, timeout={nanosecs = 110491}) at qpid/client/LocalQueue.cpp:36
#5 0x0000000000423bcd in SubscribeThread::run (this=0xa920a0) at perftest.cpp:611
#6 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable (p=0xcba83c) at qpid/sys/posix/Thread.cpp:35
#7 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#8 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 2 (Thread 1147169120 (LWP 26782)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait (this=0x661a90, timeout=Variable "timeout" is not available.
) at qpid/sys/epoll/EpollPoller.cpp:439
#2 0x00000037b00c6537 in qpid::sys::Poller::run (this=0x661a90) at qpid/sys/epoll/EpollPoller.cpp:405
#3 0x00000037b038d733 in qpid::client::TCPConnector::run (this=0x661940) at qpid/client/Connector.cpp:396
#4 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable (p=0xe) at qpid/sys/posix/Thread.cpp:35
#5 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#6 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 1 (Thread 182894103040 (LWP 26750)):
#0 0x00000037af708b3a in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
#1 0x00000037b03a7432 in qpid::sys::BlockingQueue<boost::shared_ptr<qpid::framing::FrameSet> >::pop (this=0x2a97101b20,
result=@0x7fbfffe3a0, timeout=Variable "timeout" is not available.
) at ./qpid/sys/posix/Condition.h:63
#2 0x00000037b03b660a in qpid::client::LocalQueue::get (this=0x7fbfffe8a0, result=@0x7fbfffe580, timeout={nanosecs = 1})
at /usr/include/boost/shared_ptr.hpp:252
#3 0x00000037b03b8147 in qpid::client::LocalQueue::get (this=0x7fbfffe8a0, timeout={nanosecs = 9223372036854775807})
at qpid/client/LocalQueue.cpp:40
#4 0x00000037b03ba259 in qpid::client::LocalQueue::pop (this=0x0, timeout={nanosecs = 1}) at qpid/client/LocalQueue.cpp:36
#5 0x0000000000414cf1 in Controller::process (this=Variable "this" is not available.
) at perftest.cpp:368
---Type <return> to continue, or q <return> to quit---
#6 0x000000000041647a in Controller::run (this=0x7fbfffec20) at ../../src/boost/detail/sp_counted_base_gcc_x86.hpp:159
#7 0x000000000040a3d6 in main (argc=Variable "argc" is not available.
) at ./../gen/qpid/client/Session_0_10.h:552
#8 0x00000037af21c40b in __libc_start_main () from /lib64/tls/libc.so.6
#9 0x00000000004098ea in _start ()
#10 0x0000007fbffff138 in ?? ()
#11 0x000000000000001c in ?? ()
#12 0x000000000000001c in ?? ()
#13 0x0000007fbffff570 in ?? ()
#14 0x0000007fbffff579 in ?? ()
#15 0x0000007fbffff586 in ?? ()
#16 0x0000007fbffff588 in ?? ()
#17 0x0000007fbffff592 in ?? ()
#18 0x0000007fbffff5a0 in ?? ()
#19 0x0000007fbffff5a4 in ?? ()
#20 0x0000007fbffff5b1 in ?? ()
#21 0x0000007fbffff5b7 in ?? ()
#22 0x0000007fbffff5ba in ?? ()
#23 0x0000007fbffff5c0 in ?? ()
#24 0x0000007fbffff5c7 in ?? ()
#25 0x0000007fbffff5ce in ?? ()
#26 0x0000007fbffff5d3 in ?? ()
#27 0x0000007fbffff5d5 in ?? ()
#28 0x0000007fbffff5df in ?? ()
#29 0x0000007fbffff5e3 in ?? ()
#30 0x0000007fbffff5eb in ?? ()
#31 0x0000007fbffff5ed in ?? ()
#32 0x0000007fbffff5f5 in ?? ()
#33 0x0000007fbffff5f7 in ?? ()
#34 0x0000007fbffff5ff in ?? ()
#35 0x0000007fbffff606 in ?? ()
#36 0x0000007fbffff60d in ?? ()
#37 0x0000007fbffff612 in ?? ()
#38 0x0000007fbffff617 in ?? ()
#39 0x0000007fbffff619 in ?? ()
#40 0x0000007fbffff628 in ?? ()
#41 0x0000000000000000 in ?? ()
[root@hp-bl460c-02 ~]# gdb
GNU gdb Red Hat Linux (6.3.0.0-1.159.el4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
(gdb) attach 26687
Attaching to process 26687
Reading symbols from /usr/sbin/qpidd...Reading symbols from /usr/lib/debug/usr/sbin/qpidd.debug...done.
Using host libthread_db library "/lib64/tls/libthread_db.so.1".
done.
Reading symbols from /usr/lib64/libqpidbroker.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libqpidbroker.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib64/libqpidbroker.so.0
Reading symbols from /usr/lib64/libqpidcommon.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libqpidcommon.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib64/libqpidcommon.so.0
Reading symbols from /usr/lib64/libboost_program_options.so.1...done.
Loaded symbols for /usr/lib64/libboost_program_options.so.1
Reading symbols from /usr/lib64/libboost_filesystem.so.1...done.
Loaded symbols for /usr/lib64/libboost_filesystem.so.1
Reading symbols from /lib64/libuuid.so.1...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /lib64/tls/librt.so.1...done.
Loaded symbols for /lib64/tls/librt.so.1
Reading symbols from /usr/lib64/libsasl2.so.2...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libresolv.so.2...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/lib64/libstdc++.so.6...done.
Loaded symbols for /usr/lib64/libstdc++.so.6
Reading symbols from /lib64/tls/libm.so.6...done.
Loaded symbols for /lib64/tls/libm.so.6
Reading symbols from /lib64/libgcc_s.so.1...done.
Loaded symbols for /lib64/libgcc_s.so.1
Reading symbols from /lib64/tls/libc.so.6...done.
Loaded symbols for /lib64/tls/libc.so.6
Reading symbols from /lib64/tls/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 182894103008 (LWP 26687)]
[New Thread 1147169120 (LWP 26706)]
[New Thread 1136679264 (LWP 26705)]
[New Thread 1126189408 (LWP 26704)]
[New Thread 1115699552 (LWP 26703)]
[New Thread 1105209696 (LWP 26702)]
[New Thread 1094719840 (LWP 26693)]
[New Thread 1084229984 (LWP 26692)]
Loaded symbols for /lib64/tls/libpthread.so.0
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /usr/lib64/qpid/daemon/replicating_listener.so...Reading symbols from /usr/lib/debug/usr/lib64/qpid/daemon/replicating_listener.so.debug...done.
done.
Loaded symbols for /usr/lib64/qpid/daemon/replicating_listener.so
Reading symbols from /usr/lib64/qpid/daemon/ssl.so...Reading symbols from /usr/lib/debug/usr/lib64/qpid/daemon/ssl.so.debug...done.
done.
Loaded symbols for /usr/lib64/qpid/daemon/ssl.so
Reading symbols from /usr/lib64/libsslcommon.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libsslcommon.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib64/libsslcommon.so.0
Reading symbols from /usr/lib64/libnss3.so...done.
Loaded symbols for /usr/lib64/libnss3.so
Reading symbols from /usr/lib64/libssl3.so...done.
Loaded symbols for /usr/lib64/libssl3.so
Reading symbols from /usr/lib64/libnspr4.so...done.
Loaded symbols for /usr/lib64/libnspr4.so
Reading symbols from /usr/lib64/libnssutil3.so...done.
Loaded symbols for /usr/lib64/libnssutil3.so
Reading symbols from /usr/lib64/libplc4.so...done.
Loaded symbols for /usr/lib64/libplc4.so
Reading symbols from /usr/lib64/libplds4.so...done.
Loaded symbols for /usr/lib64/libplds4.so
Reading symbols from /usr/lib64/qpid/daemon/msgstore.so...done.
Loaded symbols for /usr/lib64/qpid/daemon/msgstore.so
Reading symbols from /usr/lib64/tls/libdb_cxx-4.2.so...done.
Loaded symbols for /usr/lib64/tls/libdb_cxx-4.2.so
Reading symbols from /usr/lib64/libaio.so.1...done.
Loaded symbols for /usr/lib64/libaio.so.1
Reading symbols from /usr/lib64/qpid/daemon/replication_exchange.so...Reading symbols from /usr/lib/debug/usr/lib64/qpid/daemon/replication_exchange.so.debug...done.
done.
Loaded symbols for /usr/lib64/qpid/daemon/replication_exchange.so
Reading symbols from /usr/lib64/qpid/daemon/acl.so...Reading symbols from /usr/lib/debug/usr/lib64/qpid/daemon/acl.so.debug...done.
done.
Loaded symbols for /usr/lib64/qpid/daemon/acl.so
Reading symbols from /usr/lib64/qpid/daemon/xml.so...Reading symbols from /usr/lib/debug/usr/lib64/qpid/daemon/xml.so.debug...done.
done.
Loaded symbols for /usr/lib64/qpid/daemon/xml.so
Reading symbols from /usr/lib64/libxerces-c.so.28...done.
Loaded symbols for /usr/lib64/libxerces-c.so.28
Reading symbols from /usr/lib64/libxqilla.so.3...done.
Loaded symbols for /usr/lib64/libxqilla.so.3
0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
(gdb) info threads
8 Thread 1084229984 (LWP 26692) 0x00000037af708d2f in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
7 Thread 1094719840 (LWP 26693) 0x00000037af708d2f in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
6 Thread 1105209696 (LWP 26702) 0x00000037af70af8b in __lll_mutex_lock_wait () from /lib64/tls/libpthread.so.0
5 Thread 1115699552 (LWP 26703) 0x00000037af70af8b in __lll_mutex_lock_wait () from /lib64/tls/libpthread.so.0
4 Thread 1126189408 (LWP 26704) 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
3 Thread 1136679264 (LWP 26705) 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
2 Thread 1147169120 (LWP 26706) 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
1 Thread 182894103008 (LWP 26687) 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
(gdb) thread apply all bt
Thread 8 (Thread 1084229984 (LWP 26692)):
#0 0x00000037af708d2f in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
#1 0x00000037b050338e in qpid::broker::Timer::run (this=0x55e500) at ./qpid/sys/posix/Condition.h:69
#2 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable (p=0x55e534) at qpid/sys/posix/Thread.cpp:35
#3 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#4 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 7 (Thread 1094719840 (LWP 26693)):
#0 0x00000037af708d2f in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
#1 0x00000037b050338e in qpid::broker::Timer::run (this=0x55e5c8) at ./qpid/sys/posix/Condition.h:69
#2 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable (p=0x55e5fc) at qpid/sys/posix/Thread.cpp:35
#3 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#4 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 6 (Thread 1105209696 (LWP 26702)):
#0 0x00000037af70af8b in __lll_mutex_lock_wait () from /lib64/tls/libpthread.so.0
#1 0x0000000000567698 in ?? ()
#2 0x0000002a962a5158 in vtable for mrg::msgstore::InactivityFireEvent () from /usr/lib64/qpid/daemon/msgstore.so
#3 0x00000037af707d54 in pthread_mutex_lock () from /lib64/tls/libpthread.so.0
#4 0x0000000000000000 in ?? ()
Thread 5 (Thread 1115699552 (LWP 26703)):
#0 0x00000037af70af8b in __lll_mutex_lock_wait () from /lib64/tls/libpthread.so.0
#1 0x00000037b047e9f0 in qpid::broker::IncompleteMessageList::process ()
at /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../include/c++/3.4.6/streambuf:499
#2 0x0000002a962a5350 in vtable for mrg::msgstore::JournalImpl () from /usr/lib64/qpid/daemon/msgstore.so
#3 0x00000037af708edb in pthread_cond_signal@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
#4 0x00000037b047ea2c in qpid::broker::IncompleteMessageList::enqueueComplete (this=Variable "this" is not available.
) at ./qpid/sys/posix/Condition.h:78
#5 0x00000037b0480071 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::broker::IncompleteMessageList, boost::intrusive_ptr<qpid::broker::Message> const&>, boost::_bi::list2<boost::_bi::value<qpid::broker::IncompleteMessageList*>, boost::arg<1> > >, void, boost::intrusive_ptr<qpid::broker::Message> const&>::invoke (
function_obj_ptr=Variable "function_obj_ptr" is not available.
) at /usr/include/boost/bind/mem_fn_template.hpp:149
#6 0x00000037b047f3a7 in boost::function1<void, boost::intrusive_ptr<qpid::broker::Message> const&, std::allocator<void> >::operator() (this=0xfffffffffffffffc, a0=@0x0) at /usr/include/boost/function/function_template.hpp:581
#7 0x00000037b049e4c9 in qpid::broker::Message::allEnqueuesComplete (this=0xbc6070) at ./boost/detail/atomic_count_gcc.hpp:47
#8 0x00000037b043a829 in qpid::broker::PersistableMessage::enqueueComplete (this=0xbc6070)
at qpid/broker/PersistableMessage.cpp:82
#9 0x0000002a960e3e55 in mrg::msgstore::JournalImpl::wr_aio_cb () from /usr/lib64/qpid/daemon/msgstore.so
#10 0x0000002a9616c0e3 in mrg::journal::wmgr::get_events () from /usr/lib64/qpid/daemon/msgstore.so
#11 0x0000002a96167c3f in mrg::journal::wmgr::write_flush () from /usr/lib64/qpid/daemon/msgstore.so
#12 0x0000002a9616a24a in mrg::journal::wmgr::dequeue () from /usr/lib64/qpid/daemon/msgstore.so
#13 0x0000002a9613e1f5 in mrg::journal::jcntl::dequeue_data_record () from /usr/lib64/qpid/daemon/msgstore.so
---Type <return> to continue, or q <return> to quit---
#14 0x0000002a960eae2a in mrg::msgstore::JournalImpl::dequeue_data_record () from /usr/lib64/qpid/daemon/msgstore.so
#15 0x0000002a9610149d in mrg::msgstore::MessageStoreImpl::async_dequeue () from /usr/lib64/qpid/daemon/msgstore.so
#16 0x0000002a9610a662 in mrg::msgstore::MessageStoreImpl::dequeue () from /usr/lib64/qpid/daemon/msgstore.so
#17 0x00000037b04a64e0 in qpid::broker::MessageStoreModule::dequeue (this=Variable "this" is not available.
) at qpid/broker/MessageStoreModule.cpp:126
#18 0x00000037b0428428 in qpid::broker::Queue::dequeue (this=0x5af540, ctxt=0x0, msg=Variable "msg" is not available.
) at qpid/broker/Queue.cpp:727
#19 0x00000037b04d25f6 in qpid::broker::SemanticState::ConsumerImpl::deliver (this=0x6852d0, msg=@0x42801f60)
at /usr/include/boost/shared_ptr.hpp:252
#20 0x00000037b042ea87 in qpid::broker::Queue::dispatch (this=Variable "this" is not available.
) at /usr/include/boost/shared_ptr.hpp:252
#21 0x00000037b04c9281 in qpid::broker::SemanticState::ConsumerImpl::doOutput (this=0x6852d0)
at /usr/include/boost/shared_ptr.hpp:252
#22 0x00000037b0115783 in qpid::sys::AggregateOutput::doOutput (this=0x644080)
at /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../include/c++/3.4.6/bits/stl_vector.h:462
#23 0x00000037b04d2770 in qpid::broker::SemanticState::doOutput (this=Variable "this" is not available.
) at qpid/broker/SemanticState.h:213
#24 0x00000037b0115783 in qpid::sys::AggregateOutput::doOutput (this=0x643150)
at /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../include/c++/3.4.6/bits/stl_vector.h:462
#25 0x00000037b0444769 in qpid::broker::Connection::doOutput (this=0x6430e0) at qpid/broker/Connection.cpp:283
#26 0x00000037b03f6752 in qpid::amqp_0_10::Connection::canEncode (this=0x602510)
at /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../include/c++/3.4.6/memory:285
#27 0x00000037b04c4afa in qpid::broker::SecureConnection::canEncode (this=Variable "this" is not available.
)
at /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../include/c++/3.4.6/memory:285
#28 0x00000037b011748b in qpid::sys::AsynchIOHandler::idle (this=0x601260) at qpid/sys/AsynchIOHandler.cpp:202
#29 0x00000037b0526e71 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::AsynchIOHandler, qpid::sys::AsynchIO&>, boost::_bi::list2<boost::_bi::value<qpid::sys::AsynchIOHandler*>, boost::arg<1> > >, void, qpid::sys::AsynchIO&>::invoke (function_obj_ptr=Variable "function_obj_ptr" is not available.
) at /usr/include/boost/bind/mem_fn_template.hpp:149
#30 0x00000037b00b5477 in boost::function1<void, qpid::sys::AsynchIO&, std::allocator<boost::function_base> >::operator() (
this=0xfffffffffffffffc, a0=@0x0) at /usr/include/boost/function/function_template.hpp:581
#31 0x00000037b00b3d63 in qpid::sys::posix::AsynchIO::writeable (this=0x6021f0, h=@0x6021f8) at qpid/sys/posix/AsynchIO.cpp:561
#32 0x00000037b00b8041 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::posix::AsynchIO, qpid::sys::DispatchHandle&>, boost::_bi::list2<boost::_bi::value<qpid::sys::posix::AsynchIO*>, boost::arg<1> > >, void, qpid::sys::DispatchHandle&>::invoke (function_obj_ptr=Variable "function_obj_ptr" is not available.
) at /usr/include/boost/bind/mem_fn_template.hpp:149
#33 0x00000037b011e9e7 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator()
(this=0xfffffffffffffffc, a0=@0x0) at /usr/include/boost/function/function_template.hpp:581
#34 0x00000037b011d4e6 in qpid::sys::DispatchHandle::processEvent (this=0x6021f8, type=qpid::sys::Poller::INVALID)
at qpid/sys/DispatchHandle.cpp:447
#35 0x00000037b00c656c in qpid::sys::Poller::run (this=0x556620) at ./qpid/sys/Poller.h:122
#36 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable (p=0x5ac328) at qpid/sys/posix/Thread.cpp:35
#37 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#38 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 4 (Thread 1126189408 (LWP 26704)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait (this=0x556620, timeout=Variable "timeout" is not available.
) at qpid/sys/epoll/EpollPoller.cpp:439
#2 0x00000037b00c6537 in qpid::sys::Poller::run (this=0x556620) at qpid/sys/epoll/EpollPoller.cpp:405
---Type <return> to continue, or q <return> to quit---
#3 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable (p=0x6) at qpid/sys/posix/Thread.cpp:35
#4 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#5 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 3 (Thread 1136679264 (LWP 26705)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait (this=0x556620, timeout=Variable "timeout" is not available.
) at qpid/sys/epoll/EpollPoller.cpp:439
#2 0x00000037b00c6537 in qpid::sys::Poller::run (this=0x556620) at qpid/sys/epoll/EpollPoller.cpp:405
#3 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable (p=0x6) at qpid/sys/posix/Thread.cpp:35
#4 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#5 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 2 (Thread 1147169120 (LWP 26706)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait (this=0x556620, timeout=Variable "timeout" is not available.
) at qpid/sys/epoll/EpollPoller.cpp:439
#2 0x00000037b00c6537 in qpid::sys::Poller::run (this=0x556620) at qpid/sys/epoll/EpollPoller.cpp:405
#3 0x00000037b00ba5da in qpid::sys::(anonymous namespace)::runRunnable (p=0x6) at qpid/sys/posix/Thread.cpp:35
#4 0x00000037af706137 in start_thread () from /lib64/tls/libpthread.so.0
#5 0x00000037af2c9883 in clone () from /lib64/tls/libc.so.6
Thread 1 (Thread 182894103008 (LWP 26687)):
#0 0x00000037af2c9c5c in epoll_wait () from /lib64/tls/libc.so.6
#1 0x00000037b00c4f8c in qpid::sys::Poller::wait (this=0x556620, timeout=Variable "timeout" is not available.
) at qpid/sys/epoll/EpollPoller.cpp:439
#2 0x00000037b00c6537 in qpid::sys::Poller::run (this=0x556620) at qpid/sys/epoll/EpollPoller.cpp:405
#3 0x00000037b0400848 in qpid::broker::Broker::run (this=Variable "this" is not available.
) at qpid/broker/Broker.cpp:319
#4 0x000000000040b842 in QpiddBroker::execute (this=Variable "this" is not available.
) at /usr/include/boost/intrusive_ptr.hpp:119
#5 0x00000000004085c8 in main (argc=23, argv=0x7fbffff0f8)
at /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../include/c++/3.4.6/memory:301
#6 0x00000037af21c40b in __libc_start_main () from /lib64/tls/libc.so.6
#7 0x000000000040549a in _start ()
#8 0x0000007fbffff0e8 in ?? ()
#9 0x000000000000001c in ?? ()
#10 0x0000000000000017 in ?? ()
#11 0x0000007fbffff4f7 in ?? ()
#12 0x0000007fbffff507 in ?? ()
#13 0x0000007fbffff512 in ?? ()
#14 0x0000007fbffff57f in ?? ()
#15 0x0000007fbffff586 in ?? ()
#16 0x0000007fbffff58c in ?? ()
#17 0x0000007fbffff593 in ?? ()
#18 0x0000007fbffff596 in ?? ()
#19 0x0000007fbffff5a3 in ?? ()
#20 0x0000007fbffff5a9 in ?? ()
---Type <return> to continue, or q <return> to quit---
#21 0x0000007fbffff5b6 in ?? ()
#22 0x0000007fbffff5b9 in ?? ()
#23 0x0000007fbffff5ca in ?? ()
#24 0x0000007fbffff5cd in ?? ()
#25 0x0000007fbffff5e0 in ?? ()
#26 0x0000007fbffff5e3 in ?? ()
#27 0x0000007fbffff5f4 in ?? ()
#28 0x0000007fbffff5f7 in ?? ()
#29 0x0000007fbffff60c in ?? ()
#30 0x0000007fbffff60f in ?? ()
#31 0x0000007fbffff626 in ?? ()
#32 0x0000007fbffff629 in ?? ()
#33 0x0000007fbffff637 in ?? ()
#34 0x0000000000000000 in ?? ()
(gdb) The program is running. Quit anyway (and detach it)? (y or n) y
Detaching from program: /usr/sbin/qpidd, process 26687
[09:15:05] Machine and user info:
[09:15:05] os: Red Hat Enterprise Linux AS release 4 (Nahant Update 7)
[09:15:05] who:
[09:15:05] uname: Linux hp-bl460c-02.rhts.bos.redhat.com 2.6.9-78.ELsmp #1 SMP Wed Jul 9 15:46:26 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux
[09:15:05] uptime: 09:15:05 up 55 min, 0 users, load average: 0.32, 1.65, 3.30
[09:15:05] whoami: root (USER:, LOGNAME:, id:root, HOME:/)
[09:15:05] date: 2009-08-14 09:15:05 1250255705
[09:15:05] pwd: /mnt/tests/distribution/MRG/Messaging/qpid_test_qpidd-perftest_performance
[09:15:05] df: /dev/mapper/VolGroup00-LogVol00 30G 3.0G 25G 11% /
[09:15:05] get_cpu_info():CPU information:
processor : 0 1 2 3
vendor_id : GenuineIntel
model name : Intel(R) Xeon(R) CPU 5160 @ 3.00GHz
cpu MHz : 3000.113
cpu cores : 2
bogomips : 6004.33 6000.25 6000.19 6000.25
[09:15:05] Memory info:
total used free shared buffers cached
Mem: 2057008 448536 1608472 0 22776 314300
-/+ buffers/cache: 111460 1945548
Swap: 4095992 0 4095992
python-qpid-0.5.760500-5.el4
qpidc-0.5.752581-25.el4
qpidc-debuginfo-0.5.752581-25.el4
qpidc-devel-0.5.752581-25.el4
qpidc-perftest-0.5.752581-25.el4
qpidc-ssl-0.5.752581-25.el4
qpidd-0.5.752581-25.el4
qpidd-acl-0.5.752581-25.el4
qpidd-devel-0.5.752581-25.el4
qpid-dotnet-0.4.738274-2.el4
qpidd-ssl-0.5.752581-25.el4
qpidd-xml-0.5.752581-25.el4
qpid-java-client-0.5.751061-8.el4
qpid-java-common-0.5.751061-8.el4
I'm wondering whether combinations when --npubs does not match --nsubs are valid. Another point where it hangs is --qt 8.
The issue retested on the -26 packages with the same result:
[root@hp-dl2x170g6-02 MRG]# ps auxw | grep perftest
root 10901 0.0 0.0 3708 508 ? S Oct28 0:00 tee /mnt/tests/distribution/MRG/Messaging/qpid_test_qpidd-perftest_performance/qpid_test_qpidd-perftest_performance.log
root 18129 0.0 0.0 3912 708 pts/0 S+ 06:39 0:00 grep perftest
root 21861 0.0 1.7 196432 109112 ? Sl Oct28 0:10 /usr/sbin/qpidd --data-dir /mnt/tests/distribution/MRG/Messaging/qpid_test_qpidd-perftest_performance/rhts_qpidd/20091028_225501/b.1749 --port 42119 --auth no --log-enable info+ --num-jfiles 32 --jfile-size-pgs 64 --wcache-page-size 64 --tpl-num-jfiles 32 --tpl-jfile-size-pgs 64 --tpl-wcache-page-size 64 --mgmt-enable 0
root 22145 0.0 0.0 1652 336 ? S Oct28 0:00 /usr/bin/time -f %e -o time.transcript.log perftest --iterations 1 --summary --unique-data yes --log-enable info+ -p 42119 --mode shared --qt 1 --durable yes --npubs 2 --nsubs 1 --count 200000 --size 1024 --tx 0 --async-commit no
root 22146 0.0 0.1 94820 8364 ? Sl Oct28 0:05 perftest --iterations 1 --summary --unique-data yes --log-enable info+ -p 42119 --mode shared --qt 1 --durable yes --npubs 2 --nsubs 1 --count 200000 --size 1024 --tx 0 --async-commit no
[root@hp-dl2x170g6-02 MRG]# pstree -apA 22145
time,22145 -f %e -o time.transcript.log perftest --iterations 1 --summary --unique-data yes --log-enableinfo
`-perftest,22146 --iterations 1 --summary --unique-data yes --log-enable info+ -p 42119 --mode shared ...
|-{perftest},22194
|-{perftest},22195
`-{perftest},22196
[root@hp-dl2x170g6-02 MRG]# pstack 22145
#0 0x00f5b410 in __kernel_vsyscall ()
#1 0x00b89e6e in wait4 () from /lib/libc.so.6
#2 0x00b89e47 in wait3 () from /lib/libc.so.6
#3 0x08049766 in ?? ()
#4 0x08048c75 in ?? ()
#5 0x00b0ee8c in __libc_start_main () from /lib/libc.so.6
#6 0x080488d1 in ?? ()
[root@hp-dl2x170g6-02 MRG]# pstack 22146
Thread 4 (Thread 0xb55fcb90 (LWP 22194)):
#0 0x00d3b410 in __kernel_vsyscall ()
#1 0x00bcaaa6 in epoll_wait () from /lib/libc.so.6
#2 0x00585f79 in qpid::sys::Poller::wait () from /usr/lib/libqpidcommon.so.0
#3 0x00586fa6 in qpid::sys::Poller::run () from /usr/lib/libqpidcommon.so.0
#4 0x005d7084 in qpid::sys::Dispatcher::run ()
#5 0x00161505 in qpid::client::TCPConnector::run ()
#6 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#7 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#8 0x00bca42e in clone () from /lib/libc.so.6
Thread 3 (Thread 0xb4bfbb90 (LWP 22195)):
#0 0x00d3b410 in __kernel_vsyscall ()
#1 0x00c4e595 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2 0x00bd6b3d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3 0x001889d1 in qpid::sys::Waitable::wait () from /usr/lib/libqpidclient.so.0
#4 0x00187386 in qpid::client::LocalQueue::get ()
#5 0x00187dae in qpid::client::LocalQueue::get ()
#6 0x001880d7 in qpid::client::LocalQueue::pop ()
#7 0x08061863 in ?? ()
#8 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#9 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#10 0x00bca42e in clone () from /lib/libc.so.6
Thread 2 (Thread 0xb41fab90 (LWP 22196)):
#0 0x00d3b410 in __kernel_vsyscall ()
#1 0x00bcaaa6 in epoll_wait () from /lib/libc.so.6
#2 0x00585f79 in qpid::sys::Poller::wait () from /usr/lib/libqpidcommon.so.0
#3 0x00586fa6 in qpid::sys::Poller::run () from /usr/lib/libqpidcommon.so.0
#4 0x005d7084 in qpid::sys::Dispatcher::run ()
#5 0x00161505 in qpid::client::TCPConnector::run ()
#6 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#7 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#8 0x00bca42e in clone () from /lib/libc.so.6
Thread 1 (Thread 0xb7feb720 (LWP 22146)):
#0 0x00d3b410 in __kernel_vsyscall ()
#1 0x00c4e595 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2 0x00bd6b3d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3 0x001889d1 in qpid::sys::Waitable::wait () from /usr/lib/libqpidclient.so.0
#4 0x00187386 in qpid::client::LocalQueue::get ()
#5 0x00187dae in qpid::client::LocalQueue::get ()
#6 0x001880d7 in qpid::client::LocalQueue::pop ()
#7 0x0805bce5 in ?? ()
#8 0x0805f57e in ?? ()
#9 0x08052cbd in qpid::framing::TransferContent::getData ()
#10 0x00b0ee8c in __libc_start_main () from /lib/libc.so.6
#11 0x08051c01 in qpid::framing::TransferContent::getData ()
[root@hp-dl2x170g6-02 MRG]# gdb
GNU gdb Fedora (6.8-27.el5)
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu".
(gdb) attach 22146
Attaching to process 22146
Reading symbols from /usr/bin/perftest...Reading symbols from /usr/lib/debug/usr/bin/perftest.debug...done.
done.
Reading symbols from /usr/lib/libqpidclient.so.0...Reading symbols from /usr/lib/debug/usr/lib/libqpidclient.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib/libqpidclient.so.0
Reading symbols from /usr/lib/libqpidcommon.so.0...Reading symbols from /usr/lib/debug/usr/lib/libqpidcommon.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib/libqpidcommon.so.0
Reading symbols from /usr/lib/libboost_program_options.so.2...done.
Loaded symbols for /usr/lib/libboost_program_options.so.2
Reading symbols from /usr/lib/libboost_filesystem.so.2...done.
Loaded symbols for /usr/lib/libboost_filesystem.so.2
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/librt.so.1...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /usr/lib/libsasl2.so.2...done.
Loaded symbols for /usr/lib/libsasl2.so.2
Reading symbols from /lib/libuuid.so.1...done.
Loaded symbols for /lib/libuuid.so.1
Reading symbols from /usr/lib/libstdc++.so.6...done.
Loaded symbols for /usr/lib/libstdc++.so.6
Reading symbols from /lib/libm.so.6...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 0xb7feb720 (LWP 22146)]
[New Thread 0xb41fab90 (LWP 22196)]
[New Thread 0xb4bfbb90 (LWP 22195)]
[New Thread 0xb55fcb90 (LWP 22194)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /usr/lib/qpid/client/sslconnector.so...Reading symbols from /usr/lib/debug/usr/lib/qpid/client/sslconnector.so.debug...done.
done.
Loaded symbols for /usr/lib/qpid/client/sslconnector.so
Reading symbols from /usr/lib/libsslcommon.so.0...Reading symbols from /usr/lib/debug/usr/lib/libsslcommon.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib/libsslcommon.so.0
Reading symbols from /usr/lib/libnss3.so...done.
Loaded symbols for /usr/lib/libnss3.so
Reading symbols from /usr/lib/libssl3.so...done.
Loaded symbols for /usr/lib/libssl3.so
Reading symbols from /usr/lib/libnspr4.so...done.
Loaded symbols for /usr/lib/libnspr4.so
Reading symbols from /usr/lib/libnssutil3.so...done.
Loaded symbols for /usr/lib/libnssutil3.so
Reading symbols from /usr/lib/libplc4.so...done.
Loaded symbols for /usr/lib/libplc4.so
Reading symbols from /usr/lib/libplds4.so...done.
Loaded symbols for /usr/lib/libplds4.so
Reading symbols from /usr/lib/qpid/client/rdmaconnector.so...Reading symbols from /usr/lib/debug/usr/lib/qpid/client/rdmaconnector.so.debug...done.
done.
Loaded symbols for /usr/lib/qpid/client/rdmaconnector.so
Reading symbols from /usr/lib/librdmawrap.so.0...Reading symbols from /usr/lib/debug/usr/lib/librdmawrap.so.0.1.0.debug...done.
done.
Loaded symbols for /usr/lib/librdmawrap.so.0
Reading symbols from /usr/lib/librdmacm.so.1...done.
Loaded symbols for /usr/lib/librdmacm.so.1
Reading symbols from /usr/lib/libibverbs.so.1...done.
Loaded symbols for /usr/lib/libibverbs.so.1
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /usr/lib/sasl2/libsasldb.so.2...done.
Loaded symbols for /usr/lib/sasl2/libsasldb.so.2
Reading symbols from /usr/lib/sasl2/libplain.so.2...done.
Loaded symbols for /usr/lib/sasl2/libplain.so.2
Reading symbols from /usr/lib/sasl2/libgssapiv2.so.2...done.
Loaded symbols for /usr/lib/sasl2/libgssapiv2.so.2
Reading symbols from /usr/lib/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /usr/lib/libk5crypto.so.3...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/libcom_err.so.2...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libkrb5support.so.0...done.
Loaded symbols for /usr/lib/libkrb5support.so.0
Reading symbols from /lib/libkeyutils.so.1...done.
Loaded symbols for /lib/libkeyutils.so.1
Reading symbols from /lib/libselinux.so.1...done.
Loaded symbols for /lib/libselinux.so.1
Reading symbols from /lib/libsepol.so.1...done.
Loaded symbols for /lib/libsepol.so.1
Reading symbols from /usr/lib/sasl2/libanonymous.so.2...done.
Loaded symbols for /usr/lib/sasl2/libanonymous.so.2
Reading symbols from /usr/lib/sasl2/liblogin.so.2...done.
Loaded symbols for /usr/lib/sasl2/liblogin.so.2
0x00d3b410 in __kernel_vsyscall ()
(gdb) info threads
4 Thread 0xb55fcb90 (LWP 22194) 0x00d3b410 in __kernel_vsyscall ()
3 Thread 0xb4bfbb90 (LWP 22195) 0x00d3b410 in __kernel_vsyscall ()
2 Thread 0xb41fab90 (LWP 22196) 0x00d3b410 in __kernel_vsyscall ()
* 1 Thread 0xb7feb720 (LWP 22146) 0x00d3b410 in __kernel_vsyscall ()
(gdb) thread apply all bt
Thread 4 (Thread 0xb55fcb90 (LWP 22194)):
#0 0x00d3b410 in __kernel_vsyscall ()
#1 0x00bcaaa6 in epoll_wait () from /lib/libc.so.6
#2 0x00585f79 in qpid::sys::Poller::wait (this=<value optimized out>, timeout=Could not find the frame base for "qpid::sys::Poller::wait(qpid::sys::Duration)".
)
at qpid/sys/epoll/EpollPoller.cpp:439
#3 0x00586fa6 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405
#4 0x005d7084 in qpid::sys::Dispatcher::run (this=Could not find the frame base for "qpid::sys::Dispatcher::run()".
) at qpid/sys/Dispatcher.cpp:37
#5 0x00161505 in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:396
#6 0x0057bf01 in runRunnable (p=Could not find the frame base for "runRunnable".
) at qpid/sys/posix/Thread.cpp:35
#7 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#8 0x00bca42e in clone () from /lib/libc.so.6
Thread 3 (Thread 0xb4bfbb90 (LWP 22195)):
#0 0x00d3b410 in __kernel_vsyscall ()
#1 0x00c4e595 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2 0x00bd6b3d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3 0x001889d1 in qpid::sys::Waitable::wait (this=<value optimized out>) at qpid/sys/posix/Condition.h:63
#4 0x00187386 in qpid::client::LocalQueue::get (this=<value optimized out>, result=<value optimized out>,
timeout=<value optimized out>) at qpid/sys/BlockingQueue.h:55
#5 0x00187dae in qpid::client::LocalQueue::get (this=<value optimized out>, timeout=<value optimized out>)
at qpid/client/LocalQueue.cpp:40
#6 0x001880d7 in qpid::client::LocalQueue::pop (this=Could not find the frame base for "qpid::client::LocalQueue::pop(qpid::sys::Duration)".
) at qpid/client/LocalQueue.cpp:36
#7 0x08061863 in SubscribeThread::run (this=0xb74028d8) at perftest.cpp:611
#8 0x0057bf01 in runRunnable (p=Could not find the frame base for "runRunnable".
) at qpid/sys/posix/Thread.cpp:35
#9 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#10 0x00bca42e in clone () from /lib/libc.so.6
Thread 2 (Thread 0xb41fab90 (LWP 22196)):
#0 0x00d3b410 in __kernel_vsyscall ()
#1 0x00bcaaa6 in epoll_wait () from /lib/libc.so.6
#2 0x00585f79 in qpid::sys::Poller::wait (this=<value optimized out>, timeout=Could not find the frame base for "qpid::sys::Poller::wait(qpid::sys::Duration)".
)
at qpid/sys/epoll/EpollPoller.cpp:439
#3 0x00586fa6 in qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:405
#4 0x005d7084 in qpid::sys::Dispatcher::run (this=Could not find the frame base for "qpid::sys::Dispatcher::run()".
) at qpid/sys/Dispatcher.cpp:37
#5 0x00161505 in qpid::client::TCPConnector::run (this=<value optimized out>) at qpid/client/Connector.cpp:396
#6 0x0057bf01 in runRunnable (p=Could not find the frame base for "runRunnable".
) at qpid/sys/posix/Thread.cpp:35
#7 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#8 0x00bca42e in clone () from /lib/libc.so.6
Thread 1 (Thread 0xb7feb720 (LWP 22146)):
#0 0x00d3b410 in __kernel_vsyscall ()
#1 0x00c4e595 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2 0x00bd6b3d in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3 0x001889d1 in qpid::sys::Waitable::wait (this=<value optimized out>) at qpid/sys/posix/Condition.h:63
#4 0x00187386 in qpid::client::LocalQueue::get (this=<value optimized out>, result=<value optimized out>,
timeout=<value optimized out>) at qpid/sys/BlockingQueue.h:55
#5 0x00187dae in qpid::client::LocalQueue::get (this=<value optimized out>, timeout=<value optimized out>)
at qpid/client/LocalQueue.cpp:40
---Type <return> to continue, or q <return> to quit---
#6 0x001880d7 in qpid::client::LocalQueue::pop (this=Could not find the frame base for "qpid::client::LocalQueue::pop(qpid::sys::Duration)".
) at qpid/client/LocalQueue.cpp:36
#7 0x0805bce5 in Controller::process (this=0xbfd8b7b0, n=2, lq=
{queue = {px = 0xbfd8b57c, pn = {pi_ = 0xb7449448}}, subscription = {<qpid::client::Handle<qpid::client::SubscriptionImpl>> = {impl = 0x4}, <No data fields>}}, queue=
{static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xbfd8b5f0 "\\\224D·t¶Ø¿(¶Ø¿¸·Ø¿Ø·Ø¿Ð·Ø¿Ô·Ø¿dÔ\006\bh]C·dÔ\006\bû\225\032"}}, msgFn=
{<boost::function1<void,const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&,std::allocator<void> >> = {<boost::function_base> = {manager = 0xbfd8b570, functor = {obj_ptr = 0xbfd8b5f0, const_obj_ptr = 0xbfd8b5f0, func_ptr = 0xbfd8b5f0, data = "ð"}}, static args = <optimized out>, static arity = <optimized out>, invoker = 0xbfd8b57c}, <No data fields>}) at perftest.cpp:368
#8 0x0805f57e in Controller::run (this=0xbfd8b7b0) at perftest.cpp:408
#9 0x08052cbd in main (argc=Cannot access memory at address 0x80
) at perftest.cpp:699
(gdb)
(gdb) quit
The program is running. Quit anyway (and detach it)? (y or n) y
Detaching from program: /usr/bin/perftest, process 22146
[root@hp-dl2x170g6-02 MRG]# ps auxw | grep qpidd
root 10901 0.0 0.0 3708 508 ? S Oct28 0:00 tee /mnt/tests/distribution/MRG/Messaging/qpid_test_qpidd-perftest_performance/qpid_test_qpidd-perftest_performance.log
root 20025 0.0 0.0 3912 684 pts/0 S+ 06:43 0:00 grep qpidd
root 21861 0.0 1.7 196432 109112 ? Sl Oct28 0:10 /usr/sbin/qpidd --data-dir /mnt/tests/distribution/MRG/Messaging/qpid_test_qpidd-perftest_performance/rhts_qpidd/20091028_225501/b.1749 --port 42119 --auth no --log-enable info+ --num-jfiles 32 --jfile-size-pgs 64 --wcache-page-size 64 --tpl-num-jfiles 32 --tpl-jfile-size-pgs 64 --tpl-wcache-page-size 64 --mgmt-enable 0
[root@hp-dl2x170g6-02 MRG]# netstat -nlp | grep qpidd
tcp 0 0 0.0.0.0:42119 0.0.0.0:* LISTEN 21861/qpidd
[root@hp-dl2x170g6-02 MRG]# pstree -apA 21861
qpidd,21861 --data-dir/mnt/tests/distribution/MRG/Messaging/qpid_test_qpidd-perftest_performance/rhts_qpidd
|-{qpidd},21866
|-{qpidd},21867
|-{qpidd},21876
|-{qpidd},21877
|-{qpidd},21878
|-{qpidd},21879
`-{qpidd},21880
[root@hp-dl2x170g6-02 MRG]# pstree 21861
qpiddâââ7*[{qpidd}]
[root@hp-dl2x170g6-02 MRG]# pstack 21861
Thread 8 (Thread 0xb7f18b90 (LWP 21866)):
#0 0x007bf410 in __kernel_vsyscall ()
#1 0x00c4e8c2 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
#2 0x00bd6b84 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3 0x002a747e in qpid::broker::Timer::run () from /usr/lib/libqpidbroker.so.0
#4 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#5 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#6 0x00bca42e in clone () from /lib/libc.so.6
Thread 7 (Thread 0xb7517b90 (LWP 21867)):
#0 0x007bf410 in __kernel_vsyscall ()
#1 0x00c4e8c2 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
#2 0x00bd6b84 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3 0x002a747e in qpid::broker::Timer::run () from /usr/lib/libqpidbroker.so.0
#4 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#5 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#6 0x00bca42e in clone () from /lib/libc.so.6
Thread 6 (Thread 0xb6a18b90 (LWP 21876)):
#0 0x007bf410 in __kernel_vsyscall ()
#1 0x00c4e8c2 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
#2 0x00bd6b84 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libc.so.6
#3 0x002a747e in qpid::broker::Timer::run () from /usr/lib/libqpidbroker.so.0
#4 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#5 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#6 0x00bca42e in clone () from /lib/libc.so.6
Thread 5 (Thread 0xb6017b90 (LWP 21877)):
#0 0x007bf410 in __kernel_vsyscall ()
#1 0x00bcaaa6 in epoll_wait () from /lib/libc.so.6
#2 0x00585f79 in qpid::sys::Poller::wait () from /usr/lib/libqpidcommon.so.0
#3 0x00586fa6 in qpid::sys::Poller::run () from /usr/lib/libqpidcommon.so.0
#4 0x005d7084 in qpid::sys::Dispatcher::run ()
#5 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#6 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#7 0x00bca42e in clone () from /lib/libc.so.6
Thread 4 (Thread 0xb5616b90 (LWP 21878)):
#0 0x007bf410 in __kernel_vsyscall ()
#1 0x00bcaaa6 in epoll_wait () from /lib/libc.so.6
#2 0x00585f79 in qpid::sys::Poller::wait () from /usr/lib/libqpidcommon.so.0
#3 0x00586fa6 in qpid::sys::Poller::run () from /usr/lib/libqpidcommon.so.0
#4 0x005d7084 in qpid::sys::Dispatcher::run ()
#5 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#6 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#7 0x00bca42e in clone () from /lib/libc.so.6
Thread 3 (Thread 0xb4c15b90 (LWP 21879)):
#0 0x007bf410 in __kernel_vsyscall ()
#1 0x00bcaaa6 in epoll_wait () from /lib/libc.so.6
#2 0x00585f79 in qpid::sys::Poller::wait () from /usr/lib/libqpidcommon.so.0
#3 0x00586fa6 in qpid::sys::Poller::run () from /usr/lib/libqpidcommon.so.0
#4 0x005d7084 in qpid::sys::Dispatcher::run ()
#5 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#6 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#7 0x00bca42e in clone () from /lib/libc.so.6
Thread 2 (Thread 0xb4214b90 (LWP 21880)):
#0 0x007bf410 in __kernel_vsyscall ()
#1 0x00bcaaa6 in epoll_wait () from /lib/libc.so.6
#2 0x00585f79 in qpid::sys::Poller::wait () from /usr/lib/libqpidcommon.so.0
#3 0x00586fa6 in qpid::sys::Poller::run () from /usr/lib/libqpidcommon.so.0
#4 0x005d7084 in qpid::sys::Dispatcher::run ()
#5 0x0057bf01 in ?? () from /usr/lib/libqpidcommon.so.0
#6 0x00c4a49b in start_thread () from /lib/libpthread.so.0
#7 0x00bca42e in clone () from /lib/libc.so.6
Thread 1 (Thread 0xb7f19720 (LWP 21861)):
#0 0x007bf410 in __kernel_vsyscall ()
#1 0x00bcaaa6 in epoll_wait () from /lib/libc.so.6
#2 0x00585f79 in qpid::sys::Poller::wait () from /usr/lib/libqpidcommon.so.0
#3 0x00586fa6 in qpid::sys::Poller::run () from /usr/lib/libqpidcommon.so.0
#4 0x005d7084 in qpid::sys::Dispatcher::run ()
#5 0x001ce051 in qpid::broker::Broker::run () from /usr/lib/libqpidbroker.so.0
#6 0x0804dd44 in ?? ()
#7 0x0804c6c7 in __cxa_pure_virtual ()
#8 0x00b0ee8c in __libc_start_main () from /lib/libc.so.6
#9 0x0804c051 in __cxa_pure_virtual ()
I am trying (unsuccessfully thus far) to reproduce this problem. If I use the size and counts in the description I hit capacity exceptions on the journal even with the sizings described. If I lower the count a little for each size to stay in the limits then the test will happily loop for quite some time and I have not observed any hangs yet. Any clues? Is this still reproducible for you? The issue is still reproducible, but always shows in connection with - queue policy exception (resource-limit-exceeded: Policy exceeded on) or - Enqueue capacity threshold exceeded exception After discussion we concluded that in this particular case it is correct behavior, please? If so, feel free to move bug to CLOSE/NOTABUG, because if confirmed bug has no valid content then. That is correct; if the journal or policy limit is exceeded then the subscribers will not receive all expected messages and the test will not complete. |