Bug 1090810 - qpidd always crashes around qpid::framing::MessageProperties::print() while qpid_tests.broker_0_10.message.MessageTests.test_ttl unit test being executed
Summary: qpidd always crashes around qpid::framing::MessageProperties::print() while q...
Keywords:
Status: CLOSED DUPLICATE of bug 1030406
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp
Version: Development
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: ---
Assignee: messaging-bugs
QA Contact: MRG Quality Engineering
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-04-24 08:22 UTC by Frantisek Reznicek
Modified: 2015-11-16 01:16 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-06-15 12:49:41 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Frantisek Reznicek 2014-04-24 08:22:52 UTC
Description of problem:

qpidd sometimes crashes while qpid_tests.broker_0_10.message.MessageTests.test_ttl unit test being executed.

The automated test-suite uncovered speradic qpidd crash around 
qpid::framing::MessageProperties::print():

  #0  0x002a4a8c in qpid::framing::MessageProperties::print(std::basic_ostream<char, std::char_traits<char> >&) const () from /usr/lib/libqpidcommon.so.8
  #1  0x067fbc95 in qpid::broker::Message::printMessageProperties (
      this=0x9563258, out=...) at qpid/broker/Message.cpp:507
  #2  0x068147af in qpid::broker::Queue::isExpired (qm=...)
      at qpid/broker/Queue.cpp:614
  #3  0x0682838a in qpid::broker::Queue::consumeNextMessage (this=0x9699190, 
      m=..., c=...) at qpid/broker/Queue.cpp:412
  #4  0x068295c3 in qpid::broker::Queue::getNextMessage (this=0x9699190, m=..., 
      c=...) at qpid/broker/Queue.cpp:384
  #5  0x0682963b in qpid::broker::Queue::dispatch (this=0x9699190, c=...)
      at qpid/broker/Queue.cpp:506
  #6  0x06871661 in qpid::broker::SemanticState::ConsumerImpl::doDispatch (
      this=0x0) at qpid/broker/SemanticState.cpp:675
  ...


Seen on RHEL 5.10 Server i686 (kvm)

Version-Release number of selected component (if applicable):
qpid-cpp-*-0.18-20.el5_10

How reproducible:
very low

Steps to Reproduce:
1. execute upstream python qpid_tests.broker_0_10 unit tests utill it happens


Actual results:
  qpidd crash

Expected results:
  no qpidd crash


Additional info:
Test transcript:
[05:15:21] .Executing the python unit tests from qpid-tests (4800)
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_add_alternate_to_exchange ...................................... pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_del_alternate_to_exchange ...................................... pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_delete_while_used_by_exchange .................................. pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_delete_while_used_by_queue ..................................... pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_modify_existing_exchange_alternate ............................. pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_queue_autodelete ............................................... pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_queue_delete ................................................... pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_queue_delete_loop .............................................. pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_queue_delete_no_match .......................................... pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_reject_no_match ................................................ pass
qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_unroutable ..................................................... pass
qpid_tests.broker_0_10.stats.BrokerStatsTests.test_abandoned_alt ........ pass
qpid_tests.broker_0_10.stats.BrokerStatsTests. \
    test_discards_limit_overflow ........................................ pass
qpid_tests.broker_0_10.stats.BrokerStatsTests.test_discards_lvq_replace . pass
qpid_tests.broker_0_10.stats.BrokerStatsTests.test_discards_no_route .... pass
qpid_tests.broker_0_10.stats.BrokerStatsTests.test_discards_purge ....... pass
qpid_tests.broker_0_10.stats.BrokerStatsTests.test_discards_reject ...... pass
qpid_tests.broker_0_10.stats.BrokerStatsTests. \
    test_discards_ring_overflow ......................................... pass
qpid_tests.broker_0_10.stats.BrokerStatsTests.test_discards_ttl ......... pass
qpid_tests.broker_0_10.stats.BrokerStatsTests.test_enqueues_dequeues .... pass
qpid_tests.broker_0_10.stats.BrokerStatsTests.test_message_release ...... pass
qpid_tests.broker_0_10.stats.BrokerStatsTests.test_reroutes ............. pass
qpid_tests.broker_0_10.stats.BrokerStatsTests. \
    test_transactional_enqueues_dequeues ................................ pass
qpid_tests.broker_0_10.broker.BrokerTests.test_ack_and_no_ack ........... pass
qpid_tests.broker_0_10.broker.BrokerTests. \
    test_simple_delivery_immediate ...................................... pass
qpid_tests.broker_0_10.broker.BrokerTests.test_simple_delivery_queued ... pass
qpid_tests.broker_0_10.exchange. \
    DeclareMethodExchangeFieldReservedRuleTests.test_amq ................ pass
qpid_tests.broker_0_10.exchange. \
    DeclareMethodExchangeFieldReservedRuleTests.test_qpid ............... pass
qpid_tests.broker_0_10.exchange. \
    DeclareMethodPassiveFieldNotFoundRuleTests.test ..................... pass
qpid_tests.broker_0_10.exchange.DefaultExchangeRuleTests. \
    testDefaultExchange ................................................. pass
qpid_tests.broker_0_10.dtx.DtxTests.test_bad_resume ..................... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_commit_unknown ................. pass
qpid_tests.broker_0_10.dtx.DtxTests.test_end ............................ pass
qpid_tests.broker_0_10.dtx.DtxTests.test_end_suspend_and_fail ........... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_end_unknown_xid ................ pass
qpid_tests.broker_0_10.dtx.DtxTests.test_forget_xid_on_completion ....... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_get_timeout .................... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_get_timeout_unknown ............ pass
qpid_tests.broker_0_10.dtx.DtxTests.test_implicit_end ................... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_invalid_commit_not_ended ....... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_invalid_commit_one_phase_false . pass
qpid_tests.broker_0_10.dtx.DtxTests.test_invalid_commit_one_phase_true .. pass
qpid_tests.broker_0_10.dtx.DtxTests.test_invalid_prepare_not_ended ...... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_invalid_rollback_not_ended ..... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_prepare_unknown ................ pass
qpid_tests.broker_0_10.dtx.DtxTests.test_recover ........................ pass
qpid_tests.broker_0_10.dtx.DtxTests.test_rollback_unknown ............... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_select_required ................ pass
qpid_tests.broker_0_10.dtx.DtxTests.test_set_timeout .................... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_simple_commit .................. pass
qpid_tests.broker_0_10.dtx.DtxTests.test_simple_prepare_commit .......... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_simple_prepare_rollback ........ pass
qpid_tests.broker_0_10.dtx.DtxTests.test_simple_rollback ................ pass
qpid_tests.broker_0_10.dtx.DtxTests.test_start_already_known ............ pass
qpid_tests.broker_0_10.dtx.DtxTests.test_start_join ..................... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_start_join_and_resume .......... pass
qpid_tests.broker_0_10.dtx.DtxTests.test_suspend_resume ................. pass
qpid_tests.broker_0_10.dtx.DtxTests.test_suspend_start_end_resume ....... pass
qpid_tests.broker_0_10.qmf_events.EventTests. \
    test_queue_autodelete_exclusive ..................................... pass
qpid_tests.broker_0_10.qmf_events.EventTests. \
    test_queue_autodelete_shared ........................................ pass
qpid_tests.broker_0_10.qmf_events.EventTests.test_queue_declare ......... pass
qpid_tests.broker_0_10.qmf_events.EventTests.test_queue_delete .......... pass
qpid_tests.broker_0_10.example.ExampleTest.test_example ................. pass
qpid_tests.broker_0_10.exchange.ExchangeTests.testHeadersBindNoMatchArg . pass
qpid_tests.broker_0_10.extensions.ExtensionTests. \
    test_policy_count_as_float .......................................... pass
qpid_tests.broker_0_10.extensions.ExtensionTests. \
    test_policy_count_as_invalid_string ................................. pass
qpid_tests.broker_0_10.extensions.ExtensionTests. \
    test_policy_max_count_and_size_as_valid_strings ..................... pass
qpid_tests.broker_0_10.extensions.ExtensionTests. \
    test_policy_max_count_as_valid_string ............................... pass
qpid_tests.broker_0_10.extensions.ExtensionTests. \
    test_policy_max_size_as_valid_string ................................ pass
qpid_tests.broker_0_10.extensions.ExtensionTests. \
    test_policy_negative_count .......................................... pass
qpid_tests.broker_0_10.extensions.ExtensionTests. \
    test_policy_negative_size ........................................... pass
qpid_tests.broker_0_10.extensions.ExtensionTests. \
    test_policy_size_as_float ........................................... pass
qpid_tests.broker_0_10.extensions.ExtensionTests. \
    test_policy_size_as_invalid_string .................................. pass
qpid_tests.broker_0_10.extensions.ExtensionTests.test_timed_autodelete .. pass
qpid_tests.broker_0_10.new_api.GeneralTests. \
    test_qpid_3481_acquired_to_alt_exchange ............................. pass
qpid_tests.broker_0_10.new_api.GeneralTests. \
    test_qpid_3481_acquired_to_alt_exchange_2_consumers ................. pass
qpid_tests.broker_0_10.exchange.HeadersExchangeTests.testMatchAll ....... pass
qpid_tests.broker_0_10.exchange.HeadersExchangeTests.testMatchAny ....... pass
qpid_tests.broker_0_10.exchange.HeadersExchangeTests.testMatchVoidValue . pass
qpid_tests.broker_0_10.lvq.LVQTests.test_simple ......................... pass
qpid_tests.broker_0_10.management.ManagementTest. \
    test_binding_count_on_queue ......................................... pass
qpid_tests.broker_0_10.management.ManagementTest. \
    test_broker_connectivity_oldAPI ..................................... pass
qpid_tests.broker_0_10.management.ManagementTest.test_connection_close .. pass
qpid_tests.broker_0_10.management.ManagementTest.test_connection_stats .. pass
qpid_tests.broker_0_10.management.ManagementTest.test_get_objects ....... pass
qpid_tests.broker_0_10.management.ManagementTest.test_immediate_method .. pass
qpid_tests.broker_0_10.management.ManagementTest.test_methods_async ..... pass
qpid_tests.broker_0_10.management.ManagementTest.test_methods_sync ...... pass
qpid_tests.broker_0_10.management.ManagementTest. \
    test_move_queued_messages ........................................... pass
qpid_tests.broker_0_10.management.ManagementTest.test_purge_queue ....... pass
qpid_tests.broker_0_10.management.ManagementTest. \
    test_reroute_alternate_exchange ..................................... pass
qpid_tests.broker_0_10.management.ManagementTest. \
    test_reroute_invalid_alt_exchange ................................... pass
qpid_tests.broker_0_10.management.ManagementTest. \
    test_reroute_priority_queue ......................................... pass
qpid_tests.broker_0_10.management.ManagementTest.test_reroute_queue ..... pass
qpid_tests.broker_0_10.management.ManagementTest.test_self_session_id ... pass
qpid_tests.broker_0_10.management.ManagementTest. \
    test_standard_exchanges ............................................. pass
qpid_tests.broker_0_10.management.ManagementTest.test_timestamp_config .. pass
qpid_tests.broker_0_10.message.MessageTests.test_ack .................... pass
qpid_tests.broker_0_10.message.MessageTests.test_acquire ................ pass
qpid_tests.broker_0_10.message.MessageTests. \
    test_acquire_with_no_accept_and_credit_flow ......................... pass
qpid_tests.broker_0_10.message.MessageTests.test_cancel ................. pass
qpid_tests.broker_0_10.message.MessageTests.test_consume_exclusive ...... pass
qpid_tests.broker_0_10.message.MessageTests.test_consume_exclusive2 ..... pass
qpid_tests.broker_0_10.message.MessageTests. \
    test_consume_queue_not_found ........................................ pass
qpid_tests.broker_0_10.message.MessageTests. \
    test_consume_queue_not_specified .................................... pass
qpid_tests.broker_0_10.message.MessageTests. \
    test_consume_unique_consumers ....................................... pass
qpid_tests.broker_0_10.message.MessageTests.test_credit_flow_bytes ...... pass
qpid_tests.broker_0_10.message.MessageTests.test_credit_flow_messages ... pass
qpid_tests.broker_0_10.message.MessageTests. \
    test_credit_window_after_messagestop ................................ pass
qpid_tests.broker_0_10.message.MessageTests.test_empty_body ............. pass
qpid_tests.broker_0_10.message.MessageTests.test_incoming_start ......... pass
qpid_tests.broker_0_10.message.MessageTests.test_no_credit_wrap ......... pass
qpid_tests.broker_0_10.message.MessageTests.test_no_local ............... pass
qpid_tests.broker_0_10.message.MessageTests.test_no_local_awkward ....... pass
qpid_tests.broker_0_10.message.MessageTests. \
    test_no_local_exclusive_subscribe ................................... pass
qpid_tests.broker_0_10.message.MessageTests.test_ranged_ack ............. pass
qpid_tests.broker_0_10.message.MessageTests.test_reject ................. pass
qpid_tests.broker_0_10.message.MessageTests.test_release ................ pass
qpid_tests.broker_0_10.message.MessageTests.test_release_order .......... pass
qpid_tests.broker_0_10.message.MessageTests.test_release_ordering ....... pass
qpid_tests.broker_0_10.message.MessageTests.test_release_unacquired ..... pass
qpid_tests.broker_0_10.message.MessageTests.test_subscribe_not_acquired . pass
qpid_tests.broker_0_10.message.MessageTests. \
    test_subscribe_not_acquired_2 ....................................... pass
qpid_tests.broker_0_10.message.MessageTests. \
    test_subscribe_not_acquired_3 ....................................... pass
qpid_tests.broker_0_10.message.MessageTests.test_ttl ..................../mnt/tests/distribution/MRG/Messaging/qpid_common/qpid_test_functions_broker.sh: line 53: 25910 Segmentation fault      (core dumped) ${qpidd_wrapper} ${QPIDD_DIR}/${MRG_CPP_BROKER} ${datadir_sw} --port ${port} ${test_params} >&${transcript}
 fail
Error during test:  Traceback (most recent call last):
    File "/usr/bin/qpid-python-test", line 340, in run
      phase()
    File "/usr/lib/python2.4/site-packages/qpid_tests/broker_0_10/message.py", line 1092, in test_ttl
      session.message_flow(unit = session.credit_unit.byte, value=0xFFFFFFFFL, destination=d)
    File "/usr/lib/python2.4/site-packages/qpid/generator.py", line 25, in <lambda>
      method = lambda self, *args, **kwargs: self.invoke(op, args, kwargs)
    File "/usr/lib/python2.4/site-packages/qpid/session.py", line 138, in invoke
      return self.do_invoke(op, args, kwargs)
    File "/usr/lib/python2.4/site-packages/qpid/session.py", line 183, in do_invoke
      self.sync(self.timeout)
    File "/usr/lib/python2.4/site-packages/qpid/session.py", line 103, in sync
      raise SessionException(self.error())
  SessionException: (None, 'connection aborted')
qpid_tests.broker_0_10.message.MessageTests.test_window_flow_bytes ...... skip



Full segmentation failt backtrace:
./core.25910: ELF 32-bit LSB core file Intel 80386, version 1 (SYSV), SVR4-style, from 'qpidd'
  GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-45.el5)
  [New Thread 25922]
  [New Thread 25923]
  [New Thread 25916]
  [New Thread 25910]
  warning: .dynamic section for "/usr/lib/libqpidcommon.so.8" is not at the expected address
  warning: difference appears to be caused by prelink, adjusting expectations
  warning: .dynamic section for "/usr/lib/libxerces-c.so.28" is not at the expected address
  warning: difference appears to be caused by prelink, adjusting expectations
  warning: .dynamic section for "/usr/lib/openais/libcpg.so.2" is not at the expected address
  warning: difference appears to be caused by prelink, adjusting expectations
  warning: .dynamic section for "/usr/lib/libssl3.so" is not at the expected address
  warning: difference appears to be caused by prelink, adjusting expectations
  warning: .dynamic section for "/lib/libz.so.1" is not at the expected address
  warning: difference appears to be caused by prelink, adjusting expectations
  [Thread debugging using libthread_db enabled]
  Core was generated by `/usr/sbin/qpidd --data-dir /mnt/tests/distribution/MRG/Messaging/qpid_ptest_uni'.
  Program terminated with signal 11, Segmentation fault.

  #0  0x002a4a8c in qpid::framing::MessageProperties::print(std::basic_ostream<char, std::char_traits<char> >&) const (
  warning: (Internal error: pc 0x2a4a8c in read in psymtab, but not in symtab.)
  ) from /usr/lib/libqpidcommon.so.8
  warning: (Internal error: pc 0x2a4a8c in read in psymtab, but not in symtab.)
  (gdb) From        To          Syms Read   Shared Object Library
  0x06691d80  0x069125b4  Yes (*)     /usr/lib/libqpidbroker.so.8
  0x001ef780  0x00335f04  Yes (*)     /usr/lib/libqpidcommon.so.8
  0x00cdbc90  0x00ceac24  Yes (*)     /usr/lib/libqpidtypes.so.1
  0x00d00940  0x00d20044  Yes (*)     /usr/lib/libboost_program_options.so.2
  0x00cc7900  0x00cd00a4  Yes (*)     /usr/lib/libboost_filesystem.so.2
  0x0087cf90  0x0087ebb4  Yes (*)     /lib/libuuid.so.1
  0x00c76a70  0x00c77aa4  Yes (*)     /lib/libdl.so.2
  0x00d3c880  0x00d40c44  Yes (*)     /lib/librt.so.1
  0x00ddc190  0x00dec774  Yes (*)     /usr/lib/libsasl2.so.2
  0x00400c50  0x0047c204  Yes (*)     /usr/lib/libstdc++.so.6
  0x00c9c410  0x00cb75a4  Yes (*)     /lib/libm.so.6
  0x00d47660  0x00d4ef34  Yes (*)     /lib/libgcc_s.so.1
  0x00b2cca0  0x00c2be30  Yes (*)     /lib/libc.so.6
  0x00af87f0  0x00b0e12f  Yes (*)     /lib/ld-linux.so.2
  0x00c81210  0x00c8cbc4  Yes (*)     /lib/libpthread.so.0
  0x00d56130  0x00d61ae4  Yes (*)     /lib/libresolv.so.2
  0x004f7700  0x004fe334  Yes (*)     /lib/libcrypt.so.1
  0x004b3610  0x004bc704  Yes (*)     /usr/lib/qpid/daemon/rdma.so
  0x004c9430  0x004dc0e4  Yes (*)     /usr/lib/librdmawrap.so.8
  0x00e5e5b0  0x00e65974  Yes (*)     /usr/lib/libibverbs.so.1
  0x00e8a590  0x00e8efb4  Yes (*)     /usr/lib/librdmacm.so.1
  0x00682760  0x0068c2d4  Yes (*)     /usr/lib/qpid/daemon/replicating_listener.so
  0x00985980  0x00994a64  Yes (*)     /usr/lib/qpid/daemon/xml.so
  0x00fbc0a0  0x01179534  Yes (*)     /usr/lib/libxerces-c.so.28
  0x016acc00  0x018621d4  Yes (*)     /usr/lib/libxqilla.so.3
  0x00656040  0x0065b1c4  Yes (*)     /usr/lib/qpid/daemon/replication_exchange.so
  0x005523c0  0x00604f54  Yes (*)     /usr/lib/qpid/daemon/msgstore.so
  0x00711240  0x007d9764  Yes (*)     /usr/lib/libdb_cxx-4.3.so
  0x004e2390  0x004e26d0  Yes (*)     /usr/lib/libaio.so.1
  0x006a0ed0  0x006da134  Yes (*)     /usr/lib/qpid/daemon/acl.so
  0x008bce20  0x00947aa4  Yes (*)     /usr/lib/qpid/daemon/cluster.so
  0x004e4ed0  0x004e7224  Yes (*)     /usr/lib/openais/libcpg.so.2
  0x004e9d60  0x004ec944  Yes (*)     /usr/lib/libcman.so.2
  0x009cdd50  0x00a538d4  Yes (*)     /usr/lib/libqpidclient.so.8
  0x00635dd0  0x00641ca4  Yes (*)     /usr/lib/qpid/daemon/ssl.so
  0x0080ce60  0x00827ea4  Yes (*)     /usr/lib/libsslcommon.so.8
  0x061eeef0  0x062d5574  Yes (*)     /usr/lib/libnss3.so
  0x00835630  0x0085bbd4  Yes (*)     /usr/lib/libssl3.so
  0x00a847f0  0x00aa76d4  Yes (*)     /usr/lib/libnspr4.so
  0x00abc6c0  0x00acb454  Yes (*)     /usr/lib/libnssutil3.so
  0x00d76df0  0x00d786f4  Yes (*)     /usr/lib/libplc4.so
  0x00d70a90  0x00d71ab4  Yes (*)     /usr/lib/libplds4.so
  0x0065f5c0  0x0066a814  Yes (*)     /lib/libz.so.1
  0x07e02dd0  0x07e623e4  Yes (*)     /usr/lib/qpid/daemon/ha.so
  0x075d81e0  0x07638954  Yes (*)     /usr/lib/libqpidmessaging.so.3
  0x0086d6b0  0x00871d34  Yes (*)     /usr/lib/qpid/daemon/watchdog.so
  (*): Shared library is missing debugging information.
  warning: (Internal error: pc 0x2a4a8c in read in psymtab, but not in symtab.)
  (gdb)   4 Thread 0xb7f65950 (LWP 25910)  0x006f2402 in __kernel_vsyscall ()
    3 Thread 25916  0x006f2402 in __kernel_vsyscall ()
    2 Thread 25923  0x006f2402 in __kernel_vsyscall ()
  * 1 Thread 0xb7469b90 (LWP 25922)  warning: (Internal error: pc 0x2a4a8c in read in psymtab, but not in symtab.)
  0x002a4a8c in qpid::framing::MessageProperties::print(std::basic_ostream<char, std::char_traits<char> >&) const (warning: (Internal error: pc 0x2a4a8c in read in psymtab, but not in symtab.)
  ) from /usr/lib/libqpidcommon.so.8
  Thread 4 (Thread 0xb7f65950 (LWP 25910)):
  #0  0x006f2402 in __kernel_vsyscall ()
  #1  0x00bece46 in epoll_wait () from /lib/libc.so.6
  #2  0x00211a7a in qpid::sys::Poller::wait(qpid::sys::Duration) ()
     from /usr/lib/libqpidcommon.so.8
  #3  0x002126a3 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
  #4  0x00325e54 in qpid::sys::Dispatcher::run (this=0xbfa2e1d0)
      at qpid/sys/Dispatcher.cpp:37
  #5  0x0675946d in qpid::broker::Broker::run (this=0x94ca838)
      at qpid/broker/Broker.cpp:432
  #6  0x0804e7dc in qpid::broker::QpiddBroker::execute (this=0xbfa2e637, 
      options=0x94c4bd8) at posix/QpiddBroker.cpp:206
  #7  0x0804cc18 in qpid::broker::run_broker (argc=11, argv=0xbfa2e704, 
      hidden=false) at qpidd.cpp:106
  #8  0x0804e33a in main (argc=156837316, argv=0x959c628)
      at posix/QpiddBroker.cpp:215
  Thread 3 (Thread 25916):
  #0  0x006f2402 in __kernel_vsyscall ()
  #1  0x00c86ff2 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
     from /lib/libpthread.so.0
  #2  0x00bf8ed4 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libc.so.6
  #3  0x00328b33 in qpid::sys::Timer::run (this=0x94ca860)
      at ../include/qpid/sys/posix/Condition.h:69
  #4  0x002089a1 in qpid::sys::(anonymous namespace)::runRunnable(void*) ()
     from /usr/lib/libqpidcommon.so.8
  #5  0x00c82912 in start_thread () from /lib/libpthread.so.0
  #6  0x00bec7ce in clone () from /lib/libc.so.6
  Thread 2 (Thread 25923):
  #0  0x006f2402 in __kernel_vsyscall ()
  #1  0x00bece46 in epoll_wait () from /lib/libc.so.6
  #2  0x00211a7a in qpid::sys::Poller::wait(qpid::sys::Duration) ()
     from /usr/lib/libqpidcommon.so.8
  #3  0x002126a3 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
  #4  0x00325e54 in qpid::sys::Dispatcher::run (this=0xbfa2e1d0)
      at qpid/sys/Dispatcher.cpp:37
  #5  0x002089a1 in qpid::sys::(anonymous namespace)::runRunnable(void*) ()
     from /usr/lib/libqpidcommon.so.8
  #6  0x00c82912 in start_thread () from /lib/libpthread.so.0
  #7  0x00bec7ce in clone () from /lib/libc.so.6
  Thread 1 (Thread 0xb7469b90 (LWP 25922)):
  #0  0x002a4a8c in qpid::framing::MessageProperties::print(std::basic_ostream<char, std::char_traits<char> >&) const () from /usr/lib/libqpidcommon.so.8
  #1  0x067fbc95 in qpid::broker::Message::printMessageProperties (
      this=0x9563258, out=...) at qpid/broker/Message.cpp:507
  #2  0x068147af in qpid::broker::Queue::isExpired (qm=...)
      at qpid/broker/Queue.cpp:614
  #3  0x0682838a in qpid::broker::Queue::consumeNextMessage (this=0x9699190, 
      m=..., c=...) at qpid/broker/Queue.cpp:412
  #4  0x068295c3 in qpid::broker::Queue::getNextMessage (this=0x9699190, m=..., 
      c=...) at qpid/broker/Queue.cpp:384
  #5  0x0682963b in qpid::broker::Queue::dispatch (this=0x9699190, c=...)
      at qpid/broker/Queue.cpp:506
  #6  0x06871661 in qpid::broker::SemanticState::ConsumerImpl::doDispatch (
      this=0x0) at qpid/broker/SemanticState.cpp:675
  #7  0x0686fe58 in qpid::broker::SemanticState::ConsumerImpl::doOutput (
      this=0x958a7a0) at qpid/broker/SemanticState.cpp:744
  #8  0x00318cc0 in qpid::sys::AggregateOutput::doOutput (this=0x9552e8c)
      at qpid/sys/AggregateOutput.cpp:60
  #9  0x0678cfcb in qpid::broker::Connection::doOutput (this=0x9552e60)
      at qpid/broker/Connection.cpp:387
  #10 0x06743c0e in qpid::amqp_0_10::Connection::canEncode (this=0x95a5cf8)
      at qpid/amqp_0_10/Connection.cpp:67
  #11 0x0686dbdb in qpid::broker::SecureConnection::canEncode (this=0x9563e78)
      at qpid/broker/SecureConnection.cpp:58
  #12 0x0031ced9 in qpid::sys::AsynchIOHandler::idle (this=0x9696188)
      at qpid/sys/AsynchIOHandler.cpp:251
  #13 0x06910cad in operator() (function_obj_ptr=..., a0=...)
      at /usr/include/boost/bind/mem_fn_template.hpp:149
  #14 operator()<boost::_mfi::mf1<void, qpid::sys::AsynchIOHandler, qpid::sys::AsynchIO&>, boost::_bi::list1<qpid::sys::AsynchIO&> > (function_obj_ptr=..., 
      a0=...) at /usr/include/boost/bind.hpp:287
  #15 operator()<qpid::sys::AsynchIO> (function_obj_ptr=..., a0=...)
      at /usr/include/boost/bind/bind_template.hpp:32
  #16 boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::AsynchIOHandler, qpid::sys::AsynchIO&>, boost::_bi::list2<boost::_bi::value<qpid::sys::AsynchIOHandler*>, boost::arg<1> > >, void, qpid::sys::AsynchIO&>::invoke (function_obj_ptr=..., a0=...)
      at /usr/include/boost/function/function_template.hpp:136
  #17 0x00205df4 in boost::function1<void, qpid::sys::AsynchIO&, std::allocator<boost::function_base> >::operator()(qpid::sys::AsynchIO&) const ()
     from /usr/lib/libqpidcommon.so.8
  #18 0x0020364f in qpid::sys::posix::AsynchIO::writeable(qpid::sys::DispatchHandle&) () from /usr/lib/libqpidcommon.so.8
  #19 0x0020464d in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, qpid::sys::posix::AsynchIO, qpid::sys::DispatchHandle&>, boost::_bi::list2<boost::_bi::value<qpid::sys::posix::AsynchIO*>, boost::arg<1> > >, void, qpid::sys::DispatchHandle&>::invoke(boost::detail::function::any_pointer, qpid::sys::DispatchHandle&) ()
     from /usr/lib/libqpidcommon.so.8
  #20 0x00322c14 in boost::function1<void, qpid::sys::DispatchHandle&, std::allocator<boost::function_base> >::operator() (this=0x957edf8, a0=...)
      at /usr/include/boost/function/function_template.hpp:576
  #21 0x00322205 in qpid::sys::DispatchHandle::processEvent (this=0x957ede4, 
      type=WRITABLE) at qpid/sys/DispatchHandle.cpp:287
  #22 0x002126c1 in qpid::sys::Poller::run() () from /usr/lib/libqpidcommon.so.8
  #23 0x00325e54 in qpid::sys::Dispatcher::run (this=0xbfa2e1d0)
      at qpid/sys/Dispatcher.cpp:37
  #24 0x002089a1 in qpid::sys::(anonymous namespace)::runRunnable(void*) ()
     from /usr/lib/libqpidcommon.so.8
  #25 0x00c82912 in start_thread () from /lib/libpthread.so.0
  #26 0x00bec7ce in clone () from /lib/libc.so.6
  (gdb) quit

Comment 1 Frantisek Reznicek 2014-04-24 08:25:14 UTC
Full logs including machine specs:
https://beaker.engineering.redhat.com/recipes/1327832#task20787943

Comment 2 Stanislav Graf 2014-04-24 08:46:00 UTC
(In reply to Frantisek Reznicek from comment #0)
> qpidd sometimes crashes while
> qpid_tests.broker_0_10.message.MessageTests.test_ttl unit test being
> executed.

I've checked all our runs and it crashes always and on all platforms RHEL 5/6, i386/x86_64

Comment 3 Gordon Sim 2014-05-16 09:39:50 UTC
Fyi: I believe this is fixed by http://git.app.eng.bos.redhat.com/git/rh-qpid.git/commit/?h=0.18-mrg-kgiusti-bz1093738&id=7caf6fec80d98a4e4da504a662b6886766c3e56c

I believe its a result of having debug logging enabled when receiving messages with ttls (in DeliveryProperties) but no MessageProperties.

Comment 4 Pavel Moravec 2014-06-15 12:49:41 UTC

*** This bug has been marked as a duplicate of bug 1030406 ***


Note You need to log in before you can comment on or make changes to this bug.