Bug 586348 - qpid c++ client library crash in qpid::loadModuleDir() / qpid::loadModuleDir()->::isShlibName()
Summary: qpid c++ client library crash in qpid::loadModuleDir() / qpid::loadModuleDir(...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp
Version: Development
Hardware: All
OS: Linux
high
high
Target Milestone: 1.3
: ---
Assignee: Andrew Stitcher
QA Contact: Frantisek Reznicek
URL:
Whiteboard:
Depends On: 589675
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-04-27 11:31 UTC by Frantisek Reznicek
Modified: 2015-11-16 01:12 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-10-20 11:30:54 UTC


Attachments (Terms of Use)

Description Frantisek Reznicek 2010-04-27 11:31:56 UTC
Created attachment 409444 [details]
The testing client (g++ -Wall -g -I. -I/usr/include/qpid-boost bz452015.cpp -lpthread -lqpidclient  -o bz452015)

Description of problem:

There were observed multiple crashes:
a] directly in qpid::loadModuleDir() call (rhel5.5 case)

  Core was generated by `./bz452015'.
  Program terminated with signal 11, Segmentation fault.
  #0  qpid::loadModuleDir (dirname=..., isDefault=true) at /usr/include/c++/4.1.2/bits/basic_string.h:1563
  1563          { return this->find(__str.data(), __pos, __str.size()); }
  (gdb) info threads
  * 1 Thread 31557  qpid::loadModuleDir (dirname=..., isDefault=true) at /usr/include/c++/4.1.2/bits/basic_string.h:1563
  (gdb) bt
  #0  qpid::loadModuleDir (dirname=..., isDefault=true) at /usr/include/c++/4.1.2/bits/basic_string.h:1563
  #1  0x007e94f2 in LoadtimeInitialise (__initialize_p=<value optimized out>, __priority=<value optimized out>) at qpid/client/LoadPlugins.cpp:47
  #2  __static_initialization_and_destruction_0 (__initialize_p=<value optimized out>, __priority=<value optimized out>)
      at qpid/client/LoadPlugins.cpp:50
  #3  0x00872cb6 in __do_global_ctors_aux () from /usr/lib/libqpidclient.so.2
  #4  0x00796d95 in _init () from /usr/lib/libqpidclient.so.2
  #5  0x00217223 in call_init () from /lib/ld-linux.so.2
  #6  0x00217333 in _dl_init_internal () from /lib/ld-linux.so.2
  #7  0x0020984f in _dl_start_user () from /lib/ld-linux.so.2


b] or indirectly in ::isShlibName() called by qpid::loadModuleDir() (rhel4.8 case)

  Core was generated by `./bz452015'.
  Program terminated with signal 11, Segmentation fault.
  ...
  #0  (anonymous namespace)::isShlibName (name=@0x8247840)
      at /usr/lib/gcc/i386-redhat-linux/3.4.6/../../../../include/c++/3.4.6/bits/basic_string.h:278
  278           { return &((reinterpret_cast<_Rep*> (_M_data()))[-1]); }
  (gdb) info threads
  * 1 process 23742  (anonymous namespace)::isShlibName (name=@0x8247840)
      at /usr/lib/gcc/i386-redhat-linux/3.4.6/../../../../include/c++/3.4.6/bits/basic_string.h:278
  (gdb) bt
  #0  (anonymous namespace)::isShlibName (name=@0x8247840)
      at /usr/lib/gcc/i386-redhat-linux/3.4.6/../../../../include/c++/3.4.6/bits/basic_string.h:278
  #1  0x004ad7ef in qpid::loadModuleDir (dirname=
          {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x824823c "/usr/lib/qpid/client"}}, isDefault=true) at /usr/include/boost/filesystem/path.hpp:644
  #2  0x007ccfdb in global constructors keyed to _ZN56_GLOBAL__N_qpid_client_LoadPlugins.cpp_9D2164D3_2299BD904initE ()
      at qpid/client/LoadPlugins.cpp:47
  #3  0x0088bd95 in __do_global_ctors_aux () from /usr/lib/libqpidclient.so.2
  #4  0x0076fa01 in _init () from /usr/lib/libqpidclient.so.2
  #5  0x00ade8e8 in _dl_init_internal () from /lib/ld-linux.so.2
  #6  0x00ad27ff in _dl_start_user () from /lib/ld-linux.so.2



Version-Release number of selected component (if applicable):
qpid-java-common-0.7.934605-1.el5
qpid-cpp-client-ssl-0.7.935473-1.el5
qpid-cpp-server-store-0.7.935473-1.el5
qpid-cpp-client-devel-docs-0.7.935473-1.el5
qpid-tests-0.7.930108-1.el5
python-qpid-0.7.934605-1.el5
qpid-tools-0.7.934605-2.el5
qmf-devel-0.7.935473-1.el5
qpid-java-client-0.7.934605-1.el5
qpid-cpp-server-0.7.935473-1.el5
qpid-cpp-client-devel-0.7.935473-1.el5
qpid-cpp-server-devel-0.7.935473-1.el5
qpid-cpp-server-cluster-0.7.935473-1.el5
ruby-qpid-0.7.935473-1.el5
qpid-cpp-client-0.7.935473-1.el5
qpid-cpp-server-xml-0.7.935473-1.el5
qpid-cpp-mrg-debuginfo-0.7.935473-1.el5
python-qmf-0.7.934605-1.el5
qmf-0.7.935473-1.el5
qpid-cpp-server-ssl-0.7.935473-1.el5


How reproducible:
>90% (very rapidly with attached client)

Steps to Reproduce:
1. start qpidd broker
2. 'qpid-config add exchange direct bz452015'
3. './bz452015'
4. expect failure but not crash
5. 'qpid-config del exchange bz452015'
6. 'qpid-config --sequence add exchange direct bz452015'
7. './bz452015'
8. expect PASS


Actual results:
Attached client crashes at steps 3. and 7.

Expected results:
Attached client should not crash.

Additional info:

Comment 1 Gordon Sim 2010-04-27 12:13:01 UTC
The issue here comes from linking in pthreads before qpidclient. If you compile the client with: g++ -Wall -g -I. -I/usr/include/qpid-boost bz452015.cpp -lqpidclient  -o bz452015, then the above expectations are met. In fact even if the -lpthread is added in after the -lqpidclient then the issue is resolved.

A similar issue can be seen if you link in pthread before any qpid client based executable (e.g. direct exchange example).

Comment 2 Frantisek Reznicek 2010-04-27 14:38:49 UTC
Yes, indeed, the crash is caused by linking libpthread before libqpidclient.

Clients behave correctly now.

Comment 3 Andrew Stitcher 2010-04-27 16:45:45 UTC
The cause of this crash is undefined initialisation order:

LoadtimeInitialise() at qpid/client/LoadPlugins.cpp:47 which is part of a static global constructor and so is called before main(). It relies transitively on suffix in qpid/Modules.cpp:39 which is also a static global also constructed before main().

These things are in different files nad in fact in entirely different shared objects.

Since c++ does not define the order of construction of static globals (except in the same file) under some circumstances the construction order may work because suffix gets constructed before LoadtimeInitialise(), and in others it may not thus causing this crash.

Comment 4 Andrew Stitcher 2010-04-27 22:23:22 UTC
This should now be fixed upstream at r938701

I fixed it with the "belt and braces" approach of making LoadtimeInitialise no longer run as part of a static global constructor and also making suffix accessed using a "static singleton" pattern.

Comment 5 Frantisek Reznicek 2010-08-02 10:23:58 UTC
The issue has been fixed, tested on RHEL 4.8 / 5.5 i386 / x86_64 on packages:
python-qmf-0.7.946106-7.el5
python-qpid-0.7.946106-9.el5
qmf-0.7.946106-10.el5
qmf-devel-0.7.946106-10.el5
qpid-cpp-client*-0.7.946106-10.el5
qpid-cpp-mrg-debuginfo-0.7.946106-8.el5
qpid-cpp-server*-0.7.946106-10.el5
qpid-java-*-0.7.946106-7.el5
qpid-tools-0.7.946106-7.el5
ruby-qmf-0.7.946106-10.el5


-> VERIFIED


Note You need to log in before you can comment on or make changes to this bug.