Bug 441426 - Malloc calls from dlls that are opened with RTLD_DEEPBIND can not be interposed.
Malloc calls from dlls that are opened with RTLD_DEEPBIND can not be interposed.
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: glibc (Show other bugs)
All Linux
low Severity high
: rc
: ---
Assigned To: Jakub Jelinek
Brian Brock
: Reopened
Depends On:
  Show dependency treegraph
Reported: 2008-04-07 20:54 EDT by Seppo Sahrakorpi
Modified: 2008-06-09 13:52 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-06-09 13:52:32 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Reproducer package (862 bytes, application/x-gzip)
2008-04-07 20:54 EDT, Seppo Sahrakorpi
no flags Details
Non-malloc reproducer (1.38 KB, application/x-compressed-tar)
2008-06-09 12:45 EDT, Seppo Sahrakorpi
no flags Details

  None (edit)
Description Seppo Sahrakorpi 2008-04-07 20:54:13 EDT
Description of problem:

Malloc calls from dlls that are opened with RTLD_DEEPBIND can
not be interposed. As a result, e.g. TotalView Debugger's Heap
Interposition Agent (HIA) is not able keep track of these allocations.

Observations / Comments::

There is a new option to dlopen() called RTLD_DEEPBIND. Here is what the man
page says about it:

RTLD_DEEPBIND (since glibc 2.3.4)
    Place the lookup scope of the symbols in this library  ahead  of
    the global scope.  This means that a self-contained library will
    use its own symbols in preference to  global  symbols  with  the
    same  name contained in libraries that have already been loaded.
    This flag is not specified in POSIX.1-2001.

Also see: http://sourceware.org/ml/libc-hacker/2004-09/msg00083.html

We assume that the current situation might be as follows:

          This is 
		          DEEPBIND malloc 
                          call is here

EXE    ->    HIA    ->    USR PROVIDED   ->    LIBC

           defined here
			  no malloc(){} here   
                                              defined here

If there is no explicit malloc definition in USR PROVIDED, and the USR
PROVIDED library (with the malloc call) is called with RTLD_DEEPBIND,
the malloc in libc is used, not the malloc defined in Heap
Interposition Agent (HIA). We speculate that the failure could be due to that,
in this scenario, the search path (for malloc that was not defined in USR
PROVIDED under the RTLD_DEEPBIND mechanism) should have started from EXE
downstream, and not from USR PROVIDED downstream.

Version-Release number of selected component (if applicable):


pillbox:/home/seppo > cat /etc/*-rel*
cat: /etc/lsb-release.d: Is a directory
Red Hat Enterprise Linux Server release 5.1 (Tikanga)
pillbox:/home/seppo > 
pillbox:/home/seppo/Bugs/Bug_10786 > rpm -q glibc
pillbox:/home/seppo/Bugs/Bug_10786 > pillbox:/home/seppo/Bugs/Bug_10786 > uname -a
Linux pillbox.totalviewtech.com 2.6.18-53.el5PAE #1 SMP Wed Oct 10 16:48:18 EDT
2007 i686 i686 i386 GNU/Linux
pillbox:/home/seppo/Bugs/Bug_10786 > 

How reproducible:

Reproducer attached.

Steps to Reproduce:

> make clean 

> make all
gcc -g ldtest.cpp -Wl,-rpath=`pwd` -lstdc++ -ldl -o ldtest
gcc -g -shared myfunc.cpp -Wl,-rpath=`pwd` -lstdc++ -o libmyfunc.so

Use e.g. TotalView 8.4:

> totalview ./ldtest -a 1

In Process Window: Process > Enable Memory Debugging

Create a pending breakpoint on the function myfun by:

  In Process Window: Action Point > At Location

  In At Location dialog: Type myfun and hit OK

  In Ambig function dialog: select "Make "myfun" a pending breakpoint and hit

In Process Window: Do a Go

In Process Window: Do a Next to step over the call to malloc:

   void* leak_malloc_buf = malloc(90);

You reproduced the problem if you see that the variable leak_malloc_buf is
not annotated with 'allocated'

i.e. on the TotalView Command Line Interface (CLI):

d1.<> p leak_malloc_buf
 leak_malloc_buf = 0x0804a398 -> 0x00000000 (0)

d1.<> dheap -info 0x0804a398
process     1   (13213):


Actual results:

d1.<> p leak_malloc_buf
leak_malloc_buf = 0x0804a398 -> 0x00000000 (0)

Expected results:

Run the above reproducer, but without any arguments to ldtest:
> totalview ./ldtest 

d1.<> p leak_malloc_buf
 leak_malloc_buf = 0x08684398 (Allocated) -> 0x00000000 (0)

In this case myfun is called w/o RTLD_DEEPBIND and interposition has
been succesful.

Additional info:

This is present on several platforms with glibc 2.3.4 or higher.


Reported by

Seppo Sahrakorpi
Partner Technologies Engineer
TotalView Technologies, 24 Prime Parkway, Natick, MA 01760
seppo@totalviewtech.com -- 508-652-7781 

Comment 1 Seppo Sahrakorpi 2008-04-07 20:54:13 EDT
Created attachment 301583 [details]
Reproducer package
Comment 2 Ulrich Drepper 2008-06-06 16:36:04 EDT
And your point is?  You cannot have the cake and eat it.  It's the problem of
your proprietary technology and not that of glibc.
Comment 3 Seppo Sahrakorpi 2008-06-09 12:43:31 EDT
This is not limited to our malloc call. Attached please find a general
non-malloc reproducer, with the following logic:

  Write a small ia which interposes, e.g., strdup.
    The interposer should announc when it's called.
  Write a small main program which call strdup.
  Write a small dll which calls strdup from an exported fucntion.
  call strdup from main.
  Open the dll from main with RTLD_DEEPBIND.
  Call the exported function to demonstrate that
    the call from the dll is not interposed.

For details, see the 00readme file in bld-linux.
Comment 4 Seppo Sahrakorpi 2008-06-09 12:45:12 EDT
Created attachment 308722 [details]
Non-malloc reproducer
Comment 5 Ulrich Drepper 2008-06-09 13:52:32 EDT
The code works as it is supposed to.  There is nothing wrong except your
expectation.  And I am not going to invent more hooks for proprietary software
to work.

Note You need to log in before you can comment on or make changes to this bug.