Bug 441426 - Malloc calls from dlls that are opened with RTLD_DEEPBIND can not be interposed.
Summary: Malloc calls from dlls that are opened with RTLD_DEEPBIND can not be interposed.
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: glibc
Version: 5.1
Hardware: All
OS: Linux
low
high
Target Milestone: rc
: ---
Assignee: Jakub Jelinek
QA Contact: Brian Brock
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-04-08 00:54 UTC by Seppo Sahrakorpi
Modified: 2008-06-09 17:52 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2008-06-09 17:52:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Reproducer package (862 bytes, application/x-gzip)
2008-04-08 00:54 UTC, Seppo Sahrakorpi
no flags Details
Non-malloc reproducer (1.38 KB, application/x-compressed-tar)
2008-06-09 16:45 UTC, Seppo Sahrakorpi
no flags Details

Description Seppo Sahrakorpi 2008-04-08 00:54:13 UTC
Description of problem:

Symptom::
Malloc calls from dlls that are opened with RTLD_DEEPBIND can
not be interposed. As a result, e.g. TotalView Debugger's Heap
Interposition Agent (HIA) is not able keep track of these allocations.


Observations / Comments::

There is a new option to dlopen() called RTLD_DEEPBIND. Here is what the man
page says about it:

------------------------------------------------------------------------
RTLD_DEEPBIND (since glibc 2.3.4)
    Place the lookup scope of the symbols in this library  ahead  of
    the global scope.  This means that a self-contained library will
    use its own symbols in preference to  global  symbols  with  the
    same  name contained in libraries that have already been loaded.
    This flag is not specified in POSIX.1-2001.
------------------------------------------------------------------------

Also see: http://sourceware.org/ml/libc-hacker/2004-09/msg00083.html


We assume that the current situation might be as follows:

          This is 
          preloaded
		          DEEPBIND malloc 
                          call is here

EXE    ->    HIA    ->    USR PROVIDED   ->    LIBC

           malloc(){} 
           defined here
			  no malloc(){} here   
                                              malloc(){} 
                                              defined here


If there is no explicit malloc definition in USR PROVIDED, and the USR
PROVIDED library (with the malloc call) is called with RTLD_DEEPBIND,
the malloc in libc is used, not the malloc defined in Heap
Interposition Agent (HIA). We speculate that the failure could be due to that,
in this scenario, the search path (for malloc that was not defined in USR
PROVIDED under the RTLD_DEEPBIND mechanism) should have started from EXE
downstream, and not from USR PROVIDED downstream.


XXXXXXXXXXXXXXXXXXX
Version-Release number of selected component (if applicable):

glibc-2.5-18

pillbox:/home/seppo > cat /etc/*-rel*
cat: /etc/lsb-release.d: Is a directory
Red Hat Enterprise Linux Server release 5.1 (Tikanga)
pillbox:/home/seppo > 
pillbox:/home/seppo/Bugs/Bug_10786 > rpm -q glibc
glibc-2.5-18
pillbox:/home/seppo/Bugs/Bug_10786 > pillbox:/home/seppo/Bugs/Bug_10786 > uname -a
Linux pillbox.totalviewtech.com 2.6.18-53.el5PAE #1 SMP Wed Oct 10 16:48:18 EDT
2007 i686 i686 i386 GNU/Linux
pillbox:/home/seppo/Bugs/Bug_10786 > 


XXXXXXXXXXXXXXXXXXXXXXXXX
How reproducible:

Reproducer attached.


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Steps to Reproduce:

> make clean 

> make all
gcc -g ldtest.cpp -Wl,-rpath=`pwd` -lstdc++ -ldl -o ldtest
gcc -g -shared myfunc.cpp -Wl,-rpath=`pwd` -lstdc++ -o libmyfunc.so


Use e.g. TotalView 8.4:

> totalview ./ldtest -a 1

In Process Window: Process > Enable Memory Debugging

Create a pending breakpoint on the function myfun by:

  In Process Window: Action Point > At Location

  In At Location dialog: Type myfun and hit OK

  In Ambig function dialog: select "Make "myfun" a pending breakpoint and hit
ok

In Process Window: Do a Go

In Process Window: Do a Next to step over the call to malloc:

   void* leak_malloc_buf = malloc(90);

You reproduced the problem if you see that the variable leak_malloc_buf is
not annotated with 'allocated'

i.e. on the TotalView Command Line Interface (CLI):

d1.<> p leak_malloc_buf
 leak_malloc_buf = 0x0804a398 -> 0x00000000 (0)

d1.<> dheap -info 0x0804a398
process     1   (13213):

d1.<>


XXXXXXXXXXXXXXXXXXXXXXXXXX
Actual results:

d1.<> p leak_malloc_buf
leak_malloc_buf = 0x0804a398 -> 0x00000000 (0)
d1.<> 


XXXXXXXXXXXXXXXXXXXXXXXXXX
Expected results:

Run the above reproducer, but without any arguments to ldtest:
> totalview ./ldtest 

d1.<> p leak_malloc_buf
 leak_malloc_buf = 0x08684398 (Allocated) -> 0x00000000 (0)
d1.<> 

In this case myfun is called w/o RTLD_DEEPBIND and interposition has
been succesful.


XXXXXXXXXXXXXXXXXXXX
Additional info:

This is present on several platforms with glibc 2.3.4 or higher.


XXXXXXXXXXXXXXXXXXX

Reported by

Seppo Sahrakorpi
Partner Technologies Engineer
TotalView Technologies, 24 Prime Parkway, Natick, MA 01760
http://www.totalviewtech.com
seppo -- 508-652-7781 

XXXXXXXXXXXXXXXXXX

Comment 1 Seppo Sahrakorpi 2008-04-08 00:54:13 UTC
Created attachment 301583 [details]
Reproducer package

Comment 2 Ulrich Drepper 2008-06-06 20:36:04 UTC
And your point is?  You cannot have the cake and eat it.  It's the problem of
your proprietary technology and not that of glibc.

Comment 3 Seppo Sahrakorpi 2008-06-09 16:43:31 UTC
This is not limited to our malloc call. Attached please find a general
non-malloc reproducer, with the following logic:

  Write a small ia which interposes, e.g., strdup.
    The interposer should announc when it's called.
  Write a small main program which call strdup.
  Write a small dll which calls strdup from an exported fucntion.
  call strdup from main.
  Open the dll from main with RTLD_DEEPBIND.
  Call the exported function to demonstrate that
    the call from the dll is not interposed.

For details, see the 00readme file in bld-linux.

Comment 4 Seppo Sahrakorpi 2008-06-09 16:45:12 UTC
Created attachment 308722 [details]
Non-malloc reproducer

Comment 5 Ulrich Drepper 2008-06-09 17:52:32 UTC
The code works as it is supposed to.  There is nothing wrong except your
expectation.  And I am not going to invent more hooks for proprietary software
to work.


Note You need to log in before you can comment on or make changes to this bug.