Bug 103514 - Linker does not distinguish LD_ variables between 64 Bit and 32 bit Execurables
Linker does not distinguish LD_ variables between 64 Bit and 32 bit Execurables
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: glibc (Show other bugs)
ia64 Linux
medium Severity medium
: ---
: ---
Assigned To: Jakub Jelinek
Brian Brock
Depends On:
  Show dependency treegraph
Reported: 2003-09-01 11:40 EDT by Albert Fluegel
Modified: 2007-11-30 17:06 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2003-09-25 20:19:39 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
modifies the runtime linker to evaluate LD_PRELOAD_64 and LD_LIBRARY_PATH_64 (1.91 KB, patch)
2003-09-01 11:42 EDT, Albert Fluegel
no flags Details | Diff

  None (edit)
Description Albert Fluegel 2003-09-01 11:40:33 EDT
Description of problem:
If i want to make use of LD_PRELOAD or LD_LIBRARY_PATH in an
environment, where either 64 bit binaries and 32 bit binaries
are called or probably call each other, the mixture of found
64 and 32 bit shared objects make things fail constantly.
I'd like the linkers ld.so or ld-linux.so or however they are
called evaluate an environment like on Solaris-8 and 9. There
it looks first for respective environment variables with the
suffix _32 or _64, then if there is none of this kind found,
evaluates the normal variables LD_LIBRARY_PATH or LD_PRELOAD.
So i can set LD_PRELOAD_64 to a different object than
LD_PRELOAD (or if i like to: LD_PRELOAD_32) and everything
is cool. I shall attach a patch file that modifies the rtld.c
to do this. The modification is trivial and backward compatible.
What the patch does not do is to add the variables to the file
sysdeps/generic/unsecvars.h, what is left to be done. I think
this modification is important enough to forward it to the
real glibc-maintainers (if this is not you), at least to
discuss about how the runtime linker should deal with the different
executables. Unfortunately i don't know, how to build the
32 bit compatibility glibc (and thus ld.so) on Opteron or
Itanium from Source. Any hint would be greatly appriciated.

Version-Release number of selected component (if applicable):

How reproducible:
Try to preload a different shared object for 32 bit than for
64 bit executables, e.g. libc.

Steps to Reproduce:
1. setenv LD_PRELOAD /lib/libc.so.6.1
2. run any 32 bit binary
3. error message:
/path/to/program: error while loading shared libraries: /lib/libc.so.6.1: cannot
open shared object file: No such file or directory

Actual results:
Error messages see above

Expected results:
Both 32 and 64 Bit binaries work with their own LD_LIBRARY_PATH_XY

Additional info:
For the first step it is sufficient, that the 64 bit runtime linker
evaluates LD_XYZ_64, if present. The 32 bit compatibility ld.so
can be modified later, if at all, but it would be nice.
Comment 1 Albert Fluegel 2003-09-01 11:42:30 EDT
Created attachment 94117 [details]
modifies the runtime linker to evaluate LD_PRELOAD_64 and LD_LIBRARY_PATH_64

see bug description
(in fact it's a feature request)
Comment 2 Albert Fluegel 2003-09-01 11:54:40 EDT
Comment on attachment 94117 [details]
modifies the runtime linker to evaluate LD_PRELOAD_64 and LD_LIBRARY_PATH_64

>--- glibc-2.3.2-200308141835/elf/rtld.c	2003-09-01 17:35:46.000000000 +0200
>+++ glibc-2.3.2-af/elf/rtld.c	2003-09-01 17:17:34.000000000 +0200
>@@ -1949,6 +1949,8 @@
>   char *envline;
>   enum mode mode = normal;
>   char *debug_output = NULL;
>+  char have_archspec_preload = 0;
>+  char have_archspec_ldlibpath = 0;
>   /* This is the default place for profiling data file.  */
>   GL(dl_profile_output)
>@@ -1990,7 +1992,8 @@
> 	    }
> 	  /* List of objects to be preloaded.  */
>-	  if (memcmp (envline, "PRELOAD", 7) == 0)
>+	  if (!have_archspec_preload
>+			&& memcmp (envline, "PRELOAD", 7) == 0)
> 	    {
> 	      preloadlist = &envline[8];
> 	      break;
>@@ -2023,6 +2026,18 @@
> 	  /* Mask for the important hardware capabilities.  */
> 	  if (memcmp (envline, "HWCAP_MASK", 10) == 0)
> 	    GL(dl_hwcap_mask) = __strtoul_internal (&envline[11], NULL, 0, 0);
>+	  /* List of objects to be preloaded.  */
>+#if	__ELF_NATIVE_CLASS == 64
>+	  if (memcmp (envline, "PRELOAD_64", 10) == 0)
>+	  if (memcmp (envline, "PRELOAD_32", 10) == 0)
>+	    {
>+	      preloadlist = &envline[11];
>+	      have_archspec_preload = 1;
>+	      break;
>+	    }
> 	  break;
> 	case 11:
>@@ -2034,7 +2049,8 @@
> 	case 12:
> 	  /* The library search path.  */
>-	  if (memcmp (envline, "LIBRARY_PATH", 12) == 0)
>+	  if (!have_archspec_ldlibpath
>+			&& memcmp (envline, "LIBRARY_PATH", 12) == 0)
> 	    {
> 	      library_path = &envline[13];
> 	      break;
>@@ -2059,6 +2075,20 @@
> 	    GL(dl_profile_output) = &envline[15];
> 	  break;
>+	case 15:
>+	  /* The library search path.  */
>+#if	__ELF_NATIVE_CLASS == 64
>+	  if (memcmp (envline, "LIBRARY_PATH_64", 15) == 0)
>+	  if (memcmp (envline, "LIBRARY_PATH_32", 15) == 0)
>+	    {
>+	      library_path = &envline[16];
>+	      have_archspec_ldlibpath = 1;
>+	      break;
>+	    }
>+	  break;
> 	case 16:
> 	  /* The mode of the dynamic linker can be set.  */
> 	  if (memcmp (envline, "TRACE_PRELINKING", 16) == 0)
Comment 3 Jakub Jelinek 2003-09-02 08:29:26 EDT
No idea why you mention LD_LIBRARY_PATH, glibc ld.so transparently skips incompatible
libraries in the search path, so there is absolutely no problem with using
identical LD_LIBRARY_PATH between 32-bit and 64-bit programs.
As for LD_PRELOAD, if you don't use absolute paths and both 32-bit and 64-bit
preload library are in LD_LIBRARY_PATH or in dirs searched by default,
it will work too. And for absolute paths you can use $LIB which expands to
lib for 32-bit programs and lib64 for 64-bit programs on bi-arch setups
(so if you put 32-bit preload library in /foo/bar/lib/libfoo.so and
64-bit preload library in /foo/bar/lib64/libfoo.so, then you can use
and it will do the right thing).
One of the shared libraries can be dummy one (echo | gcc -shared -nostdlib -O2 -o libfoo.so -m{32,64}),
so that's how you can handle when you for some reason only need 32-bit or only
need 64-bit preload library.
Comment 4 Albert Fluegel 2003-09-02 09:18:38 EDT
The PRELOAD way of life seems not to work,
at least the way i'm trying here: I have 2 files in some
directory /foo/bar: _dl_loaded_32.so and _dl_loaded_64.so
I add /foo/bar to LD_LIBRARY_PATH and
setenv LD_PRELOAD "_dl_loaded_32.so _dl_loaded_64.so"
When i start any command now, it does not work, no matter
if it's a 64 or 32 bit executable, and immediately one
of the following error messages appear, respectively:
ls: error while loading shared libraries: _dl_loaded_32.so: cannot open shared
object file: No such file or directory
/usr/local/albiutils/bin/rotor: error while loading shared libraries:
_dl_loaded_64.so: cannot open shared object file: No such file or directory

The $LIB thing sounds cool, but it is not mentioned in the ld.so
man page and it seems not wor work either: I did exactly like explained:
setenv LD_PRELOAD '/foo/bar/$LIB/_dl_loaded.so'
and when starting any command the following error message appears:
ls: error while loading shared libraries: /foo/bar/$LIB/_dl_loaded.so: cannot
open shared object file: No such file or directory
but both
/foo/bar/lib/_dl_loaded.so and /foo/bar/lib64/_dl_loaded.so do exist.
Any further hint is appreciated.
Comment 5 Jakub Jelinek 2003-09-02 11:24:30 EDT
It certainly works for me:
$ echo 'int main (void) { return 0; }' | gcc -xc - -m32 -o /tmp/test32
$ echo 'int main (void) { return 0; }' | gcc -xc - -m64 -o /tmp/test64
$ LD_TRACE_LOADED_OBJECTS=1 LD_PRELOAD='/$LIB/libSegFault.so' /tmp/test32
        /$LIB/libSegFault.so => /lib/libSegFault.so (0xa0000000)
        libc.so.6 => /lib/tls/libc.so.6 (0xa001b000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
$ LD_TRACE_LOADED_OBJECTS=1 LD_PRELOAD='/$LIB/libSegFault.so' /tmp/test64
        /$LIB/libSegFault.so => /lib64/libSegFault.so (0x0000002a9566b000)
        libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a95770000)
        /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x0000002a95556000)
$ LD_PRELOAD='/$LIB/libSegFault.so' /tmp/test32; echo $?
$ LD_PRELOAD='/$LIB/libSegFault.so' /tmp/test64; echo $?

As for what you've tried first, LD_PRELOAD is a column separated list of libraries
to load, not space separated. And even if you've changed that, you'd need 32-bit
and 64-bit _dl_loaded_32.so library (and similarly with _dl_loaded_64.so - two
copies as well).
Comment 6 Albert Fluegel 2003-09-02 12:14:41 EDT
Please see yourself:
        /foo/bar/$LIB/_dl_loaded.so => /foo/bar/lib/_dl_loaded.so
        libc.so.6.1 => /lib/tls/libc.so.6.1 (0x2000000000054000)
        /lib/ld-linux-ia64.so.2 => /lib/ld-linux-ia64.so.2 (0x2000000000000000)

test64 is the ia64 binary created like you suggested, but $LIB is
not going to be lib64. So if /foo/bar/lib/_dl_loaded.so is a
32 Bit shared object and not 64 bit, the following happens:
/foo/bar/test64: error while loading shared libraries:
/home/tdscaf/$LIB/_dl_loaded.so: cannot open shared object file: No such file or

When i use the libSegFault example the following happens (this box has
no /lib64, only /emul/ia32-linux, it's Taroon-Beta ia64):
        /$LIB//libSegFault.so => /lib//libSegFault.so (0x2000000000040000)
        libc.so.6.1 => /lib/tls/libc.so.6.1 (0x2000000000054000)
        /lib/ld-linux-ia64.so.2 => /lib/ld-linux-ia64.so.2 (0x2000000000000000)
        /$LIB//libSegFault.so => /lib//libSegFault.so (0x4001c000)
        libc.so.6 => /lib/tls/libc.so.6 (0x4002c000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

so both times /lib/libSegFault.so is found, what seems ok also for the
32 Bit execurable (?!?!?)

I now copied the 64 Bit libSegFault.so to /foo/bar/lib64 and the 32 bit
libSegFault.so to /foo/bar/lib, set LD_PRELOAD to /foo/bar/$LIB/libSegFault.so
and now:
/home/tdscaf/test64: error while loading shared libraries:
/home/tdscaf/$LIB/libSegFault.so: cannot open shared object file: No such file
or directory
        /foo/bar/$LIB/libSegFault.so => /foo/bar/lib/libSegFault.so (0x4001c000)
        libc.so.6 => /lib/tls/libc.so.6 (0x4002c000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

Simply spoken: i never see any resolution of $LIB to lib64.

And BTW:
the manpage of ld.so says nothing about columns concerning LD_PRELOAD,
just 'whitespace':
              A whitespace-separated list of additional,  user-specified,  ELF
              shared  libraries  to  be loaded before all others.  This can be
              used  to  selectively  override  functions   in   other   shared
              libraries.   For  setuid/setgid  ELF binaries, only libraries in
              the standard search directories that are  also  setuid  will  be

Still my patch is the only solution to get things working
here (needed to preload a function because a commercial
software (grrrrr) used an undocumented unofficial symbol
_dl_loaded, which no longer exists in glibc or at least
is no longer external)
Comment 7 Jakub Jelinek 2003-09-02 12:49:20 EDT
Oops, this is on IA-64, sorry, were replying as if you're talking about some
true bi-arch architecture. IA-64 is not, due to bad decisions of linuxia64.org
folks in the past.
For IA-64 though I think putting the 32-bit preload library into
say /emul/ia32-linux/lib and 64-bit preload library into /lib should do the job.
I don't have any IA-64 box with IA-32 libraries installed, so I cannot test
it ATM.
Comment 8 Albert Fluegel 2003-09-02 13:09:16 EDT
I'm sorry, too, that i did not point this out more clearly
earlier. I've tried, what you suggested already last week,
but: strange enough it did not work. In strace i saw, that
the respective shared object is found and opened successfully
(filedescriptor > 0 returned), but a successive error output said:
cannot open `exactly the opened file`, what i did not understand.
Furthermore i don't want to have to put user level shared objects
into system directories.
Thus my emergency measure with _64 to work around all this
strange stuff. I guess, my linker extension won't make it's
way into the glibc source ? Though i still think it makes sense
to give people a chance to work around the phenomenon we're
experiencing here. Sigh. On opteron everything's so cool, but
this Itanium stuff ...
Comment 9 Matt Wilson 2003-09-25 20:19:39 EDT
if you have a DSO in both locations, i.e.:

[root@katzj-ia64 root]# locate libSegFault.so
[root@katzj-ia64 root]# file hello
hello: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux
2.2.5, dynamically linked (uses shared libs), not stripped
[root@katzj-ia64 root]# LD_PRELOAD=/lib/libSegFault.so ./hello
Hello, world!
[root@katzj-ia64 root]# LD_PRELOAD=/lib/libSegFault.so LD_TRACE_LOADED_OBJECTS=1
        /lib/libSegFault.so => /lib/libSegFault.so (0x40018000)
        libc.so.6 => /lib/tls/libc.so.6 (0x40034000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
[root@katzj-ia64 root]# LD_PRELOAD=/lib/libSegFault.so /bin/echo hello

(the kernel magically fools ia32 proceses into thinking that files in
/emul/ia32-linux/ are actually in /)

So -- to fix this, simply install a dummy DSO for the arch that does not need

Note You need to log in before you can comment on or make changes to this bug.