Red Hat Bugzilla – Bug 103514
Linker does not distinguish LD_ variables between 64 Bit and 32 bit Execurables
Last modified: 2007-11-30 17:06:57 EST
Description of problem: If i want to make use of LD_PRELOAD or LD_LIBRARY_PATH in an environment, where either 64 bit binaries and 32 bit binaries are called or probably call each other, the mixture of found 64 and 32 bit shared objects make things fail constantly. I'd like the linkers ld.so or ld-linux.so or however they are called evaluate an environment like on Solaris-8 and 9. There it looks first for respective environment variables with the suffix _32 or _64, then if there is none of this kind found, evaluates the normal variables LD_LIBRARY_PATH or LD_PRELOAD. So i can set LD_PRELOAD_64 to a different object than LD_PRELOAD (or if i like to: LD_PRELOAD_32) and everything is cool. I shall attach a patch file that modifies the rtld.c to do this. The modification is trivial and backward compatible. What the patch does not do is to add the variables to the file sysdeps/generic/unsecvars.h, what is left to be done. I think this modification is important enough to forward it to the real glibc-maintainers (if this is not you), at least to discuss about how the runtime linker should deal with the different executables. Unfortunately i don't know, how to build the 32 bit compatibility glibc (and thus ld.so) on Opteron or Itanium from Source. Any hint would be greatly appriciated. Version-Release number of selected component (if applicable): 2.3.2-74 How reproducible: Try to preload a different shared object for 32 bit than for 64 bit executables, e.g. libc. Steps to Reproduce: 1. setenv LD_PRELOAD /lib/libc.so.6.1 2. run any 32 bit binary 3. error message: /path/to/program: error while loading shared libraries: /lib/libc.so.6.1: cannot open shared object file: No such file or directory Actual results: Error messages see above Expected results: Both 32 and 64 Bit binaries work with their own LD_LIBRARY_PATH_XY or LD_PRELOAD_XY Additional info: For the first step it is sufficient, that the 64 bit runtime linker evaluates LD_XYZ_64, if present. The 32 bit compatibility ld.so can be modified later, if at all, but it would be nice.
Created attachment 94117 [details] modifies the runtime linker to evaluate LD_PRELOAD_64 and LD_LIBRARY_PATH_64 see bug description (in fact it's a feature request)
Comment on attachment 94117 [details] modifies the runtime linker to evaluate LD_PRELOAD_64 and LD_LIBRARY_PATH_64 >--- glibc-2.3.2-200308141835/elf/rtld.c 2003-09-01 17:35:46.000000000 +0200 >+++ glibc-2.3.2-af/elf/rtld.c 2003-09-01 17:17:34.000000000 +0200 >@@ -1949,6 +1949,8 @@ > char *envline; > enum mode mode = normal; > char *debug_output = NULL; >+ char have_archspec_preload = 0; >+ char have_archspec_ldlibpath = 0; > > /* This is the default place for profiling data file. */ > GL(dl_profile_output) >@@ -1990,7 +1992,8 @@ > } > > /* List of objects to be preloaded. */ >- if (memcmp (envline, "PRELOAD", 7) == 0) >+ if (!have_archspec_preload >+ && memcmp (envline, "PRELOAD", 7) == 0) > { > preloadlist = &envline[8]; > break; >@@ -2023,6 +2026,18 @@ > /* Mask for the important hardware capabilities. */ > if (memcmp (envline, "HWCAP_MASK", 10) == 0) > GL(dl_hwcap_mask) = __strtoul_internal (&envline[11], NULL, 0, 0); >+ >+ /* List of objects to be preloaded. */ >+#if __ELF_NATIVE_CLASS == 64 >+ if (memcmp (envline, "PRELOAD_64", 10) == 0) >+#else >+ if (memcmp (envline, "PRELOAD_32", 10) == 0) >+#endif >+ { >+ preloadlist = &envline[11]; >+ have_archspec_preload = 1; >+ break; >+ } > break; > > case 11: >@@ -2034,7 +2049,8 @@ > > case 12: > /* The library search path. */ >- if (memcmp (envline, "LIBRARY_PATH", 12) == 0) >+ if (!have_archspec_ldlibpath >+ && memcmp (envline, "LIBRARY_PATH", 12) == 0) > { > library_path = &envline[13]; > break; >@@ -2059,6 +2075,20 @@ > GL(dl_profile_output) = &envline[15]; > break; > >+ case 15: >+ /* The library search path. */ >+#if __ELF_NATIVE_CLASS == 64 >+ if (memcmp (envline, "LIBRARY_PATH_64", 15) == 0) >+#else >+ if (memcmp (envline, "LIBRARY_PATH_32", 15) == 0) >+#endif >+ { >+ library_path = &envline[16]; >+ have_archspec_ldlibpath = 1; >+ break; >+ } >+ break; >+ > case 16: > /* The mode of the dynamic linker can be set. */ > if (memcmp (envline, "TRACE_PRELINKING", 16) == 0)
No idea why you mention LD_LIBRARY_PATH, glibc ld.so transparently skips incompatible libraries in the search path, so there is absolutely no problem with using identical LD_LIBRARY_PATH between 32-bit and 64-bit programs. As for LD_PRELOAD, if you don't use absolute paths and both 32-bit and 64-bit preload library are in LD_LIBRARY_PATH or in dirs searched by default, it will work too. And for absolute paths you can use $LIB which expands to lib for 32-bit programs and lib64 for 64-bit programs on bi-arch setups (so if you put 32-bit preload library in /foo/bar/lib/libfoo.so and 64-bit preload library in /foo/bar/lib64/libfoo.so, then you can use LD_PRELOAD='/foo/bar/$LIB/libfoo.so' and it will do the right thing). One of the shared libraries can be dummy one (echo | gcc -shared -nostdlib -O2 -o libfoo.so -m{32,64}), so that's how you can handle when you for some reason only need 32-bit or only need 64-bit preload library.
The PRELOAD way of life seems not to work, at least the way i'm trying here: I have 2 files in some directory /foo/bar: _dl_loaded_32.so and _dl_loaded_64.so I add /foo/bar to LD_LIBRARY_PATH and setenv LD_PRELOAD "_dl_loaded_32.so _dl_loaded_64.so" When i start any command now, it does not work, no matter if it's a 64 or 32 bit executable, and immediately one of the following error messages appear, respectively: ls: error while loading shared libraries: _dl_loaded_32.so: cannot open shared object file: No such file or directory or: /usr/local/albiutils/bin/rotor: error while loading shared libraries: _dl_loaded_64.so: cannot open shared object file: No such file or directory The $LIB thing sounds cool, but it is not mentioned in the ld.so man page and it seems not wor work either: I did exactly like explained: setenv LD_PRELOAD '/foo/bar/$LIB/_dl_loaded.so' and when starting any command the following error message appears: ls: error while loading shared libraries: /foo/bar/$LIB/_dl_loaded.so: cannot open shared object file: No such file or directory but both /foo/bar/lib/_dl_loaded.so and /foo/bar/lib64/_dl_loaded.so do exist. Any further hint is appreciated.
It certainly works for me: $ echo 'int main (void) { return 0; }' | gcc -xc - -m32 -o /tmp/test32 $ echo 'int main (void) { return 0; }' | gcc -xc - -m64 -o /tmp/test64 $ LD_TRACE_LOADED_OBJECTS=1 LD_PRELOAD='/$LIB/libSegFault.so' /tmp/test32 /$LIB/libSegFault.so => /lib/libSegFault.so (0xa0000000) libc.so.6 => /lib/tls/libc.so.6 (0xa001b000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) $ LD_TRACE_LOADED_OBJECTS=1 LD_PRELOAD='/$LIB/libSegFault.so' /tmp/test64 /$LIB/libSegFault.so => /lib64/libSegFault.so (0x0000002a9566b000) libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a95770000) /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x0000002a95556000) $ LD_PRELOAD='/$LIB/libSegFault.so' /tmp/test32; echo $? 0 $ LD_PRELOAD='/$LIB/libSegFault.so' /tmp/test64; echo $? 0 As for what you've tried first, LD_PRELOAD is a column separated list of libraries to load, not space separated. And even if you've changed that, you'd need 32-bit and 64-bit _dl_loaded_32.so library (and similarly with _dl_loaded_64.so - two copies as well).
Please see yourself: /tmp/test64 /foo/bar/$LIB/_dl_loaded.so => /foo/bar/lib/_dl_loaded.so (0x2000000000040000) libc.so.6.1 => /lib/tls/libc.so.6.1 (0x2000000000054000) /lib/ld-linux-ia64.so.2 => /lib/ld-linux-ia64.so.2 (0x2000000000000000) test64 is the ia64 binary created like you suggested, but $LIB is not going to be lib64. So if /foo/bar/lib/_dl_loaded.so is a 32 Bit shared object and not 64 bit, the following happens: /tmp/test64 /foo/bar/test64: error while loading shared libraries: /home/tdscaf/$LIB/_dl_loaded.so: cannot open shared object file: No such file or directory When i use the libSegFault example the following happens (this box has no /lib64, only /emul/ia32-linux, it's Taroon-Beta ia64): /tmp/test64 /$LIB//libSegFault.so => /lib//libSegFault.so (0x2000000000040000) libc.so.6.1 => /lib/tls/libc.so.6.1 (0x2000000000054000) /lib/ld-linux-ia64.so.2 => /lib/ld-linux-ia64.so.2 (0x2000000000000000) /tmp/test32 /$LIB//libSegFault.so => /lib//libSegFault.so (0x4001c000) libc.so.6 => /lib/tls/libc.so.6 (0x4002c000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) so both times /lib/libSegFault.so is found, what seems ok also for the 32 Bit execurable (?!?!?) I now copied the 64 Bit libSegFault.so to /foo/bar/lib64 and the 32 bit libSegFault.so to /foo/bar/lib, set LD_PRELOAD to /foo/bar/$LIB/libSegFault.so and now: /tmp/test64 /home/tdscaf/test64: error while loading shared libraries: /home/tdscaf/$LIB/libSegFault.so: cannot open shared object file: No such file or directory /tmp/test32 /foo/bar/$LIB/libSegFault.so => /foo/bar/lib/libSegFault.so (0x4001c000) libc.so.6 => /lib/tls/libc.so.6 (0x4002c000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) Simply spoken: i never see any resolution of $LIB to lib64. And BTW: the manpage of ld.so says nothing about columns concerning LD_PRELOAD, just 'whitespace': LD_PRELOAD A whitespace-separated list of additional, user-specified, ELF shared libraries to be loaded before all others. This can be used to selectively override functions in other shared libraries. For setuid/setgid ELF binaries, only libraries in the standard search directories that are also setuid will be loaded. Still my patch is the only solution to get things working here (needed to preload a function because a commercial software (grrrrr) used an undocumented unofficial symbol _dl_loaded, which no longer exists in glibc or at least is no longer external)
Oops, this is on IA-64, sorry, were replying as if you're talking about some true bi-arch architecture. IA-64 is not, due to bad decisions of linuxia64.org folks in the past. For IA-64 though I think putting the 32-bit preload library into say /emul/ia32-linux/lib and 64-bit preload library into /lib should do the job. I don't have any IA-64 box with IA-32 libraries installed, so I cannot test it ATM.
I'm sorry, too, that i did not point this out more clearly earlier. I've tried, what you suggested already last week, but: strange enough it did not work. In strace i saw, that the respective shared object is found and opened successfully (filedescriptor > 0 returned), but a successive error output said: cannot open `exactly the opened file`, what i did not understand. Furthermore i don't want to have to put user level shared objects into system directories. Thus my emergency measure with _64 to work around all this strange stuff. I guess, my linker extension won't make it's way into the glibc source ? Though i still think it makes sense to give people a chance to work around the phenomenon we're experiencing here. Sigh. On opteron everything's so cool, but this Itanium stuff ...
if you have a DSO in both locations, i.e.: [root@katzj-ia64 root]# locate libSegFault.so /lib/libSegFault.so /emul/ia32-linux/lib/libSegFault.so [root@katzj-ia64 root]# file hello hello: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, dynamically linked (uses shared libs), not stripped [root@katzj-ia64 root]# LD_PRELOAD=/lib/libSegFault.so ./hello Hello, world! [root@katzj-ia64 root]# LD_PRELOAD=/lib/libSegFault.so LD_TRACE_LOADED_OBJECTS=1 ./hello /lib/libSegFault.so => /lib/libSegFault.so (0x40018000) libc.so.6 => /lib/tls/libc.so.6 (0x40034000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) [root@katzj-ia64 root]# LD_PRELOAD=/lib/libSegFault.so /bin/echo hello hello (the kernel magically fools ia32 proceses into thinking that files in /emul/ia32-linux/ are actually in /) So -- to fix this, simply install a dummy DSO for the arch that does not need the LD_PRELOAD.