Description of problem: I have no idea if it's intentional or not, but the MM_Unix.pm module in the Fedora distribution has removed support for LD_RUN_PATH. This is necessary when building XS modules that uses non-standard library paths (for instance). I don't know the reasons why we'd remove this, since the standard distribution of MakeMaker certainly supports it (I've verified it by building perl from original source). Diffing Fedora MM_Unix.pm vs my source built version shows: --- /export/exec/perl/lib/5.8.5/ExtUtils/MM_Unix.pm 2004-10-14 16:39:08.000000000 -0700 +++ /usr/lib/perl5/5.8.5/ExtUtils/MM_Unix.pm 2004-08-27 12:06:25.000000000 -0700 @@ -1135,7 +1135,7 @@ } push(@m, -' LD_RUN_PATH="$(LD_RUN_PATH)" $(LD) '.$ldrun.' $(LDDLFLAGS) '.$ldfrom. +' $(LD) '.$ldrun.' $(LDDLFLAGS) '.$ldfrom. ' $(OTHERLDFLAGS) -o $@ $(MYEXTLIB) $(PERL_ARCHIVE) '.$libs.' $(PERL_ARCHIVE_AFTER) $(EXPORT_LIST) $(INST_DYNAMIC_FIX)'); push @m, ' $(CHMOD) $(PERM_RWX) $@ Version-Release number of selected component (if applicable): perl-5.8.5-4 How reproducible: Building an XS module that uses libraries from a non-system lib directory. This triggers ExtUtils::Liblist to return an array where the LD_RUN_PATH component is set. Actual results: The final link command using Fedora Perl is something like: gcc -shared -L/usr/local/csdk/lib API.o ... Expected results: LD_RUN_PATH="/usr/local/csdk/lib" gcc -shared -L/usr/...
List, should this be considered an unsupported configuration?
I wont do anything here without more feedback from others.
Fwiw, this is done with the patch named perl-5.8.3-empty-rpath.patch Anyone know why we do this? Maybe the intention was to remove the LD_RUN_PATH environment variable only when $LD_RUN_PATH is empty? That would make sense, but then this patch isn't really correct. :)
IIRC it has to do with empty RPATHs (or RPATH components) and possibly buildroots having been seen included in them in some cases earlier. Coincidentally, I happened to notice this yesterday: http://seclists.org/lists/fulldisclosure/2005/Oct/0405.html http://www.gentoo.org/cgi-bin/viewcvs.cgi/dev-lang/perl/files/perl-5.8.7-MakeMaker-RUNPATH.patch?rev=. http://bugs.gentoo.org/show_bug.cgi?id=105054 (I guess $PORTAGE_TMPDIR is roughly equivalent to $RPM_BUILD_ROOT) If I understand correctly, the patch seems to be a somewhat draconian approach to the problem indeed, but it needs to be fixed carefully due to possible security issues involved. Maybe Chip could provide more insight? In the meantime, a workaround is obviously to add those non-system lib dirs to eg. /etc/ld.so.conf.d/local.conf
(In reply to comment #4) > If I understand correctly, the patch seems to be a somewhat draconian By this I mean the patch in the FC perl package. Also: * Fri Apr 16 2004 Chip Turner <cturner> 3:5.8.3-18 - add patch to fix empty RPATH issue on perl module compile
You can use perl-5.8.5-removeemptyrpath.patch from the Mandriva perl instead. It has been integrated (with some enhancements) upstream, in the CPAN distribution of ExtUtils::MakeMaker.
This is now fixed - MM_Unix uses upstream method of preventing use of empty LD_RUN_PATH while allowing non-empty LD_RUN_PATH to be used. Patch applied in: perl-5.8.7-7 (FC5), perl-5.8.6-16 (FC4), perl-5.8.5-17 (RHEL4), perl-5.8.0-90 (RHEL-3).
From User-Agent: XML-RPC perl-5.8.5-18.FC3 has been pushed for FC3, which should resolve this issue. If these problems are still present in this version, then please make note of it in this bug report.
The results of this don't look good; it seems that now _every_ library path apart from the empty one are turned into hardwired rpaths. For example, see the subversion-perl package in Rawhide; all the *.so in it contains loads of rpaths with buildroot traces (/usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/...) Note that this is a security issue for practically everyone who's following good rpm rebuilding guidelines and doing it as non-root. Another example would be to rebuild perl-Compress-Zlib with the new perl and witness how /usr/lib, a standard system dir, is unnecessarily stuffed into a rpath. The buildroot traces part of the problem could probably be handled with something like the Gentoo approach in comment 4, and the latter part by teaching MM_Unix about standard dynamic linker paths.
This is a regression, can you revert this change until something better is found? We will start accumulating packages containing binaries with bogus RPATHs again for as long as this perl package remains in the buildoot.
The default behaviour of the unmodified MakeMaker package, and the latest 6.30 upstream version, is to add every directory specified in the MakerMaker object's LIBS into the LD_RUN_PATH environment variable setting used by the ld(1) command, as documented in 'man ExtUtils::Liblist' : " LDLOADLIBS and LD_RUN_PATH List of those libraries which can or must be linked into the shared library when created using ld. These may be static or dynamic libraries. LD_RUN_PATH is a colon separated list of the directories in LDLOADLIBS. It is passed as an environment variable to the process that links the shared library. " So the ExtUtils::Liblist::Kid::_unix_os2_ext function creates the LD_RUN_PATH setting from the list of libraries to be linked. IF no explicit -rpath argument is given to ld(1), it will use a non-empty LD_RUN_PATH environment variable setting as the object's RPATH . Previously, we had removed use of LD_RUN_PATH by MakeMaker altogether with the 'perl-5.8.3-empty-rpath.patch' - this was somewhat draconian, especially as it contravenes the shipped documentation about LD_RUN_PATH usage. We've now restored the default upstream MakeMaker LD_RUN_PATH behavior, with a patch to remove an empty LD_RUN_PATH setting similar to that suggested by Rafael in Comment #6 above with Mandriva's perl-5.8.5-removeemptyrpath.patch, which is now in the upstream MakeMaker 6.30 release . LD_RUN_PATH usage is still entirely under the control of module developers . The Red Hat build root environment contains no LD_RUN_PATH setting . It is not the case that "we will start accumulating packages containing binaries with bogus RPATHs" - only if packages pass LIBS with absolute full paths into the buildroot will this occur, and if it does occur, it is only a problem if the libraries are shipped in non-standard locations, in which case it is also a module packaging issue, and would not be corrected by reverting the change . With subversion-perl, for instance, it contains paths into the build root in its Makefile.PL LIBS setting : --- ... my @ldpaths = ("$swig_builddir/perl/libsvn_swig_perl/.libs", map {"$svnlib_builddir/libsvn_$_/.libs"} (@modules, qw/diff subr ra_local ra_svn ra_dav fs_base fs_fs/)); ... my %config = ( ... LIBS => [join(' ', $apr_ldflags, (map {$_ = abs_path($_); "-L$_"} @ldpaths), @ldmodules, '-lsvn_swig_perl-1', `$swig -perl -ldflags`)], ... ); ... WriteMakefile(%config, ... --- And the subversion-perl .so objects end up with these horrible RPATH settings: $ objdump -x ./vendor_perl/5.8.7/i386-linux-thread-multi/auto/SVN/_Core/_Core.so | grep RPATH RPATH /usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/libsvn_client/.libs:/usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/libsvn_delta/.libs:/usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/libsvn_fs/.libs:/usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/libsvn_ra/.libs:/usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/libsvn_repos/.libs:/usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/libsvn_wc/.libs:/usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/libsvn_diff/.libs:/usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/libsvn_subr/.libs:/usr/src/build/652335-i386/BUILD/subversion-1.2.3/subversion/bindings/swig/perl/libsvn_swig_perl/.libs But the loader still finds the SVN libraries, because they are shipped in the default system library path : $ ldd ./vendor_perl/5.8.7/i386-linux-thread-multi/auto/SVN/_Core/_Core.so linux-gate.so.1 => (0x00b71000) libsvn_client-1.so.0 => /usr/lib/libsvn_client-1.so.0 (0x00790000) libsvn_delta-1.so.0 => /usr/lib/libsvn_delta-1.so.0 (0x00bb6000) libsvn_fs-1.so.0 => /usr/lib/libsvn_fs-1.so.0 (0x006eb000) libsvn_ra-1.so.0 => /usr/lib/libsvn_ra-1.so.0 (0x00bea000) libsvn_repos-1.so.0 => /usr/lib/libsvn_repos-1.so.0 (0x00ca0000) libsvn_wc-1.so.0 => /usr/lib/libsvn_wc-1.so.0 (0x00cfd000) libsvn_diff-1.so.0 => /usr/lib/libsvn_diff-1.so.0 (0x00e10000) libsvn_subr-1.so.0 => /usr/lib/libsvn_subr-1.so.0 (0x00801000) libsvn_swig_perl-1.so.0 => /usr/lib/libsvn_swig_perl-1.so.0 (0x00b76000) ... What subversion-perl could have done was specify its list of build tree .libs library locations in $LD_LIBRARY_PATH and specify its libs only as -l options - then no RPATH would have being inserted in the .so objects. So this issue creates no problems for perl module shared objects which link to libraries shipped in the standard locations; also, modules are now enabled to link to libraries that are not in standard locations, simply by putting the full path to the library in the MakeMaker object's LIBS . RE: > Another example would be to rebuild perl-Compress-Zlib with the new perl and > witness how /usr/lib, a standard system dir, is unnecessarily stuffed into a > rpath. I don't see this problem when building with the latest perl on FC-3, FC-4, FC-5, or RHEL-4: $ rpm -q perl-Compress-zlib perl-Compress-zlib-1.41-1 $ objdump -x /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi/auto/Compress/Zlib/Zlib.so | grep RPATH $ cd ~/.cpan/build/Compress-Zlib-1.41 $ perl Makefile.PL ... $ make ... $ objdump -x ./blib/arch/auto/Compress/Zlib/Zlib.so | grep RPATH $ So, in conclusion, I think we should retain the default upstream MakeMaker LD_RUN_PATH behaviour, allowing module developers to use libraries in non-standard locations, and also requiring module developers to be careful about what paths they include in LIBS . This issue is not a "regression", as no actual problems are caused by it, and perl now has default upstream behaviour in this respect; perhaps it was a "regression" when the perl-5.8.3-empty-rpath.patch removed support for LD_RUN_PATH contrary to the shipped perl documentation, and this "regression" has now been corrected.
ld does not search LD_LIBRARY_PATH at compile time, that's not feasible. It seems entirely broken to assume that an RPATH is needed for every library search path specified in any case.
(In reply to comment #12) > > Another example would be to rebuild perl-Compress-Zlib with the new perl and > > witness how /usr/lib, a standard system dir, is unnecessarily stuffed into a > > rpath. > I don't see this problem when building with the latest perl on > FC-3, FC-4, FC-5, or RHEL-4: [...] I get different results, this is with FC5test1 updated to latest everything from Rawhide, in a FC devel CVS tree checkout: [scop@gk012 perl-Compress-Zlib]$ make compile % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 148k 100 148k 0 0 78862 0 0:00:01 0:00:01 --:--:-- 100k -rw-r--r-- 1 scop scop 151972 Dec 7 00:53 Compress-Zlib-1.41.tar.gz rpmbuild --define [...] [...] + exit 0 [scop@gk012 perl-Compress-Zlib]$ objdump -x Compress-Zlib-1.41/blib/arch/auto/Compress/Zlib/Zlib.so | grep RPATH RPATH /usr/lib $ rpm -q perl perl-5.8.7-0.8.fc5
This is very strange. I also have an FC5test1 system updated to latest everything, and I've tried the Compress::Zlib build, both inside and out of CPAN shell, both as root and a non-root user, and I get no RPATH . Aha! I just noticed you are building from the SRPM - I was building the upstream CPAN module tarball . I see now the answer: the perl-Compress-Zlib.spec file explicitly sets the ZLIB_LIB environment variable to the full /usr/lib (on i386) path . This is presumably to stop builds on certain 64-bit platforms from picking up /usr/lib64/libz* . So because of the LD_RUN_PATH feature now being enabled in perl, this setting is now able to have the intended effect : the Zlib.so will explicitly link to /usr/lib/libz.so* instead of /usr/lib64/libz.so*, even if /usr/lib64 precedes /usr/lib in a LD_LIBRARY_PATH setting in the link-time environment - this would not have been the case with only the perl-5.8.3-empty-rpath.patch applied to perl, meaning that no RPATH would have been inserted in Zlib.so .
(In reply to comment #13) > ld does not search LD_LIBRARY_PATH at compile time, that's not feasible. > Yes, ld DOES use LD_LIBRARY_PATH if no LD_RUN_PATH or -rpath option is supplied: From man ld(1): " The linker uses the following search paths to locate required shared libraries. 1. Any directories specified by -rpath-link options. 2. Any directories specified by -rpath options. The difference between -rpath and -rpath-link is that directories specified by -rpath options are included in the executable and used at runtime, whereas the -rpath-link option is only effective at link time. It is for the native linker only. 3. On an ELF system, if the -rpath and "rpath-link" options were not used, search the contents of the environment variable "LD_RUN_PATH". It is for the native linker only. "( And then an RPATH header is inserted in the object ! )" 4. On SunOS, if the -rpath option was not used, search any directories specified using -L options. 5. For a native linker, the contents of the environment variable "LD_LIBRARY_PATH". 6. For a native ELF linker, the directories in "DT_RUNPATH" or "DT_RPATH" of a shared library are searched for shared libraries needed by it. The "DT_RPATH" entries are ignored if "DT_RUNPATH" entries exist. " There are thus many ways to avoid specifying absolute paths to libraries in LIBS and getting an RPATH inserted by MakeMaker generating LD_RUN_PATH . > It seems entirely broken to assume that an RPATH is needed for every library > search path specified in any case. Well, this is the way the upstream MakeMaker is designed and documented to work. It's not really unreasonable to those programmers less familiar with the guts of C program building - if you specify a fully qualified absolute path to a library in LIBS, MakeMaker will try to ensure that the path is stored in the resultant object with LD_RUN_PATH / RPATH . There are many ways to avoid using full paths to libraries, eg. by using only '-l' options and LD_LIBRARY_PATH.
No, really, it doesn't. That describes the algorithm used when ld searches for dependent shared libraries (e.g. when you link against -lfoo and -lfoo depends on -lbar, that is how ld will go and find -lbar). But that's entirely irrelevant. So I still see no feasible way of avoiding the bogus RPATHs.
OK, I'll submit to popular request, and restore the previous default behavior of previous Red Hat perl releases to remove the MakeMaker generated LD_RUN_PATH by default . I'll ensure that users are still able to set the LD_RUN_PATH environment variable during builds and that it will take effect if non empty . Also, as this removal of the MakeMaker generated LD_RUN_PATH contravenes the default upstream behavior and documentation, I'll ammend the MakeMaker POD documentation to explicitly state that the Red Hat distribution removes the MakeMaker generated LD_RUN_PATH. I think also we should give users a means to direct that the MakeMaker generated LD_RUN_PATH should be used; I'll add a "USE_MM_LD_RUN_PATH" member of the MakeMaker object that can be set , and document this also. Please send any objections / suggestions ASAP - thanks .
Not an objection, but a comment/question: Would -rpath-link=/some/where be the "correct" thing to use in cases like subversion-perl and libapreq2 (in Extras) where a lib needs to be linked with another one produced during the same build, but not found without specifying the directory to them? If so, ExtUtils::Liblist::Kid::_unix_os2_ext() could perhaps be taught about that (+ the corresponding -Wl,rpath-link,...), which it would process more or less like -L apart from not stuffing it into LD_RUN_PATH. Currently if one tries to use -Wl,rpath-link, MM doesn't recognize the option which seems to eventually lead to libs in that dir not being found by _unix_os2_ext() and consequently dropped from the list of libs to actually link with.
I personally find it very unfortunate that we're not compatible with upstream behaviour of MakeMaker. But if the general consensus here is to not support this feature, I'll obviously have to yield. Adding documentation and at least a way to code around this is definitely a good thing.
RE: Comment #20: Yes, I think one should easily be able to restore the generated LD_RUN_PATH usage EITHER by setting the USE_MM_LD_RUN_PATH MM object member OR by setting a USE_MM_LD_RUN_PATH environment variable, so no scripts would have to be modified. I'll ensure you'll still be able to use an explicit LD_RUN_PATH environment variable setting during a MakeMaker build. I think it is rather broken that EVERY -L option becomes a member of LD_RUN_PATH - all that would have to happen is for a library not to be installed in the system library directories, and for the build tree to have disappeared, and then the app won't link. It is also unreasonable to install an RPATH header containing references to the non-existent build tree directories in EVERY object created using -L during a system build of all perl modules in the Red Hat build tree. RE: Comment #19 : Interesting point about -rpath-link, but on further investigation I think probably MakeMaker is doing the right thing by ignoring the -rpath-link option - it is only of relevance to the linker in resolving dependencies of libraries linked to -l, not in resolving the -l library locations themselves. -rpath-link is no replacement for -L : it is used only when doing a "non-shared, non-relocatable link" - ie. an executable - to resolve dependencies of one shared library, linked to with -l (and presumably found using -L or -rpath ) on another shared library, NOT specified explicitly with -l . -rpath-link operates exactly like LD_LIBRARY_PATH, and LD_LIBRARY_PATH will be used for this purpose if the library is not found in directories specified by any of the -rpath, -L, or -rpath-link options. ie if you have this setup: executble ./t links to -lg and calls g() in lg/libg.so; ./lg/libg.so links to -lf, which should resolve to ./lf/libf.so, and g() calls f() in libf.so If you produce libg.so as follows: $ cd lg $ gcc -shared -o libg.so g.o -L../lf -lf and then you try to link t: $ gcc -o t t.o -Llg -lg /usr/bin/ld: warning libf.so, needed by ld/libg.so, not found (try using -rpath or -rpath-link) lg/libg.so: undefined reference to `f' This can be resolved by doing: $ LD_LIBRARY_PATH=lf gcc -o t t.o -Llg -lg OR by doing $ gcc -o t t.o -Wl,-rpath-link,lf -Llg -lg OR if libgo.so was linked with $ gcc -o libg.so -shared -Wl,-rpath,lf -L../lf g.o -lf and t could then be linked with: $ gcc -o t m.o -Llg -lg But -rpath-link would NOT be used when linking libg.so with libf.so: $ cd lg; $ gcc -o libg.so -shared -Wl,-rpath-link,../lf -lf /usr/bin/ld: cannot find -lf You HAVE to use -L../lf or -Wl,-rpath,../lf for the libg.so link to succeed. So I'm not sure that MakeMaker could do anything useful with -rpath-link options - if you disagree, please let me know.
If I may ask a question similar to the one in comment #19, but Perl unrelated: what is the recommended way of building a package that ships both a library one of its binaries depends on and the binary itself? I bumped into this problem a while back when building (on current Rawhide) one of my (still not in FE) packages. Basically, this is the situation: Package is supposed to build and distribute: - a binary (which happens to be an Apache module) - a library to which the binary links, which will eventually be installed into the system location (i.e. /usr/lib or /usr/lib64) During the build (in the user's home directory), the build process encodes RPATH of the library (something like /home/<user>/rpmbuild/blah/blah) into the binary. One of the build scripts then checks rpaths and bombs out, saying that you can't have that kind of stuff in the shipped binary. Which is, of course correct, but I never asked for those to be encode in the first place ;-) Is there a way to strip RPATHs from the libary or force some RPATHs not be picked in some "proper" manner? I currently do this kind of trickery in my Makefile.am to avoid the situation (basically I relink with RPATHs stripped): eval "`eval \"echo $(LINK) $(mod_spin_la_LDFLAGS) $(mod_spin_la_OBJECTS) $(mod_spin_la_LIBADD) $(LIBS) | sed -e s/'libtool'/'libtool -n'/ -e s/'install-exec-hook'/'mod_spin.la'/g\" | sed -e s@'-Wl,[-]\+rpath -Wl,[^ ]\+ '@''@g -e s@'-L\(/usr\)*/lib\(64\)*\($$\| \)'@''@g`" Ugly, error prone and most likely not generic enough. Suggestions, solutions and advice welcome. Please be gentle, I'm no expert in this (as if that wasn't obviuos enough already :-)
From User-Agent: XML-RPC perl-5.8.6-22 has been pushed for FC4, which should resolve this issue. If these problems are still present in this version, then please make note of it in this bug report.
From User-Agent: XML-RPC perl-5.8.5-22.FC3 has been pushed for FC3, which should resolve this issue. If these problems are still present in this version, then please make note of it in this bug report.