Bug 157126

Summary: ld --as-needed removes needed reference
Product: [Fedora] Fedora Reporter: Neil Conway <neilc>
Component: binutilsAssignee: Jakub Jelinek <jakub>
Status: CLOSED NOTABUG QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 3CC: db, tgl, twanger
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2005-05-20 09:58:21 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
test case none

Description Neil Conway 2005-05-07 06:23:46 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.6) Gecko/20050317 Firefox/0.10 StumbleUpon/1.998

Description of problem:
When using the --as-needed linker flag, `ld' elides a reference to a library that is actually used by the produced executable (resulting in a "symbol lookup error" on startup). This behavior has been observed with libreadline and libtermcap on FC3; I'm not sure if it is specific to those libraries or a more general problem.

See the attached test_case.c for a trivial example.

Version-Release number of selected component (if applicable):
binutils-2.15.92.0.2-5

How reproducible:
Always

Steps to Reproduce:
1. gcc -c -o test_case.o test_case.c
2. gcc -o test_case test_case.o -Wl,--as-needed -lreadline -ltermcap
3. ./test_case

Actual Results:  ./test_case: symbol lookup error: /usr/lib/libreadline.so.4: undefined symbol: BC


Expected Results:  The executable should startup successfully, print "Hello, world: " and block waiting for console input.

Additional info:

ldd on the resulting executable yields:

$ ldd test_case
        libreadline.so.4 => /usr/lib/libreadline.so.4 (0x00b28000)
        libc.so.6 => /lib/tls/libc.so.6 (0x006db000)
        /lib/ld-linux.so.2 (0x006be000)

Naturally, the `BC' symbol referenced by libreadline can be found in libtermcap, which is not linked in when using --as-needed. If --as-needed is not used, libtermcap is linked against and the resulting executable works:

$ gcc -o test_case test_case.o -lreadline -ltermcap
$ ldd test_case
        libreadline.so.4 => /usr/lib/libreadline.so.4 (0x00b28000)
        libtermcap.so.2 => /lib/libtermcap.so.2 (0x0059b000)
        libc.so.6 => /lib/tls/libc.so.6 (0x006db000)
        /lib/ld-linux.so.2 (0x006be000)
$ ./test_case 
hello, world:

Comment 1 Neil Conway 2005-05-07 06:25:13 UTC
Created attachment 114114 [details]
test case

Comment 2 Tom Lane 2005-05-07 14:49:37 UTC
I can confirm having seen this on FC3.

Comment 3 Jakub Jelinek 2005-05-20 09:58:21 UTC
It works as documented:
`--as-needed'
`--no-as-needed'
     This option affects ELF DT_NEEDED tags for dynamic libraries
     mentioned on the command line after the `--as-needed' option.
     Normally, the linker will add a DT_NEEDED tag for each dynamic
     library mentioned on the command line, regardless of whether the
     library is actually needed. `--as-needed' causes DT_NEEDED tags to
     only be emitted for libraries that satisfy some reference from
     regular objects.  `--no-as-needed' restores the default behaviour.

Note in this case -ltermcap is not needed to satisfy any reference from regular
objects, only from a shared library.

Comment 4 Tom Lane 2005-05-20 14:04:26 UTC
That may be the documented behavior, but doesn't that make the switch
essentially useless?  It certainly isn't useful for building actually executable
programs if it strips out subsidiary libraries.  I'm curious to know what you
think it *is* useful for given this behavior.

Comment 5 Martijn van Oosterhout 2005-10-27 12:10:32 UTC
(In reply to comment #4)
> That may be the documented behavior, but doesn't that make the switch
> essentially useless?  It certainly isn't useful for building actually executable
> programs if it strips out subsidiary libraries.  I'm curious to know what you
> think it *is* useful for given this behavior.

It is working as advertised. Each shared library should have the DT_NEEDED for
other libraries it need. The main program only for the libraries it *directly*
calls. The above program compiles and runs fine on my Debian system. See how the
-ltermcap is removed and ncurses added automatically. Because libreadline
declares it's own dependancy. This looks like a Fedora bug to me.

$ gcc -o test_case test_case.o -Wl,--as-needed -lreadline -ltermcap
$ readelf --all /lib/libreadline.so.5 |grep NEEDED
 0x00000001 (NEEDED)                     Shared library: [libncurses.so.5]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]
$ readelf --all /lib/libreadline.so.4 |grep NEEDED
 0x00000001 (NEEDED)                     Shared library: [libncurses.so.5]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]
$ ldd ./test_case
        linux-gate.so.1 =>  (0xffffe000)
        libreadline.so.5 => /lib/libreadline.so.5 (0x4001e000)
        libc.so.6 => /lib/tls/libc.so.6 (0x4004b000)
        libncurses.so.5 => /lib/libncurses.so.5 (0x40183000)
        /lib/ld-linux.so.2 (0x40000000)
$ ./test_case
hello, world:
$ readelf --all test_case |grep NEEDED
 0x00000001 (NEEDED)                     Shared library: [libreadline.so.5]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]


Comment 6 Martijn van Oosterhout 2005-10-27 13:47:32 UTC
(In reply to comment #5)

Confirmed it's a bug in readline. I downloaded the latest source (5.0) and
compiled it and confirmed that it doesn't use -ltermcap on the linker command
line. If you add -Wl,--no-undefined you get *lots* of undefined symbols. Debian
fixed this back in 2002 by adding SHLIB_LIBS=-lncurses to the make command line.

The effect of this is that --as-needed will fail on any program using readline.
I don't use Redhat, but I downloaded the .src.rpm and Redhat doesn't fix this
particular bug. Should this bug be reassigned to the readline package?

Even if it's fixed now, it will take years for other distributions to pick it
up. One alternative is that if you specify -Wl,--no-undefined, the missing
symbol errors appear at compile time. Hence autoconf while testing can notice
that readline only works with termcap listed on the command line also. It means
you can detect the problem.

Unfortunatly, -Wl,--no-undefined cannot be applied sensibly at the same time as
-Wl,--as-needed. The symbol check is obviously before the DT_NEEDED list is
generated. However, if you specify -Wl,--no-as-needed before just -ltermcap, it
works. Libs before are removed if not needed. Libs after the flag are
unconditionally included.

Note, readline is a really bad example for a library. If it can't find a termcap
library on your system, it includes a dummy header and proceeds to compile and
link even though it couldn't find anything on your system that will allow it to
work. Ugh!