Bug 187795 - ld very slow with DWARF-2
Summary: ld very slow with DWARF-2
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: binutils
Version: 6
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Jakub Jelinek
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2006-04-03 17:05 UTC by Stefan Ring
Modified: 2007-11-30 22:11 UTC (History)
1 user (show)

Fixed In Version: 2.17.50.0.6-5.fc6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2007-07-02 15:24:12 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
common files needed for testcase (572.90 KB, application/octet-stream)
2006-04-05 09:53 UTC, Stefan Ring
no flags Details
all the binary stuff compiled with -ggdb (dwarf2) (11.52 MB, application/octet-stream)
2006-04-05 09:58 UTC, Stefan Ring
no flags Details
all the binary stuff compiled with -gstabs+ (11.49 MB, application/octet-stream)
2006-04-05 10:09 UTC, Stefan Ring
no flags Details
common files needed for testcase (updated) (1.65 MB, application/octet-stream)
2006-04-05 10:12 UTC, Stefan Ring
no flags Details

Description Stefan Ring 2006-04-03 17:05:21 UTC
Description of problem:
Linking the same program built with -ggdb takes a long time whereas it's done
within 3 seconds when built with -gstabs+.


Version-Release number of selected component (if applicable):
GNU ld version 2.16.91.0.6 20060212


How reproducible:
(Unsure but I suspect it to be a problem with every moderately sized C++ program)


Steps to Reproduce:
1. Build any not too small C++ program with -g (or -ggdb)
2.
3.
  
Actual results:
time g++ <long gcommand line>
real    1m36.551s
user    1m23.441s
sys     0m13.109s


Expected results:

Something close to this (which is the time needed for the -gstabs+ build):
real    0m3.013s
user    0m2.484s
sys     0m0.528s


Additional info:
I wanted to try if -feliminate-dwarf2-dups would help, but this was prohibited
by https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=187787

The build consists of 84 .o files and about 10 shared libraries (all stripped).
Some of the .o files are packed together in static libraries (with debug info)
but that's only a minority.

It's also slow on i386, IIRC.

Comment 1 Jakub Jelinek 2006-04-04 09:57:30 UTC
You really need to provide a testcase, with all the *.o and *.a files you are
linking together etc., tar and bzip2 it and attach here.
Dwarf2 linking is of course expected to be slower, the debug format is much more
expressive, contains more information and ld needs to do merging of duplicate
strings.  But it shouldn't and in my experience is not that much slower.
Two years ago or so there was a problem with the string merging algorithm, but
it has been fixed for a long time.

Comment 2 Stefan Ring 2006-04-05 09:53:45 UTC
Created attachment 127335 [details]
common files needed for testcase

more files and description coming up. stay tuned...

Comment 3 Stefan Ring 2006-04-05 09:58:30 UTC
Created attachment 127338 [details]
all the binary stuff compiled with -ggdb (dwarf2)

Comment 4 Stefan Ring 2006-04-05 10:09:30 UTC
Created attachment 127339 [details]
all the binary stuff compiled with -gstabs+

Comment 5 Stefan Ring 2006-04-05 10:12:33 UTC
Created attachment 127340 [details]
common files needed for testcase (updated)

sorry, i forgot a library

Comment 6 Stefan Ring 2006-04-05 10:16:31 UTC
Ok, that should do it. To link this stuff, you need to do the following:

(in a blank directory extract bug187795-common.tar.bz2 and
bug187795-obj-dwarf2.tar.bz2)
time sh ./linkit
(takes about 3 minutes)

The same thing with stabs+:

(in a blank directory extract bug187795-common.tar.bz2 and
bug187795-obj-stabs.tar.bz2)
time sh ./linkit
(takes about 3 seconds)

Apart from the common development packages, the following are needed for this
(all contained in either fc5 core or extras):

boost-devel.x86_64
mysqlclient14-devel.x86_64
expat-devel.x86_64
python-devel.x86_64
libsigc++20-devel.x86_64
glib2-devel.x86_64


Comment 7 Stefan Ring 2006-04-05 17:07:02 UTC
After some annoyances, I've managed to build everything with gcc 3.4 on FC5, and
now it's not only just as fast as with stabs+ but also it actually works (see
bug187789).

Comment 8 Jan Kratochvil 2006-09-14 09:22:36 UTC
According to gprof(1) the primary problem is the overhead of bfd_getl64()&co.
They should be compile-time endianity optimized and also inlined.
glib GUINT64_FROM_LE()&co. could be the appropriate code to copy.


Comment 9 Stefan Ring 2006-11-14 13:10:30 UTC
Seems ok with FC6 - linking is very fast actually, even faster than with stabs
on FC5.

Comment 10 Stefan Ring 2007-01-26 10:20:55 UTC
After the latest yum update on FC6 (x86_64), linking went back to a crawl. I
cannot investigate a great deal at the moment but maybe you have an idea.

When I build my application with gcc-4.1.1-30 (the base version for FC6), it
links in about 3 seconds. Now with 4.1.1-51.fc6, it takes about 5 minutes again.
The binutils version doesn't matter. It's definitely caused by some change in gcc.


Note You need to log in before you can comment on or make changes to this bug.