Bug 916616 - clang link-time optimization broken
Summary: clang link-time optimization broken
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: llvm
Version: 23
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Nobody's working on this, feel free to take it
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-02-28 13:37 UTC by Julian Stecklina
Modified: 2016-11-25 19:07 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-25 19:07:24 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
fix paths for lto (2.51 KB, patch)
2015-03-31 15:40 UTC, Richard Henderson
no flags Details | Diff

Description Julian Stecklina 2013-02-28 13:37:22 UTC
Description of problem:

Compiling a program with clang and link-time optimization fails. Linking results in:
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error loading plugin
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error in plugin cleanup (ignored)
clang: error: linker command failed with exit code 1 (use -v to see invocation)


Version-Release number of selected component (if applicable):
clang-3.1-12.fc18.x86_64
llvm-3.1-12.fc18.x86_64
llvm-libs-3.1-12.fc18.x86_64

Steps to Reproduce:
% echo "int main() {}" > foo.c
% clang -flto -o foo foo.c 
  
Actual results:

clang aborts with:
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error loading plugin
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error in plugin cleanup (ignored)
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Expected results:

clang links the binary.

Additional info:

% clang -v -flto -o foo foo.c 
clang version 3.1 (branches/release_31)
Target: x86_64-redhat-linux-gnu
Thread model: posix
 "/usr/bin/clang" -cc1 -triple x86_64-redhat-linux-gnu -emit-llvm-bc -disable-free -disable-llvm-verifier -main-file-name foo.c -mrelocation-model static -mdisable-fp-elim -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -target-linker-version 2.23.51.0.1 -momit-leaf-frame-pointer -v -resource-dir /usr/bin/../lib/clang/3.1 -fmodule-cache-path /var/tmp/clang-module-cache -internal-isystem /usr/local/include -internal-isystem /usr/bin/../lib/clang/3.1/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 80 -mstackrealign -fgnu-runtime -fobjc-runtime-has-arc -fobjc-runtime-has-weak -fobjc-fragile-abi -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/foo-V20Zbd.o -x c foo.c
clang -cc1 version 3.1 based upon LLVM 3.1 default target x86_64-redhat-linux-gnu
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/bin/../lib/clang/3.1/include
 /usr/include
End of search list.
 "/usr/bin/ld" --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o foo /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/crtbegin.o -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2 -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../.. -L/lib -L/usr/lib -plugin /usr/bin/../lib/LLVMgold.so /tmp/foo-V20Zbd.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-redhat-linux/4.7.2/crtend.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crtn.o
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error loading plugin
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error in plugin cleanup (ignored)
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Comment 1 Jay Finger 2013-04-27 15:36:12 UTC
I have stumbled across this as well.  If you look in the output above, you will notice that the ld invocation includes the args "-plugin /usr/bin/../lib/LLVMgold.so".  This is the problem.  LLVMgold.so is in found at /usr/lib64/llvm/LLVMgold.so.  

Link time optimization (i.e. "-flto" or "-O4") is broken without this.

This is running F18 which has been yum updated as of 27-Apr-2013. 

$ yum list --noplugins clang* llvm*
Installed Packages
clang.x86_64                              3.2-2.fc18                    @updates
clang-devel.x86_64                        3.2-2.fc18                    @updates
clang-doc.noarch                          3.2-2.fc18                    @updates
llvm.x86_64                               3.2-2.fc18                    @updates
llvm-libs.x86_64                          3.2-2.fc18                    @updates

Comment 2 Jay Finger 2013-04-27 15:47:47 UTC
Note that there is probably more to it than fixing the path it is using to find LLVMgold.so.  When I create a symlink from /usr/lib/LLVMgold.so to /usr/lib64/llvm/LLVMgold.so, the linker complains of "file format not recognized".  So even with the plugin it is failing to recognize the llvm bitcode file.  

$ cat hello.c
#include <stdio.h>
int
main(
    int argc,
    char **argv)
{
    printf("hello world\n");
    return 0;
}

$ clang -O4 -c hello.c -o hello.o

$ file hello.o
hello.o: LLVM bitcode

$ clang -O4 hello.o -o hello
hello.o: file not recognized: File format not recognized
clang: error: linker command failed with exit code 1 (use -v to see invocation)

$ clang -O4 hello.o -o hello -v
clang version 3.2 (tags/RELEASE_32/final)
Target: x86_64-redhat-linux-gnu
Thread model: posix
 "/usr/bin/ld" --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o hello /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/crtbegin.o -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2 -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../.. -L/lib -L/usr/lib -plugin /usr/bin/../lib/LLVMgold.so hello.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-redhat-linux/4.7.2/crtend.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crtn.o
hello.o: file not recognized: File format not recognized
clang: error: linker command failed with exit code 1 (use -v to see invocation)
$

Comment 3 Julian Stecklina 2013-06-28 09:51:13 UTC
This still applies to F19 and clang-3.3-0.6.rc3.fc19.x86_64.

Comment 4 Benny 2014-04-22 09:14:45 UTC
This still broken in F20 for the package clang-3.4-6.fc20.x86_64

An additional link: https://stackoverflow.com/questions/16290952/clang-link-time-optimization-doesnt-work-properly-on-fedora-18

Comment 5 Mike Farnsworth 2014-09-24 05:42:43 UTC
I can get this to work with a symlink from /usr/lib/LLVMgold.so to /usr/lib64/llvm/LLVMgold.so and also setting ld.gold as the linker via "alternatives --config ld".

However, due to https://bugzilla.redhat.com/show_bug.cgi?id=1112265 and https://bugzilla.redhat.com/show_bug.cgi?id=1101144 I am unable to use clang LTO with LTO applied to static libs I'm linking in even with these workarounds.

Comment 6 Adam Jackson 2014-11-19 16:28:29 UTC
Reassign to nobody@.  I have no time and no interest in working on clang, and the only reason it's even in the llvm package is because it can't be built any other way.

If anyone wishes to volunteer for comaintainer with a focus on clang, please let me know (preferably with direct email, if it's in bz I'll likely miss the request), I'll be happy to hook you up.

Comment 7 Jan Včelák 2014-12-26 00:30:50 UTC
Still valid in F21. There is a lot of hardcoded libdir paths in the llvm/clang sources. I'm affraid this won't be easy to fix and it might need some cooperation with upstream.

Comment 8 Richard Henderson 2015-03-31 15:40:41 UTC
Created attachment 1009120 [details]
fix paths for lto

To fix the bug, we need to tell the driver where we moved
the LLVMgold.so plugin, and we also need to force the use
of the gold linker over the bfd linker.

Comment 9 Mike Farnsworth 2015-03-31 16:59:30 UTC
Awesome.  Does this mean we no longer need to run "alternatives --config ld" when using clang and LTO, since it should pick it up automatically?

Comment 10 Jan Kurik 2015-07-15 14:50:47 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle.
Changing version to '23'.

(As we did not run this process for some time, it could affect also pre-Fedora 23 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23

Comment 11 Fedora End Of Life 2016-11-24 10:57:10 UTC
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 12 Ben Liblit 2016-11-25 19:02:21 UTC
Works for me under Fedora 25 with clang-3.8.0-2.fc25.x86_64, llvm-3.8.0-1.fc25.x86_64, and llvm-libs-3.8.0-1.fc25.x86_64.  Following the original reproduction instructions:

    % echo "int main() {}" > foo.c
    % clang -flto -o foo foo.c

No errors, just a working "foo" executable.


Note You need to log in before you can comment on or make changes to this bug.