Bug 1498946

Summary:	All LLVM 5.0 unit tests fail
Product:	[Fedora] Fedora	Reporter:	Tilmann Scheller <tschelle>
Component:	llvm	Assignee:	Tom Stellard <tstellar>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	28	CC:	ajax, davejohansen, dmalcolm, ignatenko, jakub, jistone, kyle, petersen, sbergman, scottt.tw, siddharth.kde, tstellar
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-04-05 14:37:52 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Tilmann Scheller 2017-10-05 15:35:12 UTC

Description of problem:
All LLVM 5.0 unit tests fail when libLLVM.so is linked statically to libstdc++

Version-Release number of selected component (if applicable):
llvm-5.0.0-1.fc28

How reproducible:
100%

Steps to Reproduce:
1. Fetch LLVM sources
2. cmake <path to LLVM sources> -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=Debug -DLLVM_BUILD_LLVM_DYLIB:BOOL=ON -DLLVM_LINK_LLVM_DYLIB:BOOL=ON '-DCMAKE_SHARED_LINKER_FLAGS=-Wl,-Bsymbolic -static-libstdc++' -DLLVM_TARGETS_TO_BUILD=X86

3. make check-llvm-unit

Actual results:
All unit tests fail, some Orc unit tests seem to be stuck in an infinite loop

Expected results:
All unit tests should pass

Additional info:
The failures go away after dropping "-static-libstdc++" from the linker flags.
This reproduces both with GCC and Clang and affects release builds as well as debug builds.
Does not trigger on RHEL 7.4.

Upstream commit r300496 introduced this issue (this commit enabled symbol versioning by default).

When libstdc++ is linked statically to libLLVM.so all the symbols that are linked in from libstdc++ have public visibility. The linker version script introduced with the commit above also versions those libstdc++ symbols. So you end up with symbols like this:

...
0000000002f31930 g    DF .text  0000000000000016  LLVM_5.0    std::_V2::error_category::equivalent(std::error_code const
&, int) const
...

This in turn appears to cause the libstdc++ symbols to be resolved differently than when running with libLLVM.so without versioned symbols. Ultimately this results in all kinds of strange side effects causing the tests to fail.

Comment 1 Tom Stellard 2017-10-10 04:24:56 UTC

Static linking of libstdc++ is done in order to support Steam.  Steam and its games ship their own libstdc++ and the dynamic loader will fail to load LLVM, because it is linked against newer versions of the libstdc++ symbols than the libstdc++ bundled with Steam provides. 

Steam will now default to using system libstdc++ if it is newer than its bundled libstdc++, so games should work fine if we dynamically link libstdc++.  However, the steam installer still uses its own bundled versions of libstdc++, so this will still fail if we dynamically link libstdc++.

Comment 2 Fedora End Of Life 2018-02-20 15:37:31 UTC

This bug appears to have been reported against 'rawhide' during the Fedora 28 development cycle.
Changing version to '28'.

Comment 3 Stephan Bergmann 2018-02-27 16:32:58 UTC

Another consequence of that "-Wl,-Bsymbolic -static-libstdc++" is that tools like ASan will report issues in processes that include /usr/lib64/libclangFrontend.so.5 calling into /usr/lib64/libLLVM-5.0.so (e.g., through the use of /usr/lib64/libpocl.so.1).  I ran into that when running the test suite of an upstream LibreOffice built with ASan+UBSan, and happened to have pocl installed (which causes some of the tests to call into it if present).

ASan intercepts both malloc/free and global operator new/delete, but libLLVM-5.0.so prevents interception of calls to global operator new/delete within its code, so those calls will look to ASan as calls to malloc/free instead.  So, e.g., ParseCodeGenArgs in libclangFrontend.so.5 will call the intercepted global operator delete at

>   #0 0x52bfc8 in operator delete(void*) /data/sbergman/github.com/llvm-project/llvm-project-20170507/compiler-rt/lib/asan/asan_new_delete.cc:149
>   #1 0x7fc6ea11ad63  /usr/include/c++/7/ext/new_allocator.h:125
>   #2 0x7fc6ea11ad63  /usr/include/c++/7/bits/alloc_traits.h:462
>   #3 0x7fc6ea11ad63  /usr/include/c++/7/bits/basic_string.h:226
>   #4 0x7fc6ea11ad63  /usr/include/c++/7/bits/basic_string.h:221
>   #5 0x7fc6ea11ad63  /usr/include/c++/7/bits/basic_string.h:647
>   #6 0x7fc6ea11ad63  /usr/include/llvm/ADT/Triple.h:44
>   #7 0x7fc6ea11ad63 in ParseCodeGenArgs /usr/src/debug/clang-5.0.1-3.fc27.x86_64/lib/Frontend/CompilerInvocation.cpp:440
...

on a std::string that libLLVM-5.0.so had allocated through non-intercepted global operator new at

>   #0 0x4f2bd0 in __interceptor_malloc /data/sbergman/github.com/llvm-project/llvm-project-20170507/compiler-rt/lib/asan/asan_malloc_linux.cc:88
>   #1 0x7fc6e5776387 in operator new(unsigned long) (/lib64/libLLVM-5.0.so+0x21c5387)
LLVMSymbolizer: error reading file: No such file or directory
>   #2 0x7ffe06ef85e7  ([stack]+0x145e7)
>   #3 0x7fc6e4d23305  /usr/include/c++/7/bits/basic_string.h:235
>   #4 0x7fc6e4d23305  /usr/include/c++/7/bits/basic_string.h:254
>   #5 0x7fc6e4d23305  /usr/include/c++/7/bits/basic_string.h:510
>   #6 0x7fc6e4d23305 in void std::_Construct<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char const*&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, char const*&) /usr/include/c++/7/bits/stl_construct.h:75
>   #7 0x7fc6e4d23305 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >* std::__uninitialized_copy<false>::__uninit_copy<char const**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>(char const**, char const**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /usr/include/c++/7/bits/stl_uninitialized.h:83
>   #8 0x7fc6e4d23305 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >* std::uninitialized_copy<char const**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>(char const**, char const**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /usr/include/c++/7/bits/stl_uninitialized.h:134
>   #9 0x7fc6e4d23305 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >* std::__uninitialized_copy_a<char const**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const**, char const**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >&) /usr/include/c++/7/bits/stl_uninitialized.h:289
>   #10 0x7fc6e4d23305 in void std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_range_initialize<char const**>(char const**, char const**, std::forward_iterator_tag) /usr/include/c++/7/bits/stl_vector.h:1326
>   #11 0x7fc6e4d23305 in void std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_initialize_dispatch<char const**>(char const**, char const**, std::__false_type) /usr/include/c++/7/bits/stl_vector.h:1299
>   #12 0x7fc6e4d23305 in std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::vector<char const**, void>(char const**, char const**, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > const&) /usr/include/c++/7/bits/stl_vector.h:414
>   #13 0x7fc6e4d23305 in llvm::opt::ArgList::getAllArgValues[abi:cxx11](llvm::opt::OptSpecifier) const /usr/src/debug/llvm-5.0.1-3.fc27.x86_64/lib/Option/ArgList.cpp:95
...

which causes ASan to report a malloc vs operator delete alloc-dealloc-mismatch.

Comment 4 Fedora Update System 2018-04-02 14:21:36 UTC

pocl-1.1-2.fc28 lldb-6.0.0-3.fc28 clang-6.0.0-5.fc28 llvm-6.0.0-11.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-02c4091319

Comment 5 Fedora Update System 2018-04-03 15:57:17 UTC

clang-6.0.0-5.fc28, lldb-6.0.0-3.fc28, llvm-6.0.0-11.fc28, mesa-18.0.0-2.fc28.1, pocl-1.1-2.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-02c4091319

Comment 6 Fedora Update System 2018-04-03 23:25:26 UTC

clang-6.0.0-5.fc28 lldb-6.0.0-3.fc28 llvm-6.0.0-11.fc28 mesa-18.0.0-3.fc28 pocl-1.1-2.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-02c4091319

Comment 7 Fedora Update System 2018-04-04 18:35:11 UTC

clang-6.0.0-5.fc28, lldb-6.0.0-3.fc28, llvm-6.0.0-11.fc28, mesa-18.0.0-3.fc28, pocl-1.1-2.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-02c4091319

Comment 8 Fedora Update System 2018-04-05 11:47:53 UTC

clang-6.0.0-5.fc28, lldb-6.0.0-3.fc28, llvm-6.0.0-11.fc28, mesa-18.0.0-3.fc28, pocl-1.1-2.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.