Bug 1174065 - ar with lto plugin segfaults while creating 32-bit archive on 64-bit platform, unless "--target elf32-i386" is specified
Summary: ar with lto plugin segfaults while creating 32-bit archive on 64-bit platform...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: binutils
Version: 23
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Nick Clifton
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-12-15 03:31 UTC by Hin-Tak Leung
Modified: 2017-01-04 16:28 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-12-20 13:04:57 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
tgz of *.o (332.99 KB, application/octet-stream)
2014-12-16 14:36 UTC, Hin-Tak Leung
no flags Details

Description Hin-Tak Leung 2014-12-15 03:31:55 UTC
Description of problem:

I build R head with lto from time to time. It used to work with gcc 4.8, but after upgrading to f21 with gcc 4.9, 
apparently one needs to use gcc-ar to creates some of the intemediate archives. 
(see https://gcc.gnu.org/gcc-4.9/changes.html)

so I did:
./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages --with-valgrind-instrumentation=2 --enable-lto AR=gcc-ar

It segfaulted in R/src/extra/tre with:

gcc-ar cr libtre.a regcomp.o regerror.o regexec.o tre-ast.o tre-compile.o tre-match-approx.o tre-match-backtrack.o tre-match-parallel.o tre-mem.o tre-parse.o tre-stack.o xmalloc.o
/usr/bin/ar terminated with signal 11 [Segmentation fault], core dumped
Makefile:52: recipe for target 'libtre.a' failed

Version-Release number of selected component (if applicable):
binutils-2.24-21.fc21.x86_64
gcc-4.9.2-1.fc21.x86_64

How reproducible:
always

Steps to Reproduce:
1. download R (http://r-project.org), probably any recent version would do.
2. run ./configure as above, 
3. run make

Actual results:
ar segfaulted.

Expected results:
R to finish building like it did in f20 (just without the last AR=gcc-ar).

Additional info:
I set ulimit -c, and got this back trace:
(gdb) bt
#0  0x0000003e594bc50f in add_symbols () from /lib64/libbfd-2.24.so
#1  0x00007fe0f30b575e in ?? () from /usr/libexec/gcc/x86_64-redhat-linux/4.9.2/liblto_plugin.so
#2  0x0000003e594bcb07 in bfd_plugin_object_p () from /lib64/libbfd-2.24.so
#3  0x0000003e5944762f in bfd_check_format_matches () from /lib64/libbfd-2.24.so
#4  0x0000003e5943f852 in _bfd_write_archive_contents () from /lib64/libbfd-2.24.so
#5  0x0000003e59448f7f in bfd_close () from /lib64/libbfd-2.24.so
#6  0x0000000000404fa7 in write_archive ()
#7  0x00000000004055cc in replace_members ()
#8  0x0000000000402a9f in main ()
(gdb) quit

Comment 1 Hin-Tak Leung 2014-12-16 11:27:23 UTC
FWIW, I am on binutils-2.24-21.fc21.x86_64 . This is probably the same bug as https://bugzilla.redhat.com/show_bug.cgi?id=1149660 and apparently I need binutils-2.24-29.fc21+

Comment 2 Hin-Tak Leung 2014-12-16 14:32:31 UTC
upgrading to binutils-2.24-30.fc21 from koji does allow 64-bit to build. However, it still crashes with 32-bit build on 64-bit host.

Comment 3 Hin-Tak Leung 2014-12-16 14:36:09 UTC
Created attachment 969589 [details]
tgz of *.o

32-bit build still core dump at the same place. Here is the *.o's in a tgz file.

Comment 4 Hin-Tak Leung 2014-12-18 05:46:13 UTC
I got hold of the corresponding debuginfo from koji, and got the dependent debuginfo also; here it the backtrace:


Core was generated by `/bin/ar --plugin /usr/libexec/gcc/x86_64-redhat-linux/4.9.2/liblto_plugin.so -c'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  pe_bfd_object_p (abfd=abfd@entry=0x24f4efd0) at peicode.h:1274
1274	  if (bfd_seek (abfd, (file_ptr) 0, SEEK_SET) != 0
(gdb) bt
#0  pe_bfd_object_p (abfd=abfd@entry=0x24f4efd0) at peicode.h:1274
#1  0x0000003cc7ea860f in pe_bfd_object_p (abfd=0x24f4efd0) at peicode.h:1271
#2  0x0000003cc7e47100 in bfd_check_format_matches (abfd=0x24f4efd0, format=<optimized out>, matching=0x0) at format.c:336
#3  0x0000003cc7ebc9be in bfd_plugin_get_symbols_in_object_only (abfd=0x24f3ea20) at plugin.c:157
#4  add_symbols (handle=0x24f3ea20, nsyms=12, syms=<optimized out>) at plugin.c:279
#5  0x00007efef184575e in claim_file_handler (file=0x7ffeff7fef30, claimed=0x7ffeff7fef2c) at ../../lto-plugin/lto-plugin.c:929
#6  0x0000003cc7ebcf27 in bfd_plugin_object_p (abfd=0x24f3ea20) at plugin.c:450
#7  0x0000003cc7e47100 in bfd_check_format_matches (abfd=0x24f3ea20, format=<optimized out>, matching=0x0) at format.c:336
#8  0x0000003cc7ebc9be in bfd_plugin_get_symbols_in_object_only (abfd=0x24f2e470) at plugin.c:157
....

then it went on forever.

This is with the lattest binutils from koji.

Comment 5 Hin-Tak Leung 2014-12-19 23:55:57 UTC
The stack is simply trashed according to valgrind.

$ valgrind /bin/ar --plugin /usr/libexec/gcc/x86_64-redhat-linux/4.9.2/liblto_plugin.so -cr out.a regerror.o
==12810== Memcheck, a memory error detector
==12810== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==12810== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==12810== Command: /bin/ar --plugin /usr/libexec/gcc/x86_64-redhat-linux/4.9.2/liblto_plugin.so -cr out.a regerror.o
==12810== 
==12810== Stack overflow in thread 1: can't grow stack to 0xffe801e88
==12810== 
==12810== Process terminating with default action of signal 11 (SIGSEGV)
==12810==  Access not within mapped region at address 0xFFE801E88
==12810==    at 0x3CC7EA83B7: pe_bfd_object_p.part.15 (peicode.h:1274)
==12810==  If you believe this happened as a result of a stack
==12810==  overflow in your program's main thread (unlikely but
==12810==  possible), you can try to increase the size of the
==12810==  main thread stack using the --main-stacksize= flag.
==12810==  The main thread stack size used in this run was 8388608.
==12810== Stack overflow in thread 1: can't grow stack to 0xffe801e80
==12810== 
==12810== Process terminating with default action of signal 11 (SIGSEGV)
==12810==  Access not within mapped region at address 0xFFE801E80
==12810==    at 0x48016C0: _vgnU_freeres (in /usr/lib64/valgrind/vgpreload_core-amd64-linux.so)
==12810==  If you believe this happened as a result of a stack
==12810==  overflow in your program's main thread (unlikely but
==12810==  possible), you can try to increase the size of the
==12810==  main thread stack using the --main-stacksize= flag.
==12810==  The main thread stack size used in this run was 8388608.
==12810== 
==12810== HEAP SUMMARY:
==12810==     in use at exit: 573,417,226 bytes in 174,899 blocks
==12810==   total heap usage: 211,099 allocs, 36,200 frees, 588,912,466 bytes allocated
==12810== 
==12810== LEAK SUMMARY:
==12810==    definitely lost: 0 bytes in 0 blocks
==12810==    indirectly lost: 0 bytes in 0 blocks
==12810==      possibly lost: 0 bytes in 0 blocks
==12810==    still reachable: 573,417,226 bytes in 174,899 blocks
==12810==         suppressed: 0 bytes in 0 blocks
==12810== Rerun with --leak-check=full to see details of leaked memory
==12810== 
==12810== For counts of detected and suppressed errors, rerun with: -v
==12810== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Segmentation fault (core dumped)

Comment 6 Hin-Tak Leung 2014-12-20 02:00:19 UTC
A simplier command line which segfault is: gcc-nm regerror.o

"readelf -SW  regerror.o" does work, so I assume it is binutils being broken.

Comment 7 Hin-Tak Leung 2014-12-20 16:05:39 UTC
https://kojipkgs.fedoraproject.org//packages/binutils/2.24/30.fc21/data/logs/x86_64/build.log

		=== ld Summary ===
# of expected passes		1043
# of unexpected failures	3
# of expected failures		57
# of untested testcases		1

FAIL: ld-plugin/lto-3r
FAIL: ld-plugin/lto-5r
FAIL: LTO 11

I think those shouldn't be there.

Comment 8 Hin-Tak Leung 2014-12-21 03:02:31 UTC
See
https://bugzilla.redhat.com/show_bug.cgi?id=1149660#c18

gcc-nm/gcc-ar needs to have GNUTARGET=elf32-i386 set in the environment (or have "--target elf32-i386" specified) when building 32-bit archives on 64-bit platform, otherwise it would segfault. Still I would rather it would do that automatically, but at least this is a workaround...

Comment 9 Hin-Tak Leung 2014-12-24 22:01:04 UTC
I built binutils 2.25 from git, and it seems that the segfault is either fixed in 2.25, or redhat-specific. I guess we'll find out which, when a binutils 2.25 with all the redhat patches are available.

commit 15a2e5b3fab5543c499a58783d9562c0c973fc1f
Author: Tristan Gingold <tristan.gingold>
Date:   Tue Dec 23 16:48:31 2014 +0100

    Bump to version 2.25.0

Comment 10 Hin-Tak Leung 2014-12-24 22:06:29 UTC
My guess is it is redhat specific, but we'll wait and revisit this at some point...

Comment 11 H.J. Lu 2014-12-24 22:23:14 UTC
Please try hjl/lto-mixed branch at

https://sourceware.org/git/?p=binutils-gdb.git;a=summary

Comment 12 Hin-Tak Leung 2014-12-24 23:05:13 UTC
The crash is redhat-specific. commit 15a2e5b3fab5543c499a58783d9562c0c973fc1f (Bump to version 2.25.0) does not crash, 

~/rpmbuild/BUILD/binutils-2.25/binutils/.libs/lt-nm-new --plugin ...

from unpacking the binutils-2.25-1.fc22 src rpm then rpmbuild -bc, crashes.

(In reply to H.J. Lu from comment #11)
> Please try hjl/lto-mixed branch at
> 
> https://sourceware.org/git/?p=binutils-gdb.git;a=summary

Argh, that's where your stuff are - I was git remote'ing 

git://git.kernel.org/pub/scm/linux/kernel/git/hjl/binutils.git

and most of the branches to do with lto haven't been updated for a while.

since stock 2.25 does not, and redhat's 2.25 does, assuming your branch fixes this, it is still up to redhat to take up a few selected patches, though?

Comment 13 Hin-Tak Leung 2014-12-24 23:57:52 UTC
(In reply to H.J. Lu from comment #11)
> Please try hjl/lto-mixed branch at
> 
> https://sourceware.org/git/?p=binutils-gdb.git;a=summary

okay, nm from that works without crashing. Did

tmp-git/binutils/binutils/nm-new --plugin /usr/libexec/gcc/x86_64-redhat-linux/4.9.2/liblto_plugin.so regerror.o

tmp-git/binutils/binutils/ar cr --plugin /usr/libexec/gcc/x86_64-redhat-linux/4.9.2/liblto_plugin.so m *.o



$ git describe --tags
readline_4_0-82254-g04547b7

# ...tmp-git/binutils/binutils/nm-new --version
GNU nm (GNU Binutils) 2.25.51.20141224
Copyright (C) 2014 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.



I guess it is up to redhat's binutils maintainner to update the lto related redhat- extra patches, and/or dropping them.

Comment 14 H.J. Lu 2014-12-25 01:05:15 UTC
(In reply to Hin-Tak Leung from comment #13)
> (In reply to H.J. Lu from comment #11)
> > Please try hjl/lto-mixed branch at
> > 
> > https://sourceware.org/git/?p=binutils-gdb.git;a=summary
> 
> I guess it is up to redhat's binutils maintainner to update the lto related
> redhat- extra patches, and/or dropping them.

That is correct.

Comment 15 Hin-Tak Leung 2014-12-25 02:09:11 UTC
master isn't 2.25, but master differs from hjl/lto-mixed by only 3 commits,

0001-Add-lto-and-none-lto-input-support-for-ld-r.patch
0002-Add-test-for-nm-on-mixed-LTO-non-LTO-object.patch
0003-Don-t-check-the-plugin-target-twice.patch

The first one seems to be largely the same as redhat's
binutils-2.23.2-kernel-ld-r.patch , but not the same.

The 2nd one just add a test to the testsuite so probably not important; the 3rd looks promising and close to where I was single-step'ing in gdb, so I tried grafting the 3rd onto redhat's binutils-2.24-30 . The result is non-functional, with

 undefined symbol: bfd_target_vector

I think I have spent enough time on this, and the answer is somewhere between the redhat patches and hjl/lto-mixed, so I'll let the redhat people carry on...

Just thought I should mention that in the redhat patch set, I noticed there is a 
binutils-2.23.2-aarch64-em.patch
which supposedly fix problem with the ld-r patch?

"
Fix building of aarch64 targets after applying the patch for kernel ld -r modules.
Fix building when "--with debug" is specified.
"


H.J. Lu: binutils-2.23.2-aarch64-em.patch may need to be merged to hjl/lto-mixed ?
The redhat patches are at: (they are individual named patches, and changes of those, not a full fork of upsream)
http://pkgs.fedoraproject.org/cgit/binutils.git

Comment 16 H.J. Lu 2014-12-25 14:28:28 UTC
(In reply to Hin-Tak Leung from comment #15)
> 
> H.J. Lu: binutils-2.23.2-aarch64-em.patch may need to be merged to
> hjl/lto-mixed ?

I have checked a slightly different fix into hjl/lto-mixed a while back.

Comment 17 H.J. Lu 2014-12-25 14:32:27 UTC
(In reply to Hin-Tak Leung from comment #15)
> master isn't 2.25, but master differs from hjl/lto-mixed by only 3 commits,
> 
> 0001-Add-lto-and-none-lto-input-support-for-ld-r.patch
> 0002-Add-test-for-nm-on-mixed-LTO-non-LTO-object.patch
> 0003-Don-t-check-the-plugin-target-twice.patch
> 
> The first one seems to be largely the same as redhat's
> binutils-2.23.2-kernel-ld-r.patch , but not the same.
> 

binutils-2.23.2-kernel-ld-r.patch lacks of many bug fixes I made
over the years.  You can check ChangeLog.lto-mixed to see those
bug fixes.

Comment 18 Fedora End Of Life 2015-11-04 14:33:17 UTC
This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 19 Hin-Tak Leung 2015-11-04 15:17:00 UTC
Still a problem, I think. saw it recently on f23, I believe.

Comment 20 Nick Clifton 2015-11-05 17:30:58 UTC
Hi Hin-Tak,

  Please try out  binutils-2.25.1-9.fc24.

  (I tried importing H.J.'s patch but it was too complex, so I came up with a much simpler patch of my own).

Cheers
  Nick

Comment 21 Fedora End Of Life 2016-11-24 11:20:04 UTC
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 22 Fedora End Of Life 2016-12-20 13:04:57 UTC
Fedora 23 changed to end-of-life (EOL) status on 2016-12-20. Fedora 23 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 23 Hin-Tak Leung 2017-01-04 16:28:49 UTC
Apologies - I haven't been doing any R things for 16 months. Checked and glad to see that GNUTARGET=elf32-i386 is no longer needed.

# rpm -q binutils
binutils-2.26.1-1.fc25.x86_64

parent directory from latest lto build, current from the old tarball ( https://bugzilla.redhat.com/attachment.cgi?id=969589 ). all seems to work.

# gcc-ar cr libtre2.a ../*.o
# gcc-ar cr libtre2.a *.o

root@localhost a]# gcc-nm regerror.o
00000000 T tre_regerror
[root@localhost a]# gcc-nm ../regerror.o
00000000 T tre_regerror
[root@localhost a]# file regerror.o
regerror.o: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped
[root@localhost a]# file ../regerror.o
../regerror.o: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped


# rpm -q binutils
binutils-2.26.1-1.fc25.x86_64


Note You need to log in before you can comment on or make changes to this bug.