Bug 663260 - Build time core dump with R14B01 release
Summary: Build time core dump with R14B01 release
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: erlang
Version: rawhide
Hardware: Unspecified
OS: Unspecified
low
medium
Target Milestone: ---
Assignee: Hans Ulrich Niedermann
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 673055
TreeView+ depends on / blocked
 
Reported: 2010-12-15 07:05 UTC by Hans Ulrich Niedermann
Modified: 2011-01-31 11:26 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-01-31 11:26:03 UTC
Type: ---


Attachments (Terms of Use)
R14B01 update in erlang.spec sources .gitignore (excluding patch updates) (3.42 KB, patch)
2010-12-15 07:05 UTC, Hans Ulrich Niedermann
no flags Details | Diff
R14B01: update the patches as well (274.50 KB, patch)
2010-12-15 19:22 UTC, Hans Ulrich Niedermann
no flags Details | Diff
git bisect test script (CAUTION: REMOVES ALL YOUR FILES!!!) (1.04 KB, text/plain)
2011-01-25 23:55 UTC, Hans Ulrich Niedermann
no flags Details

Description Hans Ulrich Niedermann 2010-12-15 07:05:56 UTC
Created attachment 468779 [details]
R14B01 update in erlang.spec sources .gitignore (excluding patch updates)

Description of problem:

  Upstream have released R14B01 on 2010-12-08, which Fedora
  should probably update to in F14 and rawhide.

Version-Release number of selected component (if applicable):

  R14B-0.5.*

Additional info:

  I have already uploaded the R14B01 source files to the lookaside
  mirror.

  I have also locally adapted erlang.spec with the appropriate
  changes to go from R14B-0.5 to R14B-01.1 (see attached patch).

  What remains to be done is porting the patches.

Comment 1 Hans Ulrich Niedermann 2010-12-15 19:22:27 UTC
Created attachment 468943 [details]
R14B01: update the patches as well

I have now updated all the patches to the ones found in Peter Lemenkov's github repo for R14B01.

Unfortunately, a "fedpkg mockbuild" for f14 on f14 fails with a buffer overflow being detected in beam.smp:

*** buffer overflow detected ***: /builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp terminated
======= Backtrace: =========
/lib/libc.so.6(__fortify_fail+0x4d)[0x2b35dd]
/lib/libc.so.6(+0xf45fa)[0x2b15fa]
/lib/libc.so.6(__strcpy_chk+0x3f)[0x2b08af]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp[0x81906cb]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp(erts_write_to_port+0x931)[0x80c2e61]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp[0x8132ee8]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp(process_main+0x67c2)[0x8161592]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp[0x80cf42c]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp[0x81b94e6]
/lib/libpthread.so.0(+0x5f19)[0x115f19]
/lib/libc.so.6(clone+0x5e)[0x299a2e]
======= Memory map: ========
00110000-00128000 r-xp 00000000 fd:06 247277     /lib/libpthread-2.12.90.so
00128000-00129000 r--p 00017000 fd:06 247277     /lib/libpthread-2.12.90.so
00129000-0012a000 rw-p 00018000 fd:06 247277     /lib/libpthread-2.12.90.so
0012a000-0012c000 rw-p 00000000 00:00 0 
001bd000-0034a000 r-xp 00000000 fd:06 247338     /lib/libc-2.12.90.so
0034a000-0034c000 r--p 0018c000 fd:06 247338     /lib/libc-2.12.90.so
0034c000-0034d000 rw-p 0018e000 fd:06 247338     /lib/libc-2.12.90.so
0034d000-00350000 rw-p 00000000 00:00 0 
00366000-00386000 r-xp 00000000 fd:06 247284     /lib/ld-2.12.90.so
00386000-00387000 r--p 0001f000 fd:06 247284     /lib/ld-2.12.90.so
00387000-00388000 rw-p 00020000 fd:06 247284     /lib/ld-2.12.90.so
003df000-003fb000 r-xp 00000000 fd:06 247374     /lib/libgcc_s-4.5.1-20100924.so.1
003fb000-003fc000 rw-p 0001b000 fd:06 247374     /lib/libgcc_s-4.5.1-20100924.so.1
00473000-00490000 r-xp 00000000 fd:06 247344     /lib/libtinfo.so.5.7
00490000-00493000 rw-p 0001c000 fd:06 247344     /lib/libtinfo.so.5.7
00783000-0078b000 r-xp 00000000 fd:06 247384     /lib/librt-2.12.90.so
0078b000-0078c000 r--p 00007000 fd:06 247384     /lib/librt-2.12.90.so
0078c000-0078d000 rw-p 00008000 fd:06 247384     /lib/librt-2.12.90.so
00912000-00926000 r-xp 00000000 fd:06 247486     /lib/libz.so.1.2.5
00926000-00927000 rw-p 00013000 fd:06 247486     /lib/libz.so.1.2.5
009fa000-009fc000 r-xp 00000000 fd:06 247378     /lib/libutil-2.12.90.so
009fc000-009fd000 r--p 00001000 fd:06 247378     /lib/libutil-2.12.90.so
009fd000-009fe000 rw-p 00002000 fd:06 247378     /lib/libutil-2.12.90.so
00c6b000-00c8e000 r-xp 00000000 fd:06 247383     /lib/libncurses.so.5.7
00c8e000-00c8f000 rw-p 00023000 fd:06 247383     /lib/libncurses.so.5.7
00d9c000-00d9f000 r-xp 00000000 fd:06 247303     /lib/libdl-2.12.90.so
00d9f000-00da0000 r--p 00002000 fd:06 247303     /lib/libdl-2.12.90.so
00da0000-00da1000 rw-p 00003000 fd:06 247303     /lib/libdl-2.12.90.so
00e4d000-00e75000 r-xp 00000000 fd:06 247289     /lib/libm-2.12.90.so
00e75000-00e76000 r--p 00027000 fd:06 247289     /lib/libm-2.12.90.so
00e76000-00e77000 rw-p 00028000 fd:06 247289     /lib/libm-2.12.90.so
00fc2000-00fc3000 r-xp 00000000 00:00 0          [vdso]
08048000-08202000 r-xp 00000000 fd:06 2378567    /builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp
08202000-08237000 rw-p 001b9000 fd:06 2378567    /builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp
08237000-08266000 rw-p 00000000 00:00 0 
092df000-09388000 rw-p 00000000 00:00 0          [heap]
b5400000-b5408000 rw-p 00000000 00:00 0 
b5408000-b5500000 ---p 00000000 00:00 0 
b5551000-b5552000 ---p 00000000 00:00 0 
b5552000-b5d52000 rw-p 00000000 00:00 0 
b5d52000-b5d53000 ---p 00000000 00:00 0 
b5d53000-b6553000 rw-p 00000000 00:00 0 
b6553000-b6554000 ---p 00000000 00:00 0 
b6554000-b6558000 rw-p 00000000 00:00 0 
b6558000-b6559000 ---p 00000000 00:00 0 
b6559000-b6d59000 rw-p 00000000 00:00 0 
b6d59000-b6d5a000 ---p 00000000 00:00 0 
b6d5a000-b7843000 rw-p 00000000 00:00 0 
b7843000-b7844000 ---p 00000000 00:00 0 
b7844000-b784c000 rw-p 00000000 00:00 0 
b784c000-b784d000 rw-p 00000000 00:00 0 
bff12000-bff33000 rw-p 00000000 00:00 0          [stack]

Comment 2 Peter Lemenkov 2010-12-17 10:24:14 UTC
Confirmed. I tried tagged snapshot from GitHub with the same result.

Comment 3 Hans Ulrich Niedermann 2011-01-25 23:26:59 UTC
I have just run a "git bisect" to find the issue (a regression between R14B and R14B01). I am confident that the issue is in upstream code, not in the Fedora specific patches, but the exact point where the problem starts is not that easy to determine.

The immediate culprit introducing the core dump appears to be upstream commit b9101fe19b7f8d659d266152b36cc436b90c77a3 ("Make Unicode filenames work on Windows"). However, its parent commit also exhibits some peculiar behaviour to be attributed to the same area of code, so there is more to the problem than just this one commit.

Next things to examine:

  a) Does upstream happen to have fixed the issue somewhere in
     their not-yet-released code (e.g. proposed-updates branch)?

  b) How can we fix the problem in upstream, and in Fedora?

Comment 4 Hans Ulrich Niedermann 2011-01-25 23:55:34 UTC
Created attachment 475305 [details]
git bisect test script (CAUTION: REMOVES ALL YOUR FILES!!!)

CAUTION: REMOVES ALL YOUR FILES!!!

Take a git checkout of the otp source tree (in which ALL FILES WILL BE DELETED), then do something like

  $ git bisect start OTP_R14B01 OTP_R14B
  $ git bisect run ../test-build.sh

to look for the commit introducing the core dump during build.

Both successful builds and builds aborting for any other reason than a core dump will be counted as "good".

Comment 5 Hans Ulrich Niedermann 2011-01-26 00:40:57 UTC
The commits in question (from youngest to oldest):

Build failure  SHA1    Summary

core-dump      b9101fe Make Unicode filenames work on Windows
file-not-found be63e48 Correction of VS2008 redistributables location
file-not-found c5a6741 Allow installer to take redistributables from VC9
file-not-found 63eeba2 Handle binary file names and conversion of unicode strings
no-target-opt  a3ad23a Teach binary:matches not to return 'nomatch' on empty input

Where the three build failures look as follows

   no-target-opt     This is the "normal" case for a "good" build.

        No rule to make target `opt'.  Stop.             

   file-not-found    erlc cannot find its files (indication that something
                     is wrong with the file name handling)

       erlc -W  +debug_info +inline -o../ebin hipe_rtl.erl
       {"init terminating in do_boot",{'cannot get bootfile','start_clean.boot'}}

   core-dump         erlc actually dumps core

       erlc -W  +debug_info +inline -o../ebin hipe_rtl.erl
       make[3]: *** [../ebin/hipe_rtl.beam] Aborted (core dumped)

Comment 6 Hans Ulrich Niedermann 2011-01-30 16:14:00 UTC
Apparently, upstream have a fix in git:

    http://permalink.gmane.org/gmane.comp.lang.erlang.bugs/2208

so at least R14B02 could build again.

Short summary: Our compiling with -D_FORTIFY_SOURCE=2 enables a check
which detects a buffer overflow in an occasion where the (too)
"cleverly" written Erlang source would not actually produce a buffer
overflow.

From my reading of the source code, the mentioned fix in upstream
git at

    https://github.com/erlang/otp/commit/3b7fd1030711ad56fa45d18dc6e0f1d0beee2399

should fix the issue by rewriting the code work without using
undefined C behaviour, and without tricking _FORTIFY_SOURCE checks.

I am going to try and see whether we can get an updated Fedora package
built with that fix.

Comment 7 Peter Lemenkov 2011-01-30 16:42:08 UTC
(In reply to comment #6)
> Apparently, upstream have a fix in git:
> 
>     http://permalink.gmane.org/gmane.comp.lang.erlang.bugs/2208
> 
> so at least R14B02 could build again.
> 
> Short summary: Our compiling with -D_FORTIFY_SOURCE=2 enables a check
> which detects a buffer overflow in an occasion where the (too)
> "cleverly" written Erlang source would not actually produce a buffer
> overflow.
> 
> From my reading of the source code, the mentioned fix in upstream
> git at
> 
>    
> https://github.com/erlang/otp/commit/3b7fd1030711ad56fa45d18dc6e0f1d0beee2399
> 
> should fix the issue by rewriting the code work without using
> undefined C behaviour, and without tricking _FORTIFY_SOURCE checks.
> 
> I am going to try and see whether we can get an updated Fedora package
> built with that fix.

Hans, great work, indeed!

Comment 8 Hans Ulrich Niedermann 2011-01-30 21:43:11 UTC
Above proposed fix does fix the issue it is supposed to fix.

    http://koji.fedoraproject.org/koji/taskinfo?taskID=2750735

Working on the new build failure now (see above build.log, happens during "make release-docs").

Comment 9 Hans Ulrich Niedermann 2011-01-31 11:26:03 UTC
Renaming and closing this bug.

Remaining issues large enough to warrant bug tracking will be connected to bug 673055 in some way.


Note You need to log in before you can comment on or make changes to this bug.