Bug 663260

Created attachment 468943 [details]
R14B01: update the patches as well

I have now updated all the patches to the ones found in Peter Lemenkov's github repo for R14B01.

Unfortunately, a "fedpkg mockbuild" for f14 on f14 fails with a buffer overflow being detected in beam.smp:

*** buffer overflow detected ***: /builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp terminated
======= Backtrace: =========
/lib/libc.so.6(__fortify_fail+0x4d)[0x2b35dd]
/lib/libc.so.6(+0xf45fa)[0x2b15fa]
/lib/libc.so.6(__strcpy_chk+0x3f)[0x2b08af]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp[0x81906cb]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp(erts_write_to_port+0x931)[0x80c2e61]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp[0x8132ee8]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp(process_main+0x67c2)[0x8161592]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp[0x80cf42c]
/builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp[0x81b94e6]
/lib/libpthread.so.0(+0x5f19)[0x115f19]
/lib/libc.so.6(clone+0x5e)[0x299a2e]
======= Memory map: ========
00110000-00128000 r-xp 00000000 fd:06 247277     /lib/libpthread-2.12.90.so
00128000-00129000 r--p 00017000 fd:06 247277     /lib/libpthread-2.12.90.so
00129000-0012a000 rw-p 00018000 fd:06 247277     /lib/libpthread-2.12.90.so
0012a000-0012c000 rw-p 00000000 00:00 0 
001bd000-0034a000 r-xp 00000000 fd:06 247338     /lib/libc-2.12.90.so
0034a000-0034c000 r--p 0018c000 fd:06 247338     /lib/libc-2.12.90.so
0034c000-0034d000 rw-p 0018e000 fd:06 247338     /lib/libc-2.12.90.so
0034d000-00350000 rw-p 00000000 00:00 0 
00366000-00386000 r-xp 00000000 fd:06 247284     /lib/ld-2.12.90.so
00386000-00387000 r--p 0001f000 fd:06 247284     /lib/ld-2.12.90.so
00387000-00388000 rw-p 00020000 fd:06 247284     /lib/ld-2.12.90.so
003df000-003fb000 r-xp 00000000 fd:06 247374     /lib/libgcc_s-4.5.1-20100924.so.1
003fb000-003fc000 rw-p 0001b000 fd:06 247374     /lib/libgcc_s-4.5.1-20100924.so.1
00473000-00490000 r-xp 00000000 fd:06 247344     /lib/libtinfo.so.5.7
00490000-00493000 rw-p 0001c000 fd:06 247344     /lib/libtinfo.so.5.7
00783000-0078b000 r-xp 00000000 fd:06 247384     /lib/librt-2.12.90.so
0078b000-0078c000 r--p 00007000 fd:06 247384     /lib/librt-2.12.90.so
0078c000-0078d000 rw-p 00008000 fd:06 247384     /lib/librt-2.12.90.so
00912000-00926000 r-xp 00000000 fd:06 247486     /lib/libz.so.1.2.5
00926000-00927000 rw-p 00013000 fd:06 247486     /lib/libz.so.1.2.5
009fa000-009fc000 r-xp 00000000 fd:06 247378     /lib/libutil-2.12.90.so
009fc000-009fd000 r--p 00001000 fd:06 247378     /lib/libutil-2.12.90.so
009fd000-009fe000 rw-p 00002000 fd:06 247378     /lib/libutil-2.12.90.so
00c6b000-00c8e000 r-xp 00000000 fd:06 247383     /lib/libncurses.so.5.7
00c8e000-00c8f000 rw-p 00023000 fd:06 247383     /lib/libncurses.so.5.7
00d9c000-00d9f000 r-xp 00000000 fd:06 247303     /lib/libdl-2.12.90.so
00d9f000-00da0000 r--p 00002000 fd:06 247303     /lib/libdl-2.12.90.so
00da0000-00da1000 rw-p 00003000 fd:06 247303     /lib/libdl-2.12.90.so
00e4d000-00e75000 r-xp 00000000 fd:06 247289     /lib/libm-2.12.90.so
00e75000-00e76000 r--p 00027000 fd:06 247289     /lib/libm-2.12.90.so
00e76000-00e77000 rw-p 00028000 fd:06 247289     /lib/libm-2.12.90.so
00fc2000-00fc3000 r-xp 00000000 00:00 0          [vdso]
08048000-08202000 r-xp 00000000 fd:06 2378567    /builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp
08202000-08237000 rw-p 001b9000 fd:06 2378567    /builddir/build/BUILD/otp_src_R14B01/bin/i686-pc-linux-gnu/beam.smp
08237000-08266000 rw-p 00000000 00:00 0 
092df000-09388000 rw-p 00000000 00:00 0          [heap]
b5400000-b5408000 rw-p 00000000 00:00 0 
b5408000-b5500000 ---p 00000000 00:00 0 
b5551000-b5552000 ---p 00000000 00:00 0 
b5552000-b5d52000 rw-p 00000000 00:00 0 
b5d52000-b5d53000 ---p 00000000 00:00 0 
b5d53000-b6553000 rw-p 00000000 00:00 0 
b6553000-b6554000 ---p 00000000 00:00 0 
b6554000-b6558000 rw-p 00000000 00:00 0 
b6558000-b6559000 ---p 00000000 00:00 0 
b6559000-b6d59000 rw-p 00000000 00:00 0 
b6d59000-b6d5a000 ---p 00000000 00:00 0 
b6d5a000-b7843000 rw-p 00000000 00:00 0 
b7843000-b7844000 ---p 00000000 00:00 0 
b7844000-b784c000 rw-p 00000000 00:00 0 
b784c000-b784d000 rw-p 00000000 00:00 0 
bff12000-bff33000 rw-p 00000000 00:00 0          [stack]

Confirmed. I tried tagged snapshot from GitHub with the same result.

I have just run a "git bisect" to find the issue (a regression between R14B and R14B01). I am confident that the issue is in upstream code, not in the Fedora specific patches, but the exact point where the problem starts is not that easy to determine.

The immediate culprit introducing the core dump appears to be upstream commit b9101fe19b7f8d659d266152b36cc436b90c77a3 ("Make Unicode filenames work on Windows"). However, its parent commit also exhibits some peculiar behaviour to be attributed to the same area of code, so there is more to the problem than just this one commit.

Next things to examine:

  a) Does upstream happen to have fixed the issue somewhere in
     their not-yet-released code (e.g. proposed-updates branch)?

  b) How can we fix the problem in upstream, and in Fedora?

Created attachment 475305 [details]
git bisect test script (CAUTION: REMOVES ALL YOUR FILES!!!)

CAUTION: REMOVES ALL YOUR FILES!!!

Take a git checkout of the otp source tree (in which ALL FILES WILL BE DELETED), then do something like

  $ git bisect start OTP_R14B01 OTP_R14B
  $ git bisect run ../test-build.sh

to look for the commit introducing the core dump during build.

Both successful builds and builds aborting for any other reason than a core dump will be counted as "good".

The commits in question (from youngest to oldest):

Build failure  SHA1    Summary

core-dump      b9101fe Make Unicode filenames work on Windows
file-not-found be63e48 Correction of VS2008 redistributables location
file-not-found c5a6741 Allow installer to take redistributables from VC9
file-not-found 63eeba2 Handle binary file names and conversion of unicode strings
no-target-opt  a3ad23a Teach binary:matches not to return 'nomatch' on empty input

Where the three build failures look as follows

   no-target-opt     This is the "normal" case for a "good" build.

        No rule to make target `opt'.  Stop.             

   file-not-found    erlc cannot find its files (indication that something
                     is wrong with the file name handling)

       erlc -W  +debug_info +inline -o../ebin hipe_rtl.erl
       {"init terminating in do_boot",{'cannot get bootfile','start_clean.boot'}}

   core-dump         erlc actually dumps core

       erlc -W  +debug_info +inline -o../ebin hipe_rtl.erl
       make[3]: *** [../ebin/hipe_rtl.beam] Aborted (core dumped)

Apparently, upstream have a fix in git:

    http://permalink.gmane.org/gmane.comp.lang.erlang.bugs/2208

so at least R14B02 could build again.

Short summary: Our compiling with -D_FORTIFY_SOURCE=2 enables a check
which detects a buffer overflow in an occasion where the (too)
"cleverly" written Erlang source would not actually produce a buffer
overflow.

From my reading of the source code, the mentioned fix in upstream
git at

    https://github.com/erlang/otp/commit/3b7fd1030711ad56fa45d18dc6e0f1d0beee2399

should fix the issue by rewriting the code work without using
undefined C behaviour, and without tricking _FORTIFY_SOURCE checks.

I am going to try and see whether we can get an updated Fedora package
built with that fix.

(In reply to comment #6)
> Apparently, upstream have a fix in git:
> 
>     http://permalink.gmane.org/gmane.comp.lang.erlang.bugs/2208
> 
> so at least R14B02 could build again.
> 
> Short summary: Our compiling with -D_FORTIFY_SOURCE=2 enables a check
> which detects a buffer overflow in an occasion where the (too)
> "cleverly" written Erlang source would not actually produce a buffer
> overflow.
> 
> From my reading of the source code, the mentioned fix in upstream
> git at
> 
>    
> https://github.com/erlang/otp/commit/3b7fd1030711ad56fa45d18dc6e0f1d0beee2399
> 
> should fix the issue by rewriting the code work without using
> undefined C behaviour, and without tricking _FORTIFY_SOURCE checks.
> 
> I am going to try and see whether we can get an updated Fedora package
> built with that fix.

Hans, great work, indeed!

Above proposed fix does fix the issue it is supposed to fix.

    http://koji.fedoraproject.org/koji/taskinfo?taskID=2750735

Working on the new build failure now (see above build.log, happens during "make release-docs").

Renaming and closing this bug.

Remaining issues large enough to warrant bug tracking will be connected to bug 673055 in some way.