Bug 770208 - llvm 3.0 tests fail on ARM
Summary: llvm 3.0 tests fail on ARM
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: llvm
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Michel Lind
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: ARMTracker
TreeView+ depends on / blocked
 
Reported: 2011-12-24 10:37 UTC by Peter Robinson
Modified: 2012-10-04 20:47 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-10-04 20:47:09 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Peter Robinson 2011-12-24 10:37:55 UTC
Compiling llvm on ARM there's a number of test failures.

  Expected Passes    : 5534
  Expected Failures  : 65
  Unsupported Tests  : 6
  Unexpected Failures: 39

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=237761

Comment 1 Peter Robinson 2011-12-24 11:00:23 UTC
Similar failures in 2.8 as well.

  Expected Passes    : 2414
  Expected Failures  : 19
  Unexpected Failures: 92

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=237327

Comment 2 Peter Robinson 2012-01-26 14:43:42 UTC
and on rawhide. Hello anyone home?

  Expected Passes    : 5525
  Expected Failures  : 63
  Unsupported Tests  : 6
  Unexpected Passes  : 2
  Unexpected Failures: 48


http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=275973

Comment 3 Brendan Conoboy 2012-01-31 23:15:19 UTC
We continue to see this failure in rawhide on Fedora ARM with the latest gcc and glibc:

https://arm.koji.fedoraproject.org/koji/taskinfo?taskID=293141

Comment 4 Michel Lind 2012-02-02 02:05:52 UTC
Long Christmas break plus work gets in the way; my apologies. There are many build issues with gcc 4.7 at the moment -- on PPC and x86_64 the build fails outright. Once we can get a build working for x86_64 and PPC I'll look at ARM. Is there a test machine that can be used to poke around?

As for the test failures on stable releases -- should I just disable clang on those architectures for now, that way at least the LLVM RPM can be generated?

Comment 5 Michel Lind 2012-02-02 02:07:07 UTC
ehm, many LLVM test cases failed too. Nevermind -- shouldn't be trying to read debug logs at 3 AM.

Comment 6 Peter Robinson 2012-02-02 11:48:43 UTC
(In reply to comment #4)
> Long Christmas break plus work gets in the way; my apologies. There are many
> build issues with gcc 4.7 at the moment -- on PPC and x86_64 the build fails
> outright. Once we can get a build working for x86_64 and PPC I'll look at ARM.
> Is there a test machine that can be used to poke around?
> 
> As for the test failures on stable releases -- should I just disable clang on
> those architectures for now, that way at least the LLVM RPM can be generated?

More interested in rawhide. It should be the priority. It's blocking us building mesa and all dependencies. Is disabling tests a workable short term fix until you get time to investigate the failures closer? What impact would this have?

Comment 7 Peter Robinson 2012-02-06 11:46:06 UTC
  Expected Passes    : 5521
  Expected Failures  : 63
  Unsupported Tests  : 6
  Unexpected Passes  : 2
  Unexpected Failures: 52

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=331148

Latest mainline build still broken on ARM

Comment 8 Peter Robinson 2012-02-07 13:49:50 UTC
Michel: any update on this?

Comment 9 Michel Lind 2012-02-07 15:30:17 UTC
Hi Peter,

The latest Rawhide check-in (3.0-6) *should* build fine; on %{arm} I make LLVM build failures non-terminal and save the output. But the last build I attempted failed with a weird error that seems to be due to Rawhide churn.

(an earlier build actually went further, but failed due to me removing the workaround for an Ocaml build bug; a fix was committed with a lower revision number than the 3.0 release, but it was in a different branch).

Could you take a look? If the build problem is indeed just a temporary one, feel free to reissue the build when you think it's a good time:

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=346506

Comment 10 Peter Robinson 2012-02-07 15:44:43 UTC
Hi Michel,

Thanks for the update, it does look transient. I've pushed another build and we'll see how it gets on. Thanks

Comment 11 Peter Robinson 2012-02-07 20:40:44 UTC
It failed. Not sure why

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=350795

Comment 12 Brendan Conoboy 2012-02-22 23:27:52 UTC
Builds are partially working now.  The armv5tel build completes successfully.  The armv7hl build hangs during testing.  Here's the process pair causing the hang:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
499       8111  0.0  1.0  47340  9252 ?        Sl   Feb21   0:00 c-index-test -code-completion-at=/builddir/build/BUILD/llvm-3.0.src/tools/clang/test/Index/complete-preprocessor.m:4:2 /builddir/build/BUILD/llvm-3.0.src/tools/clang/test/Index/complete-preprocessor.m
499       8112  0.0  0.1   3656  1028 ?        S    Feb21   0:00 FileCheck -check-prefix=CHECK-CC1 /builddir/build/BUILD/llvm-3.0.src/tools/clang/test/Index/complete-preprocessor.m

An strace on 8112 shows it hung at "read (0, "

An strace on 9111 shows it hanging out on a futex, including two processes which no longer exist:
strace -f -p 8111
Process 8111 attached with 3 threads - interrupt to quit
[pid  8135] futex(0x41c96a78, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
[pid  8119] futex(0x42eff4c8, FUTEX_WAIT, 8135, NULL <unfinished ...>
[pid  8111] futex(0x424a44c8, FUTEX_WAIT, 8119, NULL^C <unfinished ...>
Process 8111 detached
Process 8119 detached
Process 8135 detached

(That ^C is me).

Looking through the logs, one potential candidate for why this works on armv5tel and fails on armv7hl is this clang warning:

clang: warning: unknown platform, assuming -mfloat-abi=soft

That's a problem.  It should be assuming -mfloat-abi=hard.  Not sure why it isn't as llvm target configurey is new to me.  Perhaps somebody more familiar with clang knows the right place to change this assumption for armv7hl-linux-gnueabi targets?

Comment 13 Dennis Gilmore 2012-02-27 22:19:29 UTC
Looking at the spec file, 

%check
# the Koji build server does not seem to have enough RAM
# for the default 16 threads

# LLVM test suite failing on ARM, PPC64 and s390(x)
make check LIT_ARGS="-v -j4" \
%ifarch %{arm} ppc64 s390 s390x
     | tee llvm-testlog-%{_arch}.txt
%else
 %{nil}
%endif


the builders on primary arch with 16 cores have 20gb ram.  the arm builders are dual core with 1gb ram.  so we are going from over 1gb per core to 512mb per core.  building at -j4 on them would be too much. 

you could rewrite the smp_mflags macro to limit to 4 if more than 4 cores are present.  but really something there needs fixing

certainly we should be making sure that we are using hard or soft float as appropriate  armv5tel armv6l armv7l are all using soft  and armv7hl and armv7hnl are hard float.

Comment 14 Peter Robinson 2012-03-14 11:39:16 UTC
Michel we're still seeing issues with getting reliable builds, can you review Dennis's feedback and fix according please.

Comment 15 Peter Robinson 2012-04-03 13:22:14 UTC
Michel: any update what so ever? This is now SERIOUSLY causing us blocking issues.

Comment 16 Peter Robinson 2012-10-04 20:47:09 UTC
worked around. The remaining issues will be dealt with in 803433


Note You need to log in before you can comment on or make changes to this bug.