Bug 770208 - llvm 3.0 tests fail on ARM
llvm 3.0 tests fail on ARM
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: llvm (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Michel Alexandre Salim
Fedora Extras Quality Assurance
:
Depends On:
Blocks: ARMTracker
  Show dependency treegraph
 
Reported: 2011-12-24 05:37 EST by Peter Robinson
Modified: 2012-10-04 16:47 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-10-04 16:47:09 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Peter Robinson 2011-12-24 05:37:55 EST
Compiling llvm on ARM there's a number of test failures.

  Expected Passes    : 5534
  Expected Failures  : 65
  Unsupported Tests  : 6
  Unexpected Failures: 39

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=237761
Comment 1 Peter Robinson 2011-12-24 06:00:23 EST
Similar failures in 2.8 as well.

  Expected Passes    : 2414
  Expected Failures  : 19
  Unexpected Failures: 92

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=237327
Comment 2 Peter Robinson 2012-01-26 09:43:42 EST
and on rawhide. Hello anyone home?

  Expected Passes    : 5525
  Expected Failures  : 63
  Unsupported Tests  : 6
  Unexpected Passes  : 2
  Unexpected Failures: 48


http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=275973
Comment 3 Brendan Conoboy 2012-01-31 18:15:19 EST
We continue to see this failure in rawhide on Fedora ARM with the latest gcc and glibc:

https://arm.koji.fedoraproject.org/koji/taskinfo?taskID=293141
Comment 4 Michel Alexandre Salim 2012-02-01 21:05:52 EST
Long Christmas break plus work gets in the way; my apologies. There are many build issues with gcc 4.7 at the moment -- on PPC and x86_64 the build fails outright. Once we can get a build working for x86_64 and PPC I'll look at ARM. Is there a test machine that can be used to poke around?

As for the test failures on stable releases -- should I just disable clang on those architectures for now, that way at least the LLVM RPM can be generated?
Comment 5 Michel Alexandre Salim 2012-02-01 21:07:07 EST
ehm, many LLVM test cases failed too. Nevermind -- shouldn't be trying to read debug logs at 3 AM.
Comment 6 Peter Robinson 2012-02-02 06:48:43 EST
(In reply to comment #4)
> Long Christmas break plus work gets in the way; my apologies. There are many
> build issues with gcc 4.7 at the moment -- on PPC and x86_64 the build fails
> outright. Once we can get a build working for x86_64 and PPC I'll look at ARM.
> Is there a test machine that can be used to poke around?
> 
> As for the test failures on stable releases -- should I just disable clang on
> those architectures for now, that way at least the LLVM RPM can be generated?

More interested in rawhide. It should be the priority. It's blocking us building mesa and all dependencies. Is disabling tests a workable short term fix until you get time to investigate the failures closer? What impact would this have?
Comment 7 Peter Robinson 2012-02-06 06:46:06 EST
  Expected Passes    : 5521
  Expected Failures  : 63
  Unsupported Tests  : 6
  Unexpected Passes  : 2
  Unexpected Failures: 52

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=331148

Latest mainline build still broken on ARM
Comment 8 Peter Robinson 2012-02-07 08:49:50 EST
Michel: any update on this?
Comment 9 Michel Alexandre Salim 2012-02-07 10:30:17 EST
Hi Peter,

The latest Rawhide check-in (3.0-6) *should* build fine; on %{arm} I make LLVM build failures non-terminal and save the output. But the last build I attempted failed with a weird error that seems to be due to Rawhide churn.

(an earlier build actually went further, but failed due to me removing the workaround for an Ocaml build bug; a fix was committed with a lower revision number than the 3.0 release, but it was in a different branch).

Could you take a look? If the build problem is indeed just a temporary one, feel free to reissue the build when you think it's a good time:

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=346506
Comment 10 Peter Robinson 2012-02-07 10:44:43 EST
Hi Michel,

Thanks for the update, it does look transient. I've pushed another build and we'll see how it gets on. Thanks
Comment 11 Peter Robinson 2012-02-07 15:40:44 EST
It failed. Not sure why

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=350795
Comment 12 Brendan Conoboy 2012-02-22 18:27:52 EST
Builds are partially working now.  The armv5tel build completes successfully.  The armv7hl build hangs during testing.  Here's the process pair causing the hang:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
499       8111  0.0  1.0  47340  9252 ?        Sl   Feb21   0:00 c-index-test -code-completion-at=/builddir/build/BUILD/llvm-3.0.src/tools/clang/test/Index/complete-preprocessor.m:4:2 /builddir/build/BUILD/llvm-3.0.src/tools/clang/test/Index/complete-preprocessor.m
499       8112  0.0  0.1   3656  1028 ?        S    Feb21   0:00 FileCheck -check-prefix=CHECK-CC1 /builddir/build/BUILD/llvm-3.0.src/tools/clang/test/Index/complete-preprocessor.m

An strace on 8112 shows it hung at "read (0, "

An strace on 9111 shows it hanging out on a futex, including two processes which no longer exist:
strace -f -p 8111
Process 8111 attached with 3 threads - interrupt to quit
[pid  8135] futex(0x41c96a78, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
[pid  8119] futex(0x42eff4c8, FUTEX_WAIT, 8135, NULL <unfinished ...>
[pid  8111] futex(0x424a44c8, FUTEX_WAIT, 8119, NULL^C <unfinished ...>
Process 8111 detached
Process 8119 detached
Process 8135 detached

(That ^C is me).

Looking through the logs, one potential candidate for why this works on armv5tel and fails on armv7hl is this clang warning:

clang: warning: unknown platform, assuming -mfloat-abi=soft

That's a problem.  It should be assuming -mfloat-abi=hard.  Not sure why it isn't as llvm target configurey is new to me.  Perhaps somebody more familiar with clang knows the right place to change this assumption for armv7hl-linux-gnueabi targets?
Comment 13 Dennis Gilmore 2012-02-27 17:19:29 EST
Looking at the spec file, 

%check
# the Koji build server does not seem to have enough RAM
# for the default 16 threads

# LLVM test suite failing on ARM, PPC64 and s390(x)
make check LIT_ARGS="-v -j4" \
%ifarch %{arm} ppc64 s390 s390x
     | tee llvm-testlog-%{_arch}.txt
%else
 %{nil}
%endif


the builders on primary arch with 16 cores have 20gb ram.  the arm builders are dual core with 1gb ram.  so we are going from over 1gb per core to 512mb per core.  building at -j4 on them would be too much. 

you could rewrite the smp_mflags macro to limit to 4 if more than 4 cores are present.  but really something there needs fixing

certainly we should be making sure that we are using hard or soft float as appropriate  armv5tel armv6l armv7l are all using soft  and armv7hl and armv7hnl are hard float.
Comment 14 Peter Robinson 2012-03-14 07:39:16 EDT
Michel we're still seeing issues with getting reliable builds, can you review Dennis's feedback and fix according please.
Comment 15 Peter Robinson 2012-04-03 09:22:14 EDT
Michel: any update what so ever? This is now SERIOUSLY causing us blocking issues.
Comment 16 Peter Robinson 2012-10-04 16:47:09 EDT
worked around. The remaining issues will be dealt with in 803433

Note You need to log in before you can comment on or make changes to this bug.