Bug 461793

Summary: Review Request: spu-gcc - Cross Compiling GNU GCC targeted at spu
Product: [Fedora] Fedora Reporter: Jochen Roth <jroth>
Component: Package ReviewAssignee: Nobody's working on this, feel free to take it <nobody>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: a.j.delaney, fedora-package-review, hannsj_uhl, ken, msuchy, notting, rc040203, uweigand
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: NotReady
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-11 18:11:05 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 201449, 461791, 462958    

Description Jochen Roth 2008-09-10 12:14:01 EDT
Spec URL: ftp://testcase.software.ibm.com/fromibm/linux/spu-gcc.spec
SRPM URL: ftp://testcase.software.ibm.com/fromibm/linux/spu-gcc-4.3.1-1.fc9.src.rpm
Description:
This is a Cross Compiling version of GNU GCC, which can be used to
compile for the spu platform, instead of for the
native ppc64 platform.
Comment 1 Aidan Delaney 2008-09-10 15:14:42 EDT
The IBM links are only expected to be live for 5 days.  I (at Jochen's request) have copied these files to 

http://foss.it.brighton.ac.uk/~balor/rpm/spu-gcc-4.3.1-1.fc9.src.rpm
and
http://foss.it.brighton.ac.uk/~balor/rpm/spu-gcc.spec
Comment 2 Aidan Delaney 2008-10-02 17:05:42 EDT
Jochen and I considered two methods of breaking the circularity concerning building spu-gcc and spu-newlib.  The problem is that spu-gcc needs spu-newlib during build, whereas spu-newlib needs spu-gcc during its build.  The two ways of solving this are
(a) do a build on a unified source tree for gcc and newlib, xor
(b) provide some bootstrap gcc compiler to build newlib, which can then be used to build spu-gcc.

After some investigation we found that (a) is quite brittle.  None of the configurations of gcc+newlib (that we tried) build in a unified tree.  Therefore we have opted for (b).

A simple spu-gcc-bootstrap is available here:
http://foss.it.brighton.ac.uk/~balor/rpm/spu-gcc/r1/spu-gcc-bootstrap.spec
and a spu-gcc which conflicts with spu-gcc-bootstrap is available here:
http://foss.it.brighton.ac.uk/~balor/rpm/spu-gcc/r1/spu-gcc.spec
Comment 3 Jason Tibbitts 2009-07-10 13:47:56 EDT
Wow, this is an old one.  Unfortunately I'm not at all sure how one is supposed to review this.

Generally when bootstrapping compilers, we allow the initial package to carry the necessary binaries required to build them.  After they're in the distribution, that can be turned off.  I see no reason why you couldn't just include the binaries for newlib here until a compiler could be built that could then build newlib  This would require work for each architecture, of course, but I don't see that as being insurmountable.
Comment 4 Aidan Delaney 2009-07-14 09:00:17 EDT
Thanks.  I'll upgrade the packages for Fedora 11 and do this.
Comment 5 Ralf Corsepius 2009-07-14 11:46:28 EDT
(In reply to comment #3)
> Generally when bootstrapping compilers, we allow the initial package to carry
> the necessary binaries required to build them.
This consideration doesn't apply to newlib based GCCs.

They can be bootstrapped in one single stage without any bootstrap packages.
Comment 6 Jason Tibbitts 2009-07-14 11:52:36 EDT
It applies to any package where it is necessary.

If you believe it's not necessary here, could you possibly consider lowering yourself to the plane of mere mortals and, say, provide some information on how to do it?  Because the submitter of this package said in comment #2 that it doesn't work for them.
Comment 7 Ralf Corsepius 2009-07-14 12:12:11 EDT
(In reply to comment #6)
> It applies to any package where it is necessary.
No. It's a last escape for package for which bootstrapping isn't possible.

> If you believe it's not necessary here, could you possibly consider lowering
> yourself to the plane of mere mortals and, say, provide some information on how
> to do it?
Comment #2 contains the answer.

>  Because the submitter of this package said in comment #2 that it
> doesn't work for them.  
Well, I never tried for the spu, but there is one thing I know for sure: His configure call is broken.
Comment 8 Jason Tibbitts 2009-07-14 12:25:55 EDT
(In reply to comment #7)
> No. It's a last escape for package for which bootstrapping isn't possible.

Amazing how you find ways to agree with someone while still being negative.

> Comment #2 contains the answer.

Now he speaks in riddles.

> Well, I never tried for the spu, but there is one thing I know for sure: His
> configure call is broken.  

And of course you can't be bothered to point out just how it is broken.

I know you pride yourself on being maximally contradictory and at the same time minimally helpful, and I must say that you're really outdoing yourself this time.  Bravo!
Comment 9 Ulrich Weigand 2009-07-14 13:59:32 EDT
Unfortunately Jochen is no longer with IBM; I'll see if I can continue the work he has started in order to get SPU toolchain packages into Fedora.  (I did the Cell SDK compiler packages that Jochen's Fedora packages were based on.)

As to building gcc/newlib in a combined tree, this is currently broken (there's at least latent breakage on all platforms, and it definitely does not work on SPU).

The reason for this is that in the combined tree, GCC attempts to use header files from newlib sources to mimic the set of files you'd get in a real newlib install.  This is supposed to work by adding the following include directories:
 -isystem $(build)/$(machine)/newlib/targ-include
 -isystem $(src)/newlib/libc/include

The targ-include directory is supposed to be populated by the newlib Makefile to hold the target-specific headers as they'll be installed.

Unfortunately, it seems there is some build-order problem with the current top-level Makefile, which causes the targ-include directory not to be populated before the GCC target libraries are built.

Therefore, we only get machine-independent headers; all machine-dependent headers are missing.  On the SPU, (at least) one of those headers is critical; if it is missing, common newlib header code runs into an #error.  (On other platforms, I guess the problem is still present in some latent form.)


I guess one could try and fix these issues to get the combined tree build working again.  It simply looked easier for us to build the packages separately, because it seemed the combined tree is no longer frequently used (which explains why it tends to get broken).

Note that for a bootstrap sequence of separate packages you do not necessarily need binary contents: you could do a "spu-newlib-headers" package that just contains newlib headers as appropriate for the target (copied in the proper sequence from newlib sources), followed by a "spu-gcc" that just uses this headers package, and then build a full "spu-newlib".  (This is what OpenSUSE currently does, IIRC.)

In the alternative, you can do a "spu-gcc-bootstrap" that is C only and does not require any newlib (or headers), use this to build "spu-newlib", and finally build a full "spu-gcc".  (This seems to be what Aidan proposed.)
Comment 10 Ralf Corsepius 2009-07-14 18:07:48 EDT
(In reply to comment #8)
> (In reply to comment #7)
> > No. It's a last escape for package for which bootstrapping isn't possible.
> 
> Amazing how you find ways to agree with someone while still being negative.
Rubbish - I am saying you are bogusly generalizing.

bootstrap packages are the last resort if all else fail - Not the norm!

> > Comment #2 contains the answer.
> 
> Now he speaks in riddles.
Well, make yourselves familiar with newlib and GCC. comment #2 contains all necessary information.

> > Well, I never tried for the spu, but there is one thing I know for sure: His
> > configure call is broken.  
> 
> And of course you can't be bothered to point out just how it is broken.
Have a look at how I do it:
ftp://ftp.rtems.org/pub/rtems/linux/4.10/fedora/11/SRPMS

> I know you pride yourself on being maximally contradictory and at the same time
> minimally helpful, and I must say that you're really outdoing yourself this
> time.  Bravo!

1. Tibbs, I am not willing to any longer accept this tone of yours and am expecting an excuse of yours!

2. I am trying to help them to enable them the help themselves, but refuse to do to _their_ homework.
Comment 11 Jason Tibbitts 2009-07-14 18:39:49 EDT
(In reply to comment #10)

> bootstrap packages are the last resort if all else fail - Not the norm!

I appreciate your strenuously agreeing with me, thanks.

> Have a look at how I do it:
> ftp://ftp.rtems.org/pub/rtems/linux/4.10/fedora/11/SRPMS

Well, thanks for being sort of helpful.  Perhaps Ulrich can glean some of the divine knowledge from that directory of source packages.

> 1. Tibbs, I am not willing to any longer accept this tone of yours and am
> expecting an excuse of yours!

Oh, my apologies.  I actually intended to write "contrary", not "contradictory" there.  I'm sincerely sorry for calling you contradictory, for while you often are, I don't believe you've been so in this ticket.  "Contrary", though, I'll stand behind.

Good day, sir.

> 2. I am trying to help them to enable them the help themselves, but refuse to
> do to _their_ homework.  

Ulrich, do you think you can unravel the mystery here as presented by friend Ralf?
Comment 12 Ralf Corsepius 2009-07-14 19:20:03 EDT
(In reply to comment #9)

Ulrich, could you please check the spu-packages below
http://corsepiu.fedorapeople.org/packages/
for completeness?

The spec is very rough with many improper spots, I hacked together using my rtems-gcc/newlib specs as template. They also contain the rtems rtems-gcc/newlib patches, which are supposed to address several of the issues you mentioned.

Shouldn't you find files missing, I would be willing to clean the spec up such it fits into Fedora.
Comment 13 Ulrich Weigand 2009-07-15 09:09:40 EDT
(In reply to comment #12)
> (In reply to comment #9)
> Ulrich, could you please check the spu-packages below
> http://corsepiu.fedorapeople.org/packages/
> for completeness?

Thanks for your support!

Unfortunately, while the package above does build, it results in GCC target libraries built without all pieces that require C runtime support (for example, libgcov.a consists solely of stubs).  The resulting compiler would be suitable as bootstrap compiler, but not as final system compiler ...

This is because your spec file uses --with-newlib, but *not* --with-headers as configure options.  This causes the GCC Makefile to define the "inhibit_libc" parameter, which in turn causes all GCC target library code that requires C runtime support to be #ifdef'd out.

Once you add --with-headers to the configure line in order to build a full set of GCC target libraries, you run into the very same problem I've described above -- compilation of target libraries fails because the targ-include directory is not yet populated.

I see that you're adding patches that touch this area:

diff -Naur newlib-1.17.0.orig/newlib/Makefile.am newlib-1.17.0/newlib/Makefile.am
--- newlib-1.17.0.orig/newlib/Makefile.am       2008-01-21 19:29:17.000000000 +0100
+++ newlib-1.17.0/newlib/Makefile.am    2009-05-05 07:11:13.000000000 +0200
@@ -81,7 +81,7 @@
        libc.a
 endif

-noinst_DATA = stmp-targ-include
+BUILT_SOURCES = stmp-targ-include

 toollib_DATA = $(CRT0) $(CRT1)

I'm not sure what exactly this is intended to achive, but apparently it does not help to fix the top-level build order; GCC target libraries are still built before targ-include.

Any suggestion how to fix this would certainly be appreciated!
Comment 14 Jason Tibbitts 2010-11-02 17:40:34 EDT
I note that none of the links to specs or srpms are valid.  I'm guessing this should just be closed out, which is disappointing, but I can't review this and nobody seemed to have the time to help fix the issues.

I guess if anyone does what to continue this, please provide working links and clear the whiteboard.
Comment 15 Miroslav Suchý 2012-12-11 18:11:05 EST
Closing per #14