Bug 835957
| Summary: | malformed archive error during webkitgtk3 build | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Dan Horák <dan> |
| Component: | binutils | Assignee: | Nick Clifton <nickc> |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rawhide | CC: | jakub, mjw, nickc, pbrobinson, zkabelac |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | s390x | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2012-11-25 18:42:49 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Dan Horák
2012-06-27 16:28:17 UTC
> one note - the libWebCore.a archive is > 4GB
Have you tried splitting libWebCore.a up into several smaller (sub 4Gb) libraries ?
Hi Dan, This might also be connected to this FSF binutils bug report: http://sourceware.org/bugzilla/show_bug.cgi?id=14302 Are you able to try out the patch included in that bug report or do you need me to produce an experimental binutils rpm for you ? Cheers Nick (In reply to comment #1) > > one note - the libWebCore.a archive is > 4GB > > Have you tried splitting libWebCore.a up into several smaller (sub 4Gb) > libraries ? Is there an easy way to do the split? I'd prefer not to touch webkit's buildsystem. (In reply to comment #2) > Hi Dan, > > This might also be connected to this FSF binutils bug report: > > http://sourceware.org/bugzilla/show_bug.cgi?id=14302 > > Are you able to try out the patch included in that bug report or do you need > me to produce an experimental binutils rpm for you ? I should be able to prepare patched binutils rpm, but if you have some spare time it could speed up further testing. The buildroot is on Red Hat internal system, so you can yourself take a look easily. Hi Dan,
> The buildroot is on Red Hat internal system, so you can yourself take a look
> easily.
Err, do you know how I can access the built libWebCore.a archive ? I tried following the link you posted to the koji task, but I could not work out where to go from there.
Anyway, I have created a binutils-2.22.52.0.4-3.fc18 release for rawhide which includes the patch from PR 14302. Can you give this release a go and let me know if it solves the problem ?
Cheers
Nick
Hi Dan, I am still investigating this problem, but I have some bad news. The libWebCore,a archive is definitely broken. The linker is correct in stating that it is malformed. Currently, archives bigger than 4Gb are just not supported. This is a limitation of the archive file format, not the linker. The real error here is that the library should never have been created in the first place. I am looking in to how to fix the archiver (ar) so that it will complain when creating a library that is too big. But in the meantime I think that you are going to have to delve into webkit's build system and find a way to split the libWebCore.a library up into smaller components. Sorry. Cheers Nick Hi Nick, first thanks for investigation and the news is not so bad. We know what the problem is and can react on that. I will send a mail to the fedora devel list so other people are also aware of this issue which can happen in large C++ projects (and can easily happen with webkit on other arches). Dan Naive question here - but how anything could link into 4GB size - static linking just combines other static object together and should make it rather smaller (since it might skip unused bits) So is there 4GB of .o files created during compilation ? My kernel compilation fits in 18MB - so I cannot imagine how do you make 4GB object ?? Is there something special about s390 linkage ? Isn't the ar format limitation 9.3GiB file size inside of the ar archive (above that it won't fit into 10 decimal digits reserved in the file header)? The webkit library is just an insane kitchen sink, for dwz DWARF compressor it is also a challenge to reduce the ~ 1.2GB of debug info, more than twice the maximum amount seen everywhere else. (In reply to comment #9) > Naive question here - but how anything could link into 4GB size - static > linking just combines other static object together and should make it rather > smaller (since it might skip unused bits) > > So is there 4GB of .o files created during compilation ? no, what's broken is already the .a which is (simplified) a sum of .o files > My kernel compilation fits in 18MB - so I cannot imagine how do you make 4GB > object ?? > > Is there something special about s390 linkage ? the majority of the data are the debug information that are included in every .o file (In reply to comment #11) > (In reply to comment #9) > > Naive question here - but how anything could link into 4GB size - static > > linking just combines other static object together and should make it rather > > smaller (since it might skip unused bits) > > > > So is there 4GB of .o files created during compilation ? > > no, what's broken is already the .a which is (simplified) a sum of .o files ah, I got it wrong, yes, there is over 4GB of .o files created Hi Jakub, (In reply to comment #10) > Isn't the ar format limitation 9.3GiB file size inside of the ar archive > (above that it won't fit into 10 decimal digits reserved in the file header)? There is also a 32-bit limit on the size of file offsets in symbol table at the start of the archive (if it has one). This was the problem - the linker uses these file offsets to locate individual archive members and since these had been truncated when the archive symbol table was created, the linker failed to find the archive members that it wanted. FYI - there is a 64bit archive file format, which is currently only used by the MIPS and IA64 targets. I suppose that it might be worth investigating to see if it could be made available to the S390X target as well... Cheers Nick would be interesting to see how close to the limit ppc64 and x86_64 are ... Can't we just use the 64-bit (is that the Irix format?) symbol table format for archives larger than 4GB and the current default for smaller ones, and be able to parse the 64-bit symbol tables always (or, at least when BFD64 is supported). Hi Jakub,
> Can't we just use the 64-bit (is that the Irix format?) symbol table format
Hi Jakub,
[arg - stupid bugzilla dialog boxes...]
> Can't we just use the 64-bit (is that the Irix format?) symbol table format
I am just about to start investigating that.
Cheers
Nick
Hi Dan. binutils-2.22.52.0.4-5.fc18 has an extended version of the previous patch which will make the archive program (ar) complain if there is an attempt to build a library (with a symbol index) that is larger than 4Gb. I am still looking at the possibility of supporting 64-bit libraries, but so far I have not made any progress. Cheers Nick Hi Dan, binutils-2.22.52.0.4-6.fc18 has support for s390 libraries with 64-bit archive indicies. It should, in theory, solve your build problems. Please could you give it a go and let me know if it works. Cheers Nick Hi Nick, seems your update works well, my test case was webkitgtk-1.8.1-3.fc18 and first it failed with "ar: .libs/libWebCore.a: File truncated" (binutils.s390x 0:2.22.52.0.4-5.fc18) and now it builds fine. Thanks, Dan |