Bug 913099

Summary: Unable to create immutable header region
Product: Red Hat Enterprise Linux 6 Reporter: Jonathan Underwood <jonathan.underwood>
Component: rpmAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED DEFERRED QA Contact: BaseOS QE Security Team <qe-baseos-security>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.3CC: edgar.hoch, ffesti, jzeleny, pmatilai
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-03-18 13:36:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jonathan Underwood 2013-02-20 11:38:05 UTC
Description of problem:
When creating an (admittedly large) RPM, rpmbuild bombs out with:

Processing files: matlab-2012b-1.el6.x86_64
Checking for unpackaged file(s): /usr/lib/rpm/check-files /local/home/rpmb/rpmbuild/BUILDROOT/matlab-2012b-1.el6.x86_64
Wrote: /local/home/rpmb/rpmbuild/SRPMS/matlab-2012b-1.el6.src.rpm
error: Unable to create immutable header region.


RPM build errors:
    Unable to create immutable header region.


I was a bit surprised to hit this as although I am trying to build a big package, it's well below the 16 GB limit for rpm.

buildroot details for the package:

$ find ../BUILDROOT/matlab-2012b-1.el6.x86_64/ | wc -l
247603

$ du -sh ../BUILDROOT/matlab-2012b-1.el6.x86_64/
6.9G	../BUILDROOT/matlab-2012b-1.el6.x86_64/



Version-Release number of selected component (if applicable):
rpm-4.8.0-27.el6.x86_64

How reproducible:
Everytime.

Steps to Reproduce:
1. Try to build a large rpm with a moderate number of files
2. Boom.
3.
  
Actual results:


Expected results:


Additional info:

Comment 2 Panu Matilainen 2013-02-20 12:30:48 UTC
There's no "16 GB" limit, the theoretical maximum payload size is INT64_MAX but in practise you'll run into various other issues long before that. Such as this... It's not related to the size of the contained files, but the total number of them which causes the metadata size to exceed the (artificial) limitations imposed on headers:

/**
 * Sanity check on data size and/or offset and/or count.
 * This check imposes a limit of 16 MB, more than enough.
 */
#define HEADER_DATA_MAX 0x00ffffff
#define hdrchkData(_nbytes) ((_nbytes) & (~HEADER_DATA_MAX))

/** \ingroup header
 * Maximum no. of bytes permitted in a header.
 */
static const size_t headerMaxbytes = (32*1024*1024);

See bug 462539 and bug 706935 for background and related issues.
For now, the easy workaround is to split up the package into smaller pieces.

Comment 3 Jonathan Underwood 2013-02-20 14:42:36 UTC
OK, understood, thanks. I guess that's a WONTFIX for rhel6.  Can you regard it as an RFE for rhel7 tho? :)

Comment 4 RHEL Program Management 2013-02-24 06:47:35 UTC
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

Comment 5 Florian Festi 2013-03-18 13:36:02 UTC
OK, closing it here. The issue moved to the upstream bug tracker to be fixes in future RPM versions: http://rpm.org/ticket/862

Thanks for reporting!

Comment 6 Edgar Hoch 2013-06-08 05:48:39 UTC
I ran into the same problem with fedora 18 and matlab 2013a with all matlab tool boxes. Will there be a fix in fedora 19 and redhat 7?
In the upstream bug report there are no changes since the report.

Comment 7 Jan Zeleny 2013-06-10 08:17:16 UTC
We are currently working on the improved payload format. However it is not even in upstream yet so getting it into RHEL and Fedora will take some time. We *might* be able to push it in F20 but certainly not sooner (F19 has a devel freeze in a few weeks IIRC). Then it needs some stabilization and testing before it hits RHEL. Therefore the chances of this feature hitting RHEL7 during its life cycle are very close to zero.

Comment 8 Edgar Hoch 2013-06-10 08:56:18 UTC
Thanks for the evaluation!


A hint for those who run into problems with the big data of matlab (see comment 1):

I have used subpackages in the rpm spec file to create a number of smaller rpm files instead of one big which crashes during build. For example, I used the following files declaration:

%files part-rest
%{destdir}
%exclude %{destdir}/bin
%exclude %{destdir}/help
%exclude %{destdir}/toolbox

%files part-bin
%{destdir}/bin

%files part-help1
%dir %{destdir}/help
%{destdir}/help/[a-l]*

%files part-help2
%dir %{destdir}/help
%{destdir}/help/[m-rt-z]*

%files part-help3
%dir %{destdir}/help
%{destdir}/help/[^a-rt-z]*

%files part-toolbox1
%dir %{destdir}/toolbox
%{destdir}/toolbox/[a-c]*

%files part-toolbox2
%dir %{destdir}/toolbox
%{destdir}/toolbox/[c-h]*

%files part-toolbox3
%dir %{destdir}/toolbox
%{destdir}/toolbox/[h-r]*

%files part-toolbox4
%dir %{destdir}/toolbox
%{destdir}/toolbox/[^a-r]*

%files


See the documentation for rpm spec files how to create subpackages.

Comment 9 Panu Matilainen 2013-06-10 08:58:29 UTC
Jan, this is not a payload issue but an artificial cap on header size.

Comment 10 Jan Zeleny 2013-06-10 10:41:53 UTC
(In reply to Panu Matilainen from comment #9)
> Jan, this is not a payload issue but an artificial cap on header size.

Ah, sorry, I misunderstood the description and figured it will be addressed together with the payload change. Thanks for the correction.

Anyway I still don't expect this to land neither in F19 nor in RHEL7, right?