Bug 462539 - Unable to create immutable header region. When bundling 110MB rpm from simple tgz
Summary: Unable to create immutable header region. When bundling 110MB rpm from simple...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: rpm
Version: 5.2
Hardware: x86_64
OS: Linux
medium
urgent
Target Milestone: rc
: ---
Assignee: Packaging Maintenance Team
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-09-16 23:24 UTC by Aimon Bustardo
Modified: 2013-07-30 20:31 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-03-07 13:15:28 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Aimon Bustardo 2008-09-16 23:24:46 UTC
Description of problem:
When bundling an rpm from a 97MB .tgz of files I get the error: "Unable to create immutable header region."

Version-Release number of selected component (if applicable):
- CentOS 5.2 on x86_64 Xen (Linux ip-10-251-239-95 2.6.18-xenU-ec2-v1.0 #2 SMP Mon Feb 18 14:28:43 UTC 2008 x86_64 x86_64 x86_64 GNU/Linux)
- RPM version 4.4.2


How reproducible:
Happens everytime.

Steps to Reproduce:
1. create tgz with 164550 files. (replace in spec file with yours)
2. rpmbuild -bb SPECS/gem-bundle-1.0-x86_64.spec 

  
Actual results:
Error: Unable to create immutable header region.

Expected results:
Exit 0

Additional info:


Output of: valgrind --leak-check=full --show-reachable=yes rpmbuild -bb SPECS/gem-bundle-1.0-x86_64.spec 

----------BEGIN------------------------
...
+ /usr/lib/rpm/redhat/brp-strip /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
+ /usr/lib/rpm/brp-python-bytecompile
+ /usr/lib/rpm/redhat/brp-java-repack-jars
Processing files: gems-bundle-1.0-1
==6863== 
==6863== Invalid read of size 8
==6863==    at 0x70FD330: __stpcpy_chk (in /lib64/libc-2.5.so)
==6863==    by 0x582BEF5: rpmGetPath (in /usr/lib64/librpmio-4.4.so)
==6863==    by 0x4E3117B: processBinaryFiles (in /usr/lib64/librpmbuild-4.4.so)
==6863==    by 0x4E2B711: buildSpec (in /usr/lib64/librpmbuild-4.4.so)
==6863==    by 0x402E77: (within /usr/lib/rpm/rpmb)
==6863==    by 0x403142: (within /usr/lib/rpm/rpmb)
==6863==    by 0x403CFF: (within /usr/lib/rpm/rpmb)
==6863==    by 0x70368B3: (below main) (libc-start.c:231)
==6863==  Address 0x7AFF518 is 32 bytes inside a block of size 34 alloc'd
==6863==    at 0x4C1F809: malloc (vg_replace_malloc.c:149)
==6863==    by 0x582BFBA: rpmGetPath (in /usr/lib64/librpmio-4.4.so)
==6863==    by 0x4E35261: parseFiles (in /usr/lib64/librpmbuild-4.4.so)
==6863==    by 0x4E3976F: parseSpec (in /usr/lib64/librpmbuild-4.4.so)
==6863==    by 0x402B8F: (within /usr/lib/rpm/rpmb)
==6863==    by 0x403142: (within /usr/lib/rpm/rpmb)
==6863==    by 0x403CFF: (within /usr/lib/rpm/rpmb)
==6863==    by 0x70368B3: (below main) (libc-start.c:231)
Killed
-bash-3.2#
----------END--------------------------

Spec file:
----------BEGIN------------------------
%define _unpackaged_files_terminate_build 0
%define _missing_doc_files_terminate_build 0 

Summary: Ruby gems bundle 1.0 
Name: gems-bundle
Version: 1.0
Release: 1
License: GPL
Group: System Environment/Base
Source: gems-bundle-1.0-x86_64.tgz
BuildRoot: %{_tmppath}%{name}-root

%description
Ruby gems bundle 1.0 64 bit

%build
rm -rf usr %{buildroot}

%install
# Create the directory hierarchy for building our RPM.
mkdir -p %{buildroot}
cd %{buildroot}
tar -xzvf /usr/src/redhat/SOURCES/gems-bundle-1.0-x86_64.tgz
# Copy whichever files that RPM will contain into it.
install -d --mode=0644 * %{buildroot}
# Build a manifest of the RPM's directory hierarchy.
echo "%%defattr(-, root, root)" >MANIFEST
(cd %{buildroot}; find . -type f -or -type l | sed -e "/MANIFEST/d" -e "s/^.//" -e "/^$/d" -e '/ /d' ) >>MANIFEST

%clean
rm -rf src %{buildroot} MANIFEST

%files -f %{buildroot}/MANIFEST
-----------------END--------------

Comment 1 Aimon Bustardo 2008-09-16 23:38:51 UTC
Output from free:

-bash-3.2# free
             total       used       free     shared    buffers     cached
Mem:       7864460    2039720    5824740          0     142404    1015364
-/+ buffers/cache:     881952    6982508
Swap:            0          0          0
-bash-3.2#

Comment 2 Aimon Bustardo 2008-09-16 23:56:36 UTC
I reproduced this also on a default 5.2 install:

Processing files: gems-bundle-1.0-1
Provides: RMagick2.so()(64bit) bcrypt_ext.so()(64bit) do_mysql_ext.so()(64bit) fast_xs.so()(64bit) fastthread.so()(64bit) generator.so()(64bit) hpricot_scan.so()(64bit) http11.so()(64bit) ldap.so ldap.so()(64bit) mysql.so()(64bit) mysql_c.so()(64bit) parser.so()(64bit) perl(do_mysql) postgres.so()(64bit) redcloth_scan.so()(64bit)
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires: /bin/bash /bin/sh /usr/bin/env /usr/bin/ruby /usr/bin/ruby18 /usr/local/bin/ruby /usr/local/bin/ruby17 libMagick.so.10()(64bit) libMagickCore.so.1()(64bit) libMagickWand.so.1()(64bit) libc.so.6 libc.so.6()(64bit) libc.so.6(GLIBC_2.0) libc.so.6(GLIBC_2.1.3) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libcrypt.so.1 libcrypt.so.1()(64bit) libcrypto.so.6 libcrypto.so.6()(64bit) libdl.so.2 libdl.so.2()(64bit) libfreetype.so.6()(64bit) liblber-2.3.so.0 liblber-2.3.so.0()(64bit) libldap-2.3.so.0 libldap-2.3.so.0()(64bit) libldap_r-2.3.so.0 libldap_r-2.3.so.0()(64bit) libm.so.6 libm.so.6()(64bit) libm.so.6(GLIBC_2.2.5)(64bit) libmysqlclient.so.15()(64bit) libmysqlclient.so.15(libmysqlclient_15)(64bit) libnsl.so.1 libnsl.so.1()(64bit) libpq.so.4()(64bit) libpthread.so.0 libpthread.so.0()(64bit) libresolv.so.2 libresolv.so.2()(64bit) libssl.so.6 libssl.so.6()(64bit) libz.so.1()(64bit) rtld(GNU_HASH)
Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmpgems-bundle-root
warning: Installed (but unpackaged) file(s) found:
   /MANIFEST
   /usr/local/lib/ruby/gems/1.8/doc/facets-2.4.1/rdoc/files/MANIFEST.html
   /usr/local/lib/ruby/gems/1.8/doc/facets-2.4.3/rdoc/files/MANIFEST.html
   /usr/local/lib/ruby/gems/1.8/gems/english-0.2.0/meta/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.1/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.1/doc/html/api/core/files/MANIFEST.html
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.1/doc/html/api/more/files/MANIFEST.html
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.3/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.4/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/ibm_db-0.9.5/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.7.8/test/htdocs/link
with
space.html
   /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.2/ext/http11/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/rmagick-2.1.0/ext/RMagick/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/rmagick-2.2.2/ext/RMagick/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/rmagick-2.3.0/ext/RMagick/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/rmagick-2.6.0/ext/RMagick/MANIFEST
error: Unable to create immutable header region.


RPM build errors:
    Installed (but unpackaged) file(s) found:
   /MANIFEST
   /usr/local/lib/ruby/gems/1.8/doc/facets-2.4.1/rdoc/files/MANIFEST.html
   /usr/local/lib/ruby/gems/1.8/doc/facets-2.4.3/rdoc/files/MANIFEST.html
   /usr/local/lib/ruby/gems/1.8/gems/english-0.2.0/meta/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.1/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.1/doc/html/api/core/files/MANIFEST.html
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.1/doc/html/api/more/files/MANIFEST.html
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.3/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/facets-2.4.4/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/ibm_db-0.9.5/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/mechanize-0.7.8/test/htdocs/link
with
space.html
   /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.2/ext/http11/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/rmagick-2.1.0/ext/RMagick/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/rmagick-2.2.2/ext/RMagick/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/rmagick-2.3.0/ext/RMagick/MANIFEST
   /usr/local/lib/ruby/gems/1.8/gems/rmagick-2.6.0/ext/RMagick/MANIFEST
    Unable to create immutable header region.
[root@localhost redhat]# uname -a
Linux localhost.localdomain 2.6.18-92.el5xen #1 SMP Tue Jun 10 19:20:18 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux
[root@localhost redhat]#

Comment 3 Aimon Bustardo 2008-09-17 01:42:11 UTC
By reducing the file size and count of the src tgz file it will work. :/

Comment 4 Aimon Bustardo 2008-09-17 01:44:27 UTC
Let me know if you would like me to attach the tgz src file. It is 97MB.

Comment 5 Panu Matilainen 2008-09-24 10:25:15 UTC
No need, I was able to reproduce this already.

Comment 6 Panu Matilainen 2008-09-24 12:04:02 UTC
The sheer number of files causes this header size limit to be reached:
/**
 * Sanity check on data size and/or offset and/or count.
 * This check imposes a limit of 16 MB, more than enough.
 */

The limit could be upped, but the resulting packages would appear unreadable to all currently deployed rpm versions. And quite likely this is just the tip of the iceberg...

In any case, you'll be better off splitting the package up to smaller pieces, either by splitting up the bundle across several specs + tarballs or creating sub-packages from a single spec.

Comment 8 Joe Krahn 2010-09-24 15:43:01 UTC
I just ran into this limit. It would not be that hard to split files, but it would be much easier if it were documented. Even better, print the current file size+count in the error message when the limit is reached.

Comment 9 Panu Matilainen 2013-03-07 13:15:28 UTC
This request was evaluated by Red Hat Engineering for inclusion in a Red Hat Enterprise Linux maintenance release.

Red Hat does not currently plan to provide this change in a Red Hat Enterprise Linux update release for currently deployed products.

With the goal of minimizing risk of change for deployed systems, and in response to customer and partner requirements, Red Hat takes a conservative approach when evaluating enhancements for inclusion in maintenance updates for currently deployed products. The primary objectives of update releases are to enable new hardware platform support and to resolve critical defects.

However, Red Hat will further review this request for potential inclusion in future major releases of Red Hat Enterprise Linux.

As explained in comment #6, addressing the underlying limitation is not appropriate for a maintenance update. The recommended workaround is to split large packages to smaller pieces.


Note You need to log in before you can comment on or make changes to this bug.