Bugzilla will be upgraded to version 5.0 on a still to be determined date in the near future. The original upgrade date has been delayed.
Bug 166029 - rpmbuild crashes when rpm-ing up an 8 gig file
rpmbuild crashes when rpm-ing up an 8 gig file
Product: Fedora
Classification: Fedora
Component: rpm (Show other bugs)
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Paul Nasrat
Mike McLean
Depends On:
  Show dependency treegraph
Reported: 2005-08-15 19:35 EDT by Wolfgang Rupprecht
Modified: 2007-11-30 17:11 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2006-02-24 13:25:40 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Wolfgang Rupprecht 2005-08-15 19:35:51 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; OpenBSD amd64; en-US; rv:1.7.10) Gecko/20050724 Firefox/1.0.6

Description of problem:
rpmbuild takes a long time and finally segv's when trying to create an
rpm if one of the enclosed files is an 7.7GiB database file.  Building an
rpm from an essentially identical spec file that specifies a smaller
500MB database file succeeds.  The only difference is the size of the
enclosed file.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.generate a spec file to rpm up a data directory containing 8GB of data
2.rpmbuild -vvv -bb foo-data.spec

Actual Results:  a long wait and rpmbuild says it segv-ed

Expected Results:  rpmbuild should produce an rpm file in /usr/share/redhat/RPMS/

Additional info:

This is a show-stopper bug for releasing a large dataset using RPM.
Can someone that understands this code please do a quick once over and
see if somebody made the obvious mistake of trying to stuff an
offset_t into an int?

This short demo doesn't show the full file sizes before it crashes,
but the real spec file with real data succeeded in printing out the
data file's size.  It was a negative number.  This seems like a hint
that the size of the file size variable was a bit too small for its
own good.

$ cat bigfile-crash-demo.spec

Buildroot: /tmp/bigfile-crash-demo

Summary: big datafile crash demo
Name: bigfile-crash-demo
Version: 0.1
Release: 1
Group: Applications/devel
BuildArch: noarch
Distribution: none
Packager: wolfgang
ExcludeOS: anything-from-redmond 
License: freebeer

demonstrate how rpmbuild rolls over and dies with big data files.
invode with "rpmbuild -vv -bb bigfile-crash-demo.spec"



rm -rf /tmp/bigfile-crash-demo
mkdir -p $RPM_BUILD_ROOT/bigfile-crash-demo
cd $RPM_BUILD_ROOT/bigfile-crash-demo && dd if=/dev/zero of=crashdemo.dat bs=1000000 count=7702

rm -rf /tmp/bigfile-crash-demo



$ time nice -20 rpmbuild -vv -bb bigfile-crash-demo.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.42125
+ umask 022
+ cd /usr/src/redhat/BUILD
+ export LANG
+ unset DISPLAY
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.42125
+ umask 022
+ cd /usr/src/redhat/BUILD
+ export LANG
+ unset DISPLAY
+ exit 0
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.42125
+ umask 022
+ cd /usr/src/redhat/BUILD
+ export LANG
+ unset DISPLAY
+ rm -rf /tmp/bigfile-crash-demo
+ mkdir -p /tmp/bigfile-crash-demo/bigfile-crash-demo
+ cd /tmp/bigfile-crash-demo/bigfile-crash-demo
+ dd if=/dev/zero of=crashdemo.dat bs=1000000 count=7702
7702+0 records in
7702+0 records out
+ /usr/lib/rpm/redhat/brp-compress
+ /usr/lib/rpm/redhat/brp-strip /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
Processing files: bigfile-crash-demo-0.1-1
D: /tmp/bigfile-crash-demo/bigfile-crash-demo: directory
D: /tmp/bigfile-crash-demo/bigfile-crash-demo/crashdemo.dat: data
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /tmp/bigfile-crash-demo
D:      execv(/usr/lib/rpm/check-files) pid 6372
D:      waitpid(6372) rc 6372 status 0
D: fini      040755  2 (   0,   0)         0 /bigfile-crash-demo 
Segmentation fault

real    9m16.665s
user    3m4.307s
sys     0m51.148s
Comment 1 Jeff Johnson 2005-08-25 12:56:27 EDT
Yup, rpm (and cpio used as payload format) do not handle files >2Gb.
Comment 2 John Mahowald 2005-10-03 00:10:34 EDT
Supposedly cpio >2 GB has been fixed with bug #145238. Might this be something
else, like check-files itself?
Comment 3 Paul Nasrat 2005-11-28 13:47:57 EST
rpm uses it's own cpio implementation that does not support >2Gb
Comment 4 Jeff Johnson 2006-02-24 13:25:40 EST
rpm-4.4.5 should handle up to 4 Gb files.

More than 4 Gb files will require data type changes within *.rpm packages.
That work is well underway, but will take time because of incomaptibilities.

DEFERRED until somewhen.

Note You need to log in before you can comment on or make changes to this bug.