Bug 166029 - rpmbuild crashes when rpm-ing up an 8 gig file
Summary: rpmbuild crashes when rpm-ing up an 8 gig file
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: 4
Hardware: x86_64
OS: Linux
Target Milestone: ---
Assignee: Paul Nasrat
QA Contact: Mike McLean
Depends On:
TreeView+ depends on / blocked
Reported: 2005-08-15 23:35 UTC by Wolfgang Rupprecht
Modified: 2007-11-30 22:11 UTC (History)
1 user (show)

Clone Of:
Last Closed: 2006-02-24 18:25:40 UTC

Attachments (Terms of Use)

Description Wolfgang Rupprecht 2005-08-15 23:35:51 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; OpenBSD amd64; en-US; rv:1.7.10) Gecko/20050724 Firefox/1.0.6

Description of problem:
rpmbuild takes a long time and finally segv's when trying to create an
rpm if one of the enclosed files is an 7.7GiB database file.  Building an
rpm from an essentially identical spec file that specifies a smaller
500MB database file succeeds.  The only difference is the size of the
enclosed file.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.generate a spec file to rpm up a data directory containing 8GB of data
2.rpmbuild -vvv -bb foo-data.spec

Actual Results:  a long wait and rpmbuild says it segv-ed

Expected Results:  rpmbuild should produce an rpm file in /usr/share/redhat/RPMS/

Additional info:

This is a show-stopper bug for releasing a large dataset using RPM.
Can someone that understands this code please do a quick once over and
see if somebody made the obvious mistake of trying to stuff an
offset_t into an int?

This short demo doesn't show the full file sizes before it crashes,
but the real spec file with real data succeeded in printing out the
data file's size.  It was a negative number.  This seems like a hint
that the size of the file size variable was a bit too small for its
own good.

$ cat bigfile-crash-demo.spec

Buildroot: /tmp/bigfile-crash-demo

Summary: big datafile crash demo
Name: bigfile-crash-demo
Version: 0.1
Release: 1
Group: Applications/devel
BuildArch: noarch
Distribution: none
Packager: wolfgang
ExcludeOS: anything-from-redmond 
License: freebeer

demonstrate how rpmbuild rolls over and dies with big data files.
invode with "rpmbuild -vv -bb bigfile-crash-demo.spec"



rm -rf /tmp/bigfile-crash-demo
mkdir -p $RPM_BUILD_ROOT/bigfile-crash-demo
cd $RPM_BUILD_ROOT/bigfile-crash-demo && dd if=/dev/zero of=crashdemo.dat bs=1000000 count=7702

rm -rf /tmp/bigfile-crash-demo



$ time nice -20 rpmbuild -vv -bb bigfile-crash-demo.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.42125
+ umask 022
+ cd /usr/src/redhat/BUILD
+ export LANG
+ unset DISPLAY
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.42125
+ umask 022
+ cd /usr/src/redhat/BUILD
+ export LANG
+ unset DISPLAY
+ exit 0
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.42125
+ umask 022
+ cd /usr/src/redhat/BUILD
+ export LANG
+ unset DISPLAY
+ rm -rf /tmp/bigfile-crash-demo
+ mkdir -p /tmp/bigfile-crash-demo/bigfile-crash-demo
+ cd /tmp/bigfile-crash-demo/bigfile-crash-demo
+ dd if=/dev/zero of=crashdemo.dat bs=1000000 count=7702
7702+0 records in
7702+0 records out
+ /usr/lib/rpm/redhat/brp-compress
+ /usr/lib/rpm/redhat/brp-strip /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
Processing files: bigfile-crash-demo-0.1-1
D: /tmp/bigfile-crash-demo/bigfile-crash-demo: directory
D: /tmp/bigfile-crash-demo/bigfile-crash-demo/crashdemo.dat: data
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /tmp/bigfile-crash-demo
D:      execv(/usr/lib/rpm/check-files) pid 6372
D:      waitpid(6372) rc 6372 status 0
D: fini      040755  2 (   0,   0)         0 /bigfile-crash-demo 
Segmentation fault

real    9m16.665s
user    3m4.307s
sys     0m51.148s

Comment 1 Jeff Johnson 2005-08-25 16:56:27 UTC
Yup, rpm (and cpio used as payload format) do not handle files >2Gb.

Comment 2 John Mahowald 2005-10-03 04:10:34 UTC
Supposedly cpio >2 GB has been fixed with bug #145238. Might this be something
else, like check-files itself?

Comment 3 Paul Nasrat 2005-11-28 18:47:57 UTC
rpm uses it's own cpio implementation that does not support >2Gb

Comment 4 Jeff Johnson 2006-02-24 18:25:40 UTC
rpm-4.4.5 should handle up to 4 Gb files.

More than 4 Gb files will require data type changes within *.rpm packages.
That work is well underway, but will take time because of incomaptibilities.

DEFERRED until somewhen.

Note You need to log in before you can comment on or make changes to this bug.