Red Hat Bugzilla – Bug 166029
rpmbuild crashes when rpm-ing up an 8 gig file
Last modified: 2007-11-30 17:11:11 EST
From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; OpenBSD amd64; en-US; rv:1.7.10) Gecko/20050724 Firefox/1.0.6 Description of problem: rpmbuild takes a long time and finally segv's when trying to create an rpm if one of the enclosed files is an 7.7GiB database file. Building an rpm from an essentially identical spec file that specifies a smaller 500MB database file succeeds. The only difference is the size of the enclosed file. Version-Release number of selected component (if applicable): rpm-build-4.4.1-22.x86_64 How reproducible: Always Steps to Reproduce: 1.generate a spec file to rpm up a data directory containing 8GB of data 2.rpmbuild -vvv -bb foo-data.spec 3.wait Actual Results: a long wait and rpmbuild says it segv-ed Expected Results: rpmbuild should produce an rpm file in /usr/share/redhat/RPMS/ Additional info: This is a show-stopper bug for releasing a large dataset using RPM. Can someone that understands this code please do a quick once over and see if somebody made the obvious mistake of trying to stuff an offset_t into an int? This short demo doesn't show the full file sizes before it crashes, but the real spec file with real data succeeded in printing out the data file's size. It was a negative number. This seems like a hint that the size of the file size variable was a bit too small for its own good. $ cat bigfile-crash-demo.spec Buildroot: /tmp/bigfile-crash-demo Summary: big datafile crash demo Name: bigfile-crash-demo Version: 0.1 Release: 1 Group: Applications/devel BuildArch: noarch Distribution: none Packager: wolfgang ExcludeOS: anything-from-redmond License: freebeer %description demonstrate how rpmbuild rolls over and dies with big data files. invode with "rpmbuild -vv -bb bigfile-crash-demo.spec" %prep %build %install rm -rf /tmp/bigfile-crash-demo mkdir -p $RPM_BUILD_ROOT/bigfile-crash-demo cd $RPM_BUILD_ROOT/bigfile-crash-demo && dd if=/dev/zero of=crashdemo.dat bs=1000000 count=7702 %clean rm -rf /tmp/bigfile-crash-demo %files /bigfile-crash-demo #end $ time nice -20 rpmbuild -vv -bb bigfile-crash-demo.spec Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.42125 + umask 022 + cd /usr/src/redhat/BUILD + LANG=C + export LANG + unset DISPLAY + exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.42125 + umask 022 + cd /usr/src/redhat/BUILD + LANG=C + export LANG + unset DISPLAY + exit 0 Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.42125 + umask 022 + cd /usr/src/redhat/BUILD + LANG=C + export LANG + unset DISPLAY + rm -rf /tmp/bigfile-crash-demo + mkdir -p /tmp/bigfile-crash-demo/bigfile-crash-demo + cd /tmp/bigfile-crash-demo/bigfile-crash-demo + dd if=/dev/zero of=crashdemo.dat bs=1000000 count=7702 7702+0 records in 7702+0 records out + /usr/lib/rpm/redhat/brp-compress + /usr/lib/rpm/redhat/brp-strip /usr/bin/strip + /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip + /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump Processing files: bigfile-crash-demo-0.1-1 D: /tmp/bigfile-crash-demo/bigfile-crash-demo: directory D: /tmp/bigfile-crash-demo/bigfile-crash-demo/crashdemo.dat: data Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 Checking for unpackaged file(s): /usr/lib/rpm/check-files /tmp/bigfile-crash-demo D: execv(/usr/lib/rpm/check-files) pid 6372 D: waitpid(6372) rc 6372 status 0 D: fini 040755 2 ( 0, 0) 0 /bigfile-crash-demo Segmentation fault real 9m16.665s user 3m4.307s sys 0m51.148s
Yup, rpm (and cpio used as payload format) do not handle files >2Gb.
Supposedly cpio >2 GB has been fixed with bug #145238. Might this be something else, like check-files itself?
rpm uses it's own cpio implementation that does not support >2Gb
rpm-4.4.5 should handle up to 4 Gb files. More than 4 Gb files will require data type changes within *.rpm packages. That work is well underway, but will take time because of incomaptibilities. DEFERRED until somewhen.