Description of problem: when asked to package a huge directory tree that contains lots of files with the same name, rpmbuild will eventually segfault in the "Processing files" stage. We have a test case, which gives: rpmbuild -ba .../ ... + cd /var/tmp/test-root + python /var/tmp/test-root/createDirs.py Processing files: test-0-0 Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 182894245600 (LWP 1567)] 0x000000342142e21c in compressFilelist (h=0x527120) at legacy.c:355 355 baseName = strrchr(fileNames[i], '/') + 1; (gdb) bt #0 0x000000342142e21c in compressFilelist (h=0x527120) at legacy.c:355 #1 0x0000003421a0c1f6 in genCpioListAndHeader (fl=0x7fbfffb370, fip=0x518e70, h=0x527120, isSrc=0) at files.c:1342 #2 0x0000003421a0e500 in processBinaryFiles (spec=0x5282a0, installSpecialDoc=4, test=0) at files.c:2156 #3 0x0000003421a08a5c in buildSpec (ts=0x527e90, spec=0x5282a0, what=223, test=0) at build.c:349 #4 0x0000000000402e18 in ?? () #5 0x00000000004035b1 in ?? () #6 0x0000000000404157 in ?? () #7 0x000000342071c3fb in __libc_start_main () from /lib64/tls/libc.so.6 #8 0x0000000000402ada in ?? () #9 0x0000007fbffff938 in ?? () #10 0x000000000000001c in ?? () #11 0x0000000000000004 in ?? () #12 0x0000007fbffffb61 in ?? () #13 0x0000007fbffffb73 in ?? () #14 0x0000007fbffffb77 in ?? () #15 0x0000007fbffffb7b in ?? () #16 0x0000000000000000 in ?? () (gdb) print i $1 = 0 (gdb) print fileNames $2 = (char **) 0x2aace97010 (gdb) print fileNames[i] $3 = 0x2aadcdd010 "/src" (gdb) Notes: * memory usage increases slowly while the program runs, to several 100MB * trying to trigger the issue via a lower "ulimit -v" at 100MB steps: 100-400MB are handled gracefully, 500MB shows the issue but brings no significant speedup. * symbols above are as on RHEL4.6/x86_64, rpm-build-4.3.3-23_nonptl * The use of python in the .spec is just to speed up directory creation, the bug equally happens on a prepopulated tree, or via shell. Directory tree deletion/creation of course takes ages, since on ext3... * this actually gets triggered in production for one of our experiments build system, where the use of same-name files cannot be avoided easily. Version-Release number of selected component (if applicable): rpm-build-4.3.3-23_nonptl How reproducible: 100% Steps to Reproduce: 1. see attached spec file 2. 3. Actual results: Segfault Expected results: No segfault Additional info: https://lists.dulug.duke.edu/pipermail/rpm-maint/2007-October/000553.html
Created attachment 298882 [details] reproducer
Yup, fixed upstream already: https://lists.dulug.duke.edu/pipermail/rpm-maint/2007-December/000652.html Easy to backport, and needed for RHEL 5 too.
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux maintenance release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Update release for currently deployed products. This request is not yet committed for inclusion in an Update release.
Fix for this issue is now applied in CVS.
Hello, old version segfaulting like in Comment #0 but new version doesn't work neither. It has something to common with bug: Bug 462539 - Unable to create immutable header region. When building 110MB rpm from simple tgz --This^ bug couldn't be fixed, bug 462539#c6 I used reproducer spec file: # rpmbuild -ba huge-dir-tree.spec Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.41642 + umask 022 + cd /usr/src/redhat/BUILD + LANG=C + export LANG + unset DISPLAY + exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.41642 + umask 022 + cd /usr/src/redhat/BUILD + LANG=C + export LANG + unset DISPLAY + exit 0 Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.41642 + umask 022 + cd /usr/src/redhat/BUILD + LANG=C + export LANG + unset DISPLAY + rm -rf /var/tmp/huge-dir-tree-root + '[' '!' -d /var/tmp/huge-dir-tree-root/ ']' + mkdir -p /var/tmp/huge-dir-tree-root/ + cat + cd /var/tmp/huge-dir-tree-root + python /var/tmp/huge-dir-tree-root/createDirs.py Processing files: huge-dir-tree-0-0 Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/huge-dir-tree-root Wrote: /usr/src/redhat/SRPMS/huge-dir-tree-0-0.src.rpm error: Unable to create immutable header region. RPM build errors: Unable to create immutable header region. --------------- what's the fix for this bug: not segfaulting but another issue or successful built - which couldn't be done ?
I was able to built hugedirtree with lower amount of files - 130 000, but then I wasn't able to uninstalled that package, filled a bug 483235
Is it possible to test it by customer? Please can you provide customer latest packages for RHEL4.8?
The customer has been provided with test packages and has provided the following feedback: I confirm that rpm-4.3.3-32 addresses our reproducer. However, the new "rpm-devel-4.3.3-32_nonptl.i386" RPM has picked up two new dependencies, on beecrypt-devel and elfutils-libelf-devel - these might be packaging errors (why would the executable depend on some header files?)
The executable doesn't require headers, and the executable also doesn't come from rpm-devel package (whose dependencies are correct now) but rpm-build package :)
Hello Jan, You wrote: > I confirm that rpm-4.3.3-32 addresses our reproducer. That's good to know. Thank you for your feedback. > However, the new "rpm-devel-4.3.3-32_nonptl.i386" RPM has picked up two new > dependencies, on beecrypt-devel and elfutils-libelf-devel - these might be > packaging errors (why would the executable depend on some header files?) Erm, the rpm executable is in the plain "rpm" package (not in "rpm-devel"), and that package does not depend on any -devel packages. As for the rpm-devel package depending on other -devel packages, there's a change here indeed. Having a -devel package depending on another -devel package isn't surprising to me (after all, in order to use the devel package to build code, you generally need to have the headers of libraries it depends on installed), but perhaps that's my Debian bias talking - I haven't done enough RPM building to know what the consensus is in RPM land. Are you OK with writing this off as an artefact of the test packages, or do you want me to dig deeper here? Kind regards, Ray Category set to: Devel Tools Internal Status set to 'Waiting on Customer' Status set to: Waiting on Client Interim resolution set to 'Custom Package' This event sent from IssueTracker by rdassen issue 141055
~~ Attention Partners! ~~ RHEL 4.8Beta has been released on partners.redhat.com. There should be a fix present, which addresses this bug. Please test and report back results on this OtherQA Partner bug at your earliest convenience. If you encounter any issues, please set the bug back to the ASSIGNED state and describe any issues you encountered. If you have found a NEW bug, clone this bug and describe the issues you've encountered. Further questions can be directed to your Red Hat Partner Manager. If you have VERIFIED the bug fix. Please select your PartnerID from the Verified field above. Please leave a comment with your test results details. Include which arches tested, package version and any applicable logs. - Red Hat QE Partner Management
Setting to verified based on Customer Verification results in comment #10. If there are any additional issues that need to be addressed, please clone this bug and make a new request. If it is found that this bug has not really been resolved, please reset to ASSIGNED state and describe the issues you are encountering.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2009-0951.html