Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Created attachment 1692701[details]
Trivial SPEC file illustrating the problem.
Description of problem:
Trying to build an RPM on an NFS filesystem fails with "rm" complaining a directory is not empty.
Version-Release number of selected component (if applicable):
rpm-build-4.14.2-37.el8.x86_64
How reproducible:
Every time
Steps to Reproduce:
1. Copy the attached trivial SPEC file
2. rpmbuild apa.spec
Actual results:
A build ending in
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.ju6DaJ
+ umask 022
+ cd /users/atlas/rpmbuild/BUILD
+ /usr/bin/rm -rf /users/atlas/rpmbuild/BUILDROOT/apa-1-1.x86_64
/usr/bin/rm: cannot remove '/users/atlas/rpmbuild/BUILDROOT/apa-1-1.x86_64/usr/bin': Directory not empty
error: Bad exit status from /var/tmp/rpm-tmp.ju6DaJ (%clean)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.ju6DaJ (%clean)
Expected results:
Successful build
Additional info:
The user running the build has the home directory "/users/atlas" on an NFS file system. If I define "_topdir" to be on some local file system the build succeeds.
By adding an "lsof" command to the %clean section I see "rpmbuild" has the file "apa" still mapped into it's memory when the cleanup commands are run. I believe this explains the behaviour. When the last file system reference to a file is removed on an NFS file system, but the file is still open by some process (on the same client), the file entry is renamed rather than removed until the process dies. That is why "rm" can't remove the directory. Later when "rpmbuild" terminates, the OS will remove the .nfs* file, but by then the build has already failed.
I can reproduce this issue and found the culprit.
It turns out rpmbuild is processing binary files with libelf for selecting buildids (for debuginfo the packages). If the file is not actually an elf binary the libelf handle is not closed properly. This is not an issue in most cases as the file is only read and the resources are freed when rpmbuild finishes. Unfortunately it keeps the file mmap'ed up to the point when %clean is executed as described above- with the results described above.
Moving the elf_free call out of the if block solves the issue for the test case and probably in general.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (rpm bug fix and enhancement update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2021:1606
Created attachment 1692701 [details] Trivial SPEC file illustrating the problem. Description of problem: Trying to build an RPM on an NFS filesystem fails with "rm" complaining a directory is not empty. Version-Release number of selected component (if applicable): rpm-build-4.14.2-37.el8.x86_64 How reproducible: Every time Steps to Reproduce: 1. Copy the attached trivial SPEC file 2. rpmbuild apa.spec Actual results: A build ending in Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.ju6DaJ + umask 022 + cd /users/atlas/rpmbuild/BUILD + /usr/bin/rm -rf /users/atlas/rpmbuild/BUILDROOT/apa-1-1.x86_64 /usr/bin/rm: cannot remove '/users/atlas/rpmbuild/BUILDROOT/apa-1-1.x86_64/usr/bin': Directory not empty error: Bad exit status from /var/tmp/rpm-tmp.ju6DaJ (%clean) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.ju6DaJ (%clean) Expected results: Successful build Additional info: The user running the build has the home directory "/users/atlas" on an NFS file system. If I define "_topdir" to be on some local file system the build succeeds. By adding an "lsof" command to the %clean section I see "rpmbuild" has the file "apa" still mapped into it's memory when the cleanup commands are run. I believe this explains the behaviour. When the last file system reference to a file is removed on an NFS file system, but the file is still open by some process (on the same client), the file entry is renamed rather than removed until the process dies. That is why "rm" can't remove the directory. Later when "rpmbuild" terminates, the OS will remove the .nfs* file, but by then the build has already failed.