RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1693608 - Rear daily cron run fills temp directory upon failure
Summary: Rear daily cron run fills temp directory upon failure
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: rear
Version: 7.6
Hardware: All
OS: All
unspecified
high
Target Milestone: rc
: ---
Assignee: Pavel Cahyna
QA Contact: David Jež
Mariya Pershina
URL:
Whiteboard:
Depends On:
Blocks: 1122832 1716965 1729501
TreeView+ depends on / blocked
 
Reported: 2019-03-28 10:02 UTC by Oliver Falk
Modified: 2023-09-07 19:52 UTC (History)
6 users (show)

Fixed In Version: rear-2.4-11.el7
Doc Type: Bug Fix
Doc Text:
.`ReaR` updates RHEL 7.8 introduces a number of updates to the Relax-and-Recover (`ReaR`) utility. The build directory handling has been changed. Previously, the build directory was kept in a temporary location in case `ReaR` encountered a failure. With this update, the build directory is deleted by default in non-interactive runs to prevent consuming disk space. The semantics of the `KEEP_BUILD_DIR` configuration variable has been enhanced to include a new `errors` value. You can set the `KEEP_BUILD_DIR` variable to the following values: * `errors` to preserve the build directory on errors for debugging (the previous behavior) * `y` (`true`) to always preserve the build directory * `n` (`false`) to never preserve the build directory The default value is an empty string with the meaning of `errors` when `ReaR` is being executed interactively (in a terminal) and `false` if `ReaR` is being executed non-interactively. Note that `KEEP_BUILD_DIR` is automatically set to `true` in debug mode (`-d`) and in debugscript mode (`-D`); this behavior has not been changed. Notable bug fixes include: * Support for NetBackup 8.0 has been fixed. * `ReaR` no longer aborts with a bash error similar to `xrealloc: cannot allocate` on systems with a large number of users, groups, and users per group. * The `bconsole` command now shows its prompt, which enables you to perform a restore operation when using the Bacula integration. * `ReaR` now correctly backs up files also in situations when the `docker` service is running but no `docker` root directory has been defined, or when it is impossible to determine the status of the `docker` service. * Recovery no longer fails when using thin pools or recovering a system in Migration Mode. * Extremely slow rebuild of `initramfs` during the recovery process with LVM has been fixed.
Clone Of:
: 1729501 (view as bug list)
Environment:
Last Closed: 2020-03-31 20:06:59 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github /rear rear pull 2218 0 None None None 2020-06-11 15:55:09 UTC
Github rear rear issues 2121 0 'None' closed ReaR not deleting folders from /tmp 2021-02-17 00:55:56 UTC
Red Hat Product Errata RHBA-2020:1153 0 None None None 2020-03-31 20:07:39 UTC

Description Oliver Falk 2019-03-28 10:02:58 UTC 
Description of problem:
If rear fails, for whatever reasons, it leaves behind the temporary files/directories. This can easily fill up the disk space and lead to service interruptions => Therefore severity high.


Version-Release number of selected component (if applicable): 2.4


How reproducible: Always


Steps to Reproduce:
1. Install rear
2. Make sure rear fails (eg. make sure some binary is included that uses a library, that is not automatically included)
3. Run sudo rear mkrescue -v


Actual results:
Running mkrescue -v gives you the right hint already:

"""
You should also rm -Rf /var/tmp/rear.Vow2ZOUr9jUanAG
"""

While this is perfectly fine when running rear interactively, running it via cron should always clean up the temporary files.

Expected results:
Rear cron job shouldn't leave behind temporary directories, when not run interactively.


Additional info:
A probable workaround could be to source the rear config and remove $TMPDIR/rear.*. Eventually check if $TMPDIR is set before; Else TMPDIR => /tmp.

Customer case will be attached in a hurry.
Comment 3 Pavel Cahyna 2019-08-27 12:57:43 UTC 
Some notes:
- this behavior is intentional, ReaR leaves around the build directory for debugging in case of errors. See the upstream issue https://github.com/rear/rear/issues/2121 where a user complained about it and the "solution" was just to inform about it in a comment in the default configuration file.
- it should not be the case that daily cron runs will fill the temp directory. The crontab contains
/usr/sbin/rear checklayout || /usr/sbin/rear mkrescue
so the build directory will be left on first run, but on the subsequent ones checklayout will succeed and further build directories should not be produced, so the used space should not grow without bounds and the damage should be limited (unless the disk layout changes often for some reason).
- even given the above, I agree that the current behavior is unfortunate and I submitted PR2218 upstream to not leave the build directory around in noninteractive runs and warn more prominently about it (without the -v flag) in interactive runs.
Comment 4 David Jež 2019-08-29 09:56:06 UTC 
Pavle,
Let's add a new option KEEP_BUILD_DIR_ON_ERRORS. If set to yes, keep build dir on errors even in non interactive mode, which is really useful for troubleshoot. If empty, fill KEEP_BUILD_DIR based on tty. Only KEEP_BUILD_DIR itself could not be enough because it always remove directory or always keep it which is not so useful for troubleshoot.

Thank you,
DJ
Comment 5 Pavel Cahyna 2019-09-02 15:42:22 UTC 
Instead of introducing a new variable KEEP_BUILD_DIR_ON_ERRORS, we plan with upstream to introduce a new possible value for KEEP_BUILD_DIR, KEEP_BUILD_DIR="errors", which will have the same semantics as the proposed KEEP_BUILD_DIR_ON_ERRORS=yes. See https://github.com/rear/rear/pull/2218#issuecomment-526679705
Comment 13 Pavel Cahyna 2019-09-13 12:44:04 UTC 
PR merged upstream without modification.
Comment 22 errata-xmlrpc 2020-03-31 20:06:59 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1153
Note You need to log in before you can comment on or make changes to this bug.