Bug 2021935 - RHEL 8.5 /etc/cron.d/rear creates output & email with STDERR
Summary: RHEL 8.5 /etc/cron.d/rear creates output & email with STDERR
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: rear
Version: 8.5
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Pavel Cahyna
QA Contact: David Jež
Šárka Jana
URL:
Whiteboard:
Depends On: 2119501
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-11-10 13:06 UTC by James Hartsock
Modified: 2022-11-08 11:09 UTC (History)
5 users (show)

Fixed In Version: rear-2.6-7.el8
Doc Type: Bug Fix
Doc Text:
.ReaR no longer displays a false error message about missing symlink targets Previously, ReaR displayed incorrect error messages about missing symlink targets for the `build` and `source` symlinks under `/usr/lib/modules/` when creating the rescue image. This situation was harmless, and you could safely ignore the error message. With this update, ReaR does not report a false error message about missing symlink targets in this situation.
Clone Of:
: 2119501 (view as bug list)
Environment:
Last Closed: 2022-11-08 10:02:43 UTC
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github rear rear issues 2739 0 None closed Fails to copy all kernel modules if there are dangling symlinks in /lib/modules/... 2022-03-23 15:53:34 UTC
Red Hat Issue Tracker RHELPLAN-102406 0 None None None 2021-11-10 18:19:28 UTC
Red Hat Knowledge Base (Solution) 6895811 0 None None None 2022-06-14 06:22:13 UTC
Red Hat Product Errata RHBA-2022:7654 0 None None None 2022-11-08 10:02:50 UTC

Description James Hartsock 2021-11-10 13:06:21 UTC
Description of problem:
RHEL 8.5 /etc/cron.d/rear creates output & email with STDERR

Version-Release number of selected component (if applicable):
rear-2.6-3.el8.x86_64


How reproducible:
100%

Steps to Reproduce:
1. Install rear
2. wait for cron or run manullay
3.

Actual results:
# cat /etc/cron.d/rear
30 1 * * * root test -f /var/lib/rear/layout/disklayout.conf && /usr/sbin/rear checklayout || /usr/sbin/rear mkrescue

# test -f /var/lib/rear/layout/disklayout.conf && /usr/sbin/rear checklayout || /usr/sbin/rear mkrescue
updatedb
Symlink '/usr/lib/modules/4.18.0-348.el8.x86_64/source' -> '/usr/src/kernels/4.18.0-348.el8.x86_64' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/4.18.0-348.el8.x86_64' via the 'COPY_AS_IS' configuration variable.
Symlink '/usr/lib/modules/4.18.0-348.el8.x86_64/build' -> '/usr/src/kernels/4.18.0-348.el8.x86_64' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/4.18.0-348.el8.x86_64' via the 'COPY_AS_IS' configuration variable.


Expected results:

# test -f /var/lib/rear/layout/disklayout.conf && /usr/sbin/rear checklayout || /usr/sbin/rear mkrescue
#

Additional info:

# ll -d /usr/lib/modules/4.18.0-348.el8.x86_64/{build,source} /usr/src/kernels/4.18.0-348.el8.x86_64
lrwxrwxrwx.  1 root root   38 Oct  4 11:42 /usr/lib/modules/4.18.0-348.el8.x86_64/build -> /usr/src/kernels/4.18.0-348.el8.x86_64
lrwxrwxrwx.  1 root root    5 Oct  4 11:42 /usr/lib/modules/4.18.0-348.el8.x86_64/source -> build
drwxr-xr-x. 23 root root 4096 Nov  9 15:03 /usr/src/kernels/4.18.0-348.el8.x86_64

Comment 1 James Hartsock 2021-11-10 13:09:22 UTC
So probably need kernel-devel RPM installed to reproduce too

# rpm -qf /usr/lib/modules/4.18.0-348.el8.x86_64/{build,source} /usr/src/kernels/4.18.0-348.el8.x86_64
kernel-core-4.18.0-348.el8.x86_64
kernel-core-4.18.0-348.el8.x86_64
kernel-devel-4.18.0-348.el8.x86_64

Comment 2 James Hartsock 2021-11-10 13:14:35 UTC
From log...
2021-11-10 07:02:40.343886684 Including build/default/490_fix_broken_links.sh
/tmp/rear.xYg0IJsajKexszh/rootfs /usr/lib/modules
2021-11-10 07:02:40.414246180 Symlink '/usr/lib/modules/4.18.0-348.el8.x86_64/source' -> '/usr/src/kernels/4.18.0-348.el8.x86_64' refers to a non-existing directory on the recovery system.
2021-11-10 07:02:40.415361204 It will not be copied by default. You can include '/usr/src/kernels/4.18.0-348.el8.x86_64' via the 'COPY_AS_IS' configuration variable.
2021-11-10 07:02:40.417487261 Symlink '/usr/lib/modules/4.18.0-348.el8.x86_64/build' -> '/usr/src/kernels/4.18.0-348.el8.x86_64' refers to a non-existing directory on the recovery system.
2021-11-10 07:02:40.418621347 It will not be copied by default. You can include '/usr/src/kernels/4.18.0-348.el8.x86_64' via the 'COPY_AS_IS' configuration variable.

Comment 3 James Hartsock 2021-11-10 14:06:17 UTC
Perhaps should include  /usr/src/kernels/$KERNEL_VERSION/ in...

# tail -n 6 /usr/share/rear/rescue/GNU/Linux/240_kernel_modules.sh
COPY_AS_IS+=(
    /lib/modules/$KERNEL_VERSION/modules.*
    /etc/modules*
    /etc/modules-load?d
    /etc/modprobe*
)

---

Or maybe better yet exclude those links.  At first, thought the COPY_AS_IS_EXCLUDE+=, but that isn't what includes it from my view.
Appears this is from the (a default since ReaR 2.5 per build/GNU/Linux/400_copy_modules.sh):
  MODULES=( 'loaded_modules' )

# echo "MODULES=( 'loaded_modules' )" >> /etc/rear/local.conf 
# rm -rf /var/lib/rear/{layout,recovery}/* /var/log/rear/*
# test -f /var/lib/rear/layout/disklayout.conf && /usr/sbin/rear checklayout || /usr/sbin/rear mkrescue
#


So looks like this is introduced with https://access.redhat.com/errata/RHEA-2021:4344 that updated ReaR to 2.6 (from 2.4 version)
  rhel8.4/AppStream/Packages/rear-2.4-18.el8.x86_64.rpm
  rhel8.5/AppStream/Packages/rear-2.6-3.el8.x86_64.rpm

Comment 4 James Hartsock 2021-11-10 15:17:15 UTC
(In reply to James Hartsock from comment #3)
> Perhaps should include  /usr/src/kernels/$KERNEL_VERSION/ in...
> 
> # tail -n 6 /usr/share/rear/rescue/GNU/Linux/240_kernel_modules.sh
> COPY_AS_IS+=(
>     /lib/modules/$KERNEL_VERSION/modules.*
>     /etc/modules*
>     /etc/modules-load?d
>     /etc/modprobe*
> )
> 
> ---
> 
> Or maybe better yet exclude those links.  At first, thought the
> COPY_AS_IS_EXCLUDE+=, but that isn't what includes it from my view.
> Appears this is from the (a default since ReaR 2.5 per
> build/GNU/Linux/400_copy_modules.sh):
>   MODULES=( 'loaded_modules' )

Correction, default since 2.5 is 'all_modules' ... and I tried 'loaded_modules' as work-around.
# grep -m1 'all_modules' build/GNU/Linux/400_copy_modules.sh
    # Since ReaR version 2.5 we have MODULES=( 'all_modules' ) in default.conf


Sorry about that copy/paste mistake.

Comment 5 Pavel Cahyna 2021-11-10 18:29:03 UTC
Hello, the bug report is not really about the cron job, right? The actual problem is the newly appearing warning. The cron job is just what triggered it in your case.

Comment 7 Pavel Cahyna 2022-03-23 15:53:34 UTC
There is an upstream issue which is closed, but the fix is more a mitigation than a complete solution. We could backport it if not better solution is found.

Comment 8 James Hartsock 2022-03-23 16:15:03 UTC
So (In reply to Pavel Cahyna from comment #5)
> Hello, the bug report is not really about the cron job, right? The actual
> problem is the newly appearing warning. The cron job is just what triggered
> it in your case.

The issue gets noticed raised by customers because they monitor e-mail from cron jobs which create incidents.

So yes, ideally fix the error/warning the command produces is best because simply redirecting cron output or something masks not only this problem but any others that may be legitimate issues.

Comment 13 Pavel Cahyna 2022-08-17 10:17:53 UTC
For the record, the problem can show up in two ways. If package providing the symlink target (kernel-debug-devel or kernel-devel, matching the kernel version) are not installed, one gets messages like

Broken symlink '/usr/lib/modules/4.18.0-416.el8.x86_64/build' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/modules/4.18.0-416.el8.x86_64/source' in recovery system because 'readlink' cannot determine its link target

If the target exist, one gets messages like the one in the original report:

Symlink '/usr/lib/modules/4.18.0-416.el8.x86_64/build' -> '/usr/src/kernels/4.18.0-416.el8.x86_64' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/4.18.0-416.el8.x86_64' via the 'COPY_AS_IS' configuration variable.
Symlink '/usr/lib/modules/4.18.0-416.el8.x86_64/source' -> '/usr/src/kernels/4.18.0-416.el8.x86_64' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/4.18.0-416.el8.x86_64' via the 'COPY_AS_IS' configuration variable.

Comment 25 errata-xmlrpc 2022-11-08 10:02:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (rear bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:7654


Note You need to log in before you can comment on or make changes to this bug.