Bug 1506231

Summary: Netbackup 7 agent is not started automatically upon recovery
Product: Red Hat Enterprise Linux 7 Reporter: Renaud Métrich <rmetrich>
Component: rearAssignee: Pavel Cahyna <pcahyna>
Status: CLOSED ERRATA QA Contact: Tereza Cerna <tcerna>
Severity: medium Docs Contact: Petr Bokoc <pbokoc>
Priority: urgent    
Version: 7.3CC: fkrska, ovasik, pbokoc, pcahyna, tbowling, tcerna, toneata
Target Milestone: rcKeywords: EasyFix, Patch, Reproducer, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rear-2.00-6.el7 Doc Type: Bug Fix
Doc Text:
*ReaR* no longer fails with `NetBackup` and has more reliable network configuration Previously, two problems in the startup procedure of the rescue system caused the *ReaR* (Relax-and-Recover) restore process to fail when using the `NetBackup` method. The system's init scripts were sourced instead of executed when used by *ReaR*. As a consequence, the `NetBackup` init script aborted the system-setup process. Additionally, processes created by the system setup were immediately terminated. This affected the "dhclient" tool as well, and in some cases caused an IP address conflict. With this update, both bugs have been fixed. As a result, *ReaR* works properly with the `NetBackup` method, and network configuration using DHCP is more reliable.
Story Points: ---
Clone Of:
: 1545597 (view as bug list) Environment:
Last Closed: 2018-04-10 18:43:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1545597    
Attachments:
Description Flags
Proposed 450_prepare_netbackup.sh replacement none

Description Renaud Métrich 2017-10-25 13:08:54 UTC
Created attachment 1343240 [details]
Proposed 450_prepare_netbackup.sh replacement

Description of problem:

When using NetBackup 7 for ReaR backup (BACKUP=NBU in /etc/rear/local.conf), the NetBackup initscript (/etc/init.d/netbackup) is automatically added to the ReaR rescue image as /etc/scripts/system-setup.d/90-netbackup.sh.

Upon recovery, the agent is *not* started automatically.
Also the /etc/scripts/system-setup aborts immediately after sourcing NetBackup initscript, causing auto-recovery to not work.

Version-Release number of selected component (if applicable):

rear-2.00-3.el7_4.x86_64


How reproducible:

Always


Steps to Reproduce:
1. Set up Rear to backup using NetBackup (no need to have NetBackup)

echo "BACKUP=NBU" >> /etc/rear/local.conf


2. Create fake NetBackup initscript

# cat > /etc/init.d/netbackup << EOF
#!/bin/bash

echo "This is Fake NetBackup"
exit 0
EOF

# mkdir -p /usr/openv/netbackup/bin/
# echo "NetBackup-RedHat2.6.18 7.6.0.1" > /usr/openv/netbackup/bin/version


3. Create rear rescue image

# rear mkrescue


4. Boot rescue image with "debug" kernel parameter


Actual results:

Just after running 90-netbackup.sh (after prompt "Press ENTER to run 90-netbackup.sh"), the script aborts, without running further scripts (in particular 99-makedev.sh).


Expected results:

99-makedev.sh runs (after prompt "Press ENTER to run 99-makedev.sh")


Additional info:

The root cause /etc/scripts/system-setup sourcing initscripts, instead of executing them. This is OK for other internal scripts, but not for 3rd-party initscripts ReaR doesn't control.
In particular, netbackup executes "exit XX" at its end, causing /etc/scripts/system-setup to exit.

Suggested fix is to modify /usr/share/rear/rescue/NBU/default/450_prepare_netbackup.sh so that the "/etc/scripts/system-setup.d/netbackup" script runs in a sub-shell, as shown in attachment:

- /etc/init.d/netbackup is copied as /etc/scripts/system-setup.d/netbackup.real
- /etc/scripts/system-setup.d/90-netbackup.sh is created to encapsulate /etc/scripts/system-setup.d/netbackup.real execution in a sub-shell

Comment 7 Tereza Cerna 2018-02-09 13:13:24 UTC
Hi, there is QE ack.

I was not able to reproduce this bug, so QE will provide:
  * run regression tests and check that there is no fail
  * check that patch is applied

Comment 13 Pavel Cahyna 2018-02-16 19:40:42 UTC
Notes for testing:
- I was able to reproduce the problem easily using Renaud's instructions above. My configuration file /etc/rear/local.conf:

BACKUP=NBU
OUTPUT_URL=file://
ISO_DEFAULT="automatic"
KERNEL_CMDLINE=debug

- The fix also corrects a problem with dhclient, as noted in upstream PR#1610. To test this: check that the dhclient process exists in the rescue system after booting from the rescue CD. (Before the fix: no dhclient process is there, after the fix: a dhclient process is running.)

Comment 14 Tereza Cerna 2018-02-20 16:49:33 UTC
==========================
Verified in:
    rear-2.00-6.el7.x86_64
PASS
==========================

  * patch was applied
  * script 99-makedev.sh
  * dhclient process is not killed

# cat /etc/rear/local.conf
BACKUP=NBU
OUTPUT_URL=file://
ISO_DEFAULT="automatic"
KERNEL_CMDLINE=debug

# rear -v mkrescue
...
'/etc/init.d/netbackup' -> '/tmp/rear.zzk53MjDnWFaNhZ/rootfs/etc/scripts/system-setup.d/netbackup.real'
mode of '/tmp/rear.zzk53MjDnWFaNhZ/rootfs/etc/scripts/system-setup.d/netbackup.real' changed from 0644 (rw-r--r--) to 0755 (rwxr-xr-x)
mode of '/tmp/rear.zzk53MjDnWFaNhZ/rootfs/etc/scripts/system-setup.d/90-netbackup.sh' changed from 0644 (rw-r--r--) to 0755 (rwxr-xr-x)
...

>> we can see that pach was applied

# reboot

>> Select Relax-and-Recover

>> see in serial console:

Press ENTER to run 90-netbackup.sh 
+ source /etc/scripts/system-setup.d/90-netbackup.sh
++ /etc/scripts/system-setup.d/netbackup.real
This is Fake NetBackup
+ set +x
Press ENTER to run 99-makedev.sh 
+ source /etc/scripts/system-setup.d/99-makedev.[  OK  ] Started Initialize Rescue System.
[  OK  ] Started Getty on tty2.
         Starting Getty on tty2...
[  OK  ] Started Getty on tty4.
         Starting Getty on tty4...
[  OK  ] Started Getty on tty1.
         Starting Getty on tty1...
[  OK  ] Started Getty on tty3.
         Starting Getty on tty3...
[  OK  ] Reached target Login Prompts.
[  OK  ] Reached target Multi-User.

>> Script 99-makedev.sh was run.

>> Ssh to machine (rescue mode)

# ps aux | grep dhclient
root       344  0.0  0.0 107376  2296 ?        Ss   11:43   0:00 dhclient -lf /var/lib/dhclient/dhclient.leases.em1 -pf /var/run/dhclient.pid -cf /etc/dhclient.conf em1
root       376  0.0  0.0   9088   660 pts/0    S+   11:43   0:00 grep dhclient

>> There is dhclient process.


============================
Reproduced in:
    rear-2.00-3.el7_4.x86_64
FAIL
============================

# cat /etc/rear/local.conf
BACKUP=NBU
OUTPUT_URL=file://
ISO_DEFAULT="automatic"
KERNEL_CMDLINE=debug

# rear -v mkrescue
...
'/etc/init.d/netbackup' -> '/tmp/rear.TQIzIIvGQ3ve7jB/rootfs/etc/scripts/system-setup.d/90-netbackup.sh'
mode of '/tmp/rear.TQIzIIvGQ3ve7jB/rootfs/etc/scripts/system-setup.d/90-netbackup.sh' changed from 0644 (rw-r--r--) to 0755 (rwxr-xr-x)
...

# reboot

>> Select Relax-and-Recover

>> See in serial console:

Press ENTER to run 90-netbackup.sh 
+ source /etc/sc[  OK  ] Started Initialize Rescue System.
[  OK  ] Started Getty on tty1.
         Starting Getty on tty1...
[  OK  ] Started Getty on tty2.
         Starting Getty on tty2...
[  OK  ] Started Getty on tty4.
         Starting Getty on tty4...
[  OK  ] Started Getty on tty3.
         Starting Getty on tty3...
[  OK  ] Reached target Login Prompts.
[  OK  ] Reached target Multi-User.

>> The script aborted, further scripts was not run

>> Ssh to machine (rescue mode)

# ps aux | grep dhclient
root       384  0.0  0.0   9088   664 pts/0    S+   11:08   0:00 grep dhclient

>> No dhclient process is there

Comment 17 errata-xmlrpc 2018-04-10 18:43:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1000

Comment 18 Red Hat Bugzilla 2023-09-15 00:04:46 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days