Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
DescriptionMilan Kerslager
2017-03-09 10:24:49 UTC
Description of problem:
Sometimes, probably after Bacula director restart, storage daemon hangs and director is waiting indefinitely. So backup not run.
Version-Release number of selected component (if applicable):
bacula-storage-5.2.13-23.1.el7.x86_64
How reproducible:
Change configuration, restart director multiple times. Or restart machine and try to run job. It mostly works, hangs sometimes.
Steps to Reproduce:
1. run job
2. check status: echo status dir | bconsole | grep "waiting on Storage"
3. restart storage daemon, job will run automagically
Actual results:
Baucla is waiting on SD to be ready even SD is running.
Expected results:
Storage daemon recover itself.
Additional info:
strace on storage daemon when hanged, process list, systemctl status
[root@bacula ~]# ps xa | grep bacula-sd
bacula 784 0.0 0.2 407632 4636 ? Ssl 10:15 0:00 /usr/sbin/bacula-sd -f -c /etc/bacula/bacula-sd.conf -u bacula -g tape
[root@bacula ~]# strace -p 784
Process 784 attached
select(4, [3], NULL, NULL, NULL^CProcess 784 detached
<detached ...>
[root@bacula ~]# ll /proc/784/fd
celkem 0
lr-x------. 1 root root 64 9. bře 10.15 0 -> /dev/null
lrwx------. 1 root root 64 9. bře 10.15 1 -> socket:[14708]
lrwx------. 1 root root 64 9. bře 10.15 2 -> socket:[14708]
lrwx------. 1 root root 64 9. bře 10.15 3 -> socket:[15186]
[root@bacula ~]# systemctl status bacula-sd.service
● bacula-sd.service - Bacula-StorageDaemon, the storage-server
Loaded: loaded (/usr/lib/systemd/system/bacula-sd.service; enabled; vendor preset: disabled)
Active: active (running) since Čt 2017-03-09 10:15:27 CET; 6min ago
Docs: man:bacula-sd(8)
Main PID: 784 (bacula-sd)
CGroup: /system.slice/bacula-sd.service
└─784 /usr/sbin/bacula-sd -f -c /etc/bacula/bacula-sd.conf -u bacula -g tape
bře 09 10:15:27 bacula.pslib.cz systemd[1]: Started Bacula-StorageDaemon, the storage-server.
bře 09 10:15:27 bacula.pslib.cz systemd[1]: Starting Bacula-StorageDaemon, the storage-server...
Workaroud: Use this script which check director status and restart storage daemon only if there is no other job running. Run it periodically by cron.
#!/bin/bash
if echo status dir | /usr/sbin/bconsole | grep -q "waiting on Storage"; then
if ! echo status dir | /usr/sbin/bconsole | grep -q "is running"; then
/usr/bin/systemctl restart bacula-sd.service
echo "systemctl restart bacula-sd.service"
fi
fi
Red Hat Enterprise Linux version 7 entered the Maintenance Support 1 Phase in August 2019. In this phase only qualified Critical and Important Security errata advisories (RHSAs) and Urgent Priority Bug Fix errata advisories (RHBAs) may be released as they become available. Other errata advisories may be delivered as appropriate.
This bug has been reviewed by Support and Engineering representative and does not meet the inclusion criteria for Maintenance Support 1 Phase. If this issue still exists in newer major version of Red Hat Enterprise Linux, it has been cloned there and work will continue in the cloned bug.
For more information about Red Hat Enterprise Linux Lifecycle, please see https://access.redhat.com/support/policy/updates/errata/
Description of problem: Sometimes, probably after Bacula director restart, storage daemon hangs and director is waiting indefinitely. So backup not run. Version-Release number of selected component (if applicable): bacula-storage-5.2.13-23.1.el7.x86_64 How reproducible: Change configuration, restart director multiple times. Or restart machine and try to run job. It mostly works, hangs sometimes. Steps to Reproduce: 1. run job 2. check status: echo status dir | bconsole | grep "waiting on Storage" 3. restart storage daemon, job will run automagically Actual results: Baucla is waiting on SD to be ready even SD is running. Expected results: Storage daemon recover itself. Additional info: strace on storage daemon when hanged, process list, systemctl status [root@bacula ~]# ps xa | grep bacula-sd bacula 784 0.0 0.2 407632 4636 ? Ssl 10:15 0:00 /usr/sbin/bacula-sd -f -c /etc/bacula/bacula-sd.conf -u bacula -g tape [root@bacula ~]# strace -p 784 Process 784 attached select(4, [3], NULL, NULL, NULL^CProcess 784 detached <detached ...> [root@bacula ~]# ll /proc/784/fd celkem 0 lr-x------. 1 root root 64 9. bře 10.15 0 -> /dev/null lrwx------. 1 root root 64 9. bře 10.15 1 -> socket:[14708] lrwx------. 1 root root 64 9. bře 10.15 2 -> socket:[14708] lrwx------. 1 root root 64 9. bře 10.15 3 -> socket:[15186] [root@bacula ~]# systemctl status bacula-sd.service ● bacula-sd.service - Bacula-StorageDaemon, the storage-server Loaded: loaded (/usr/lib/systemd/system/bacula-sd.service; enabled; vendor preset: disabled) Active: active (running) since Čt 2017-03-09 10:15:27 CET; 6min ago Docs: man:bacula-sd(8) Main PID: 784 (bacula-sd) CGroup: /system.slice/bacula-sd.service └─784 /usr/sbin/bacula-sd -f -c /etc/bacula/bacula-sd.conf -u bacula -g tape bře 09 10:15:27 bacula.pslib.cz systemd[1]: Started Bacula-StorageDaemon, the storage-server. bře 09 10:15:27 bacula.pslib.cz systemd[1]: Starting Bacula-StorageDaemon, the storage-server... Workaroud: Use this script which check director status and restart storage daemon only if there is no other job running. Run it periodically by cron. #!/bin/bash if echo status dir | /usr/sbin/bconsole | grep -q "waiting on Storage"; then if ! echo status dir | /usr/sbin/bconsole | grep -q "is running"; then /usr/bin/systemctl restart bacula-sd.service echo "systemctl restart bacula-sd.service" fi fi