Bug 799167

Summary: service start/status hang on sysv init classics
Product: [Fedora] Fedora Reporter: John Florian <jflorian>
Component: bacula2Assignee: Jon Ciesla <limburgher>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 16CC: cpanceac, johannbg, limburgher, metherid, mschmidt, notting, plautrba, systemd-maint
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: bacula2-2.4.4-11.fc17 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-04-11 22:44:08 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:

Description John Florian 2012-03-01 19:51:37 EST
Description of problem:
I'm seeing puppet struggle with several services (e.g., bacula2-fd, puppet) when it tests to see if a service is enabled or currently running and when starting them.

Version-Release number of selected component (if applicable):
systemd-37-13.fc16.x86_64
bacula2-client-2.4.4-7.fc15.x86_64
puppet-2.6.13-2.fc16.noarch

How reproducible:
nearly always

Steps to Reproduce:

# systemctl start bacula2-fd.service 

Actual results:

Looks hung; hit ctrl-c.  ps shows that the daemon started promptly, but systemd can't tell:

^C
#  ps ax | grep bacula
 4060 ?        Ssl    1:11 bacula2-fd -c /etc/bacula2/bacula-fd.conf
# systemctl status bacula2-fd.service 
bacula2-fd.service - SYSV: bacula2-fd is a Backup-client, which is the program that enables the bacula-server to backup the local machine.
          Loaded: loaded (/etc/rc.d/init.d/bacula2-fd)
          Active: activating (start) since Thu, 01 Mar 2012 19:43:49 -0500; 25s ago
         Process: 1145 ExecStart=/etc/rc.d/init.d/bacula2-fd start (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/bacula2-fd.service
                  └ 4060 bacula2-fd -c /etc/bacula2/bacula-fd.con



Puppet will eventually time out on the service, but it makes developing puppet manifests a PITA.


Expected results:


Additional info:
Comment 1 John Florian 2012-03-01 19:53:34 EST
I think something similar has bit others, perhaps in more specific ways: #789683, #787366.
Comment 2 Michal Schmidt 2012-03-01 20:06:49 EST
systemd is sensitive to wrong "# pidfile:" headers in initscripts (and wrong PIDFile= stanzas in systemd unit files).
I suspect this is the case with the bacula2-fd initscript.

See bug 786625 for a similar problem in a different package and a more detailed explanation.
Comment 3 Jon Ciesla 2012-03-02 12:29:22 EST
John, what's the correct PIDFile, /run/bacula2-fd.9102.pid?
Comment 4 John Florian 2012-03-02 19:14:24 EST
Exactly guys, sorry I should have thought of that.

= Bacula =

== Header ==
# pidfile: /var/run/bacula2-fd.9102.pid

== In Use ==
           /var/run/bacula-fd.9102.pid

Fedora ships two major versions so I'd say the header is most likely correct and use is wrong.  Conceivably someone might want both versions on the same system, especially if they're migrating  ;-)


= Puppet =

== Header ==
Has no declaration of pidfile in the header, but it does have:
pidfile=${PIDFILE-/var/run/puppet/agent.pid}

== In Use ==
/var/run/puppet/agent.pid

So where does systemd look for a pidfile on a traditional script that doesn't have the pidfile header entry?
Comment 5 Michal Schmidt 2012-03-05 18:28:58 EST
(In reply to comment #4)
> So where does systemd look for a pidfile on a traditional script that doesn't
> have the pidfile header entry?

If there's no pidfile header in the initscript, systemd does not do anything to detect the main PID of the service.

It will also treat the service like no-daemon-spawning initscripts (such as iptables) are treated - the service will have "RemainAfterExit=yes" enabled, which has the effect that when all processes of the service exit, the service will enter the "active (exited)" state rather than the usual "inactive (dead)". This may be confusing, so declaring pidfiles for actual daemons is recommended.
Comment 6 Jon Ciesla 2012-03-06 12:49:24 EST
Ok, I can fix the unit file easily enough for f18 and f17, but f16 is still using the initscript.  My sysv is very poor, how is that best corrected?
Comment 7 Fedora Update System 2012-03-06 13:01:25 EST
bacula2-2.4.4-10.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/bacula2-2.4.4-10.fc17
Comment 8 Fedora Update System 2012-03-07 02:21:40 EST
Package bacula2-2.4.4-10.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing bacula2-2.4.4-10.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-3169/bacula2-2.4.4-10.fc17
then log in and leave karma (feedback).
Comment 9 Michal Schmidt 2012-03-07 07:12:01 EST
(In reply to comment #6)
> Ok, I can fix the unit file easily enough for f18 and f17, but f16 is still
> using the initscript.  My sysv is very poor, how is that best corrected?

Just correct the "# pidfile:" header in /etc/init.d/bacula2-fd to reflect the reality (the discrepancy was noted in comment #4):
> == Header ==
> # pidfile: /var/run/bacula2-fd.9102.pid
> 
> == In Use ==
>            /var/run/bacula-fd.9102.pid
Comment 10 Jon Ciesla 2012-03-12 12:05:26 EDT
The init header in git f16 matches the pidfile I just set in the unit file for 18 and 17:

[limb@bamboo bacula2]$ grep pid bacula2-fd.init 
# pidfile: /var/run/bacula2-fd.9102.pid
Comment 11 Michal Schmidt 2012-03-12 12:23:21 EDT
(In reply to comment #10)

On my F16 system, while attempting to start the bacula2-fd service:

$ ls /var/run/bacu*
/var/run/bacula-fd.9102.pid
Comment 12 Jon Ciesla 2012-03-12 12:32:23 EDT
What does /etc/init.d/bacula2-fd say?
Comment 13 Michal Schmidt 2012-03-12 12:39:38 EDT
/etc/init.d/bacula2-fd has:
# pidfile: /var/run/bacula2-fd.9102.pid

(unmodified, straight from the bacula2-2.4.4-7.fc15 package)
Comment 14 Jon Ciesla 2012-03-12 12:56:48 EDT
So it looks like it's ignoring the pidfile entry, and possibly generating the pidfile name from the /etc/bacula2/bacula-fd.conf.  Try changing Name in FileDaemon to bacula2-fd.
Comment 15 Michal Schmidt 2012-03-12 13:35:13 EDT
(In reply to comment #14)
> Try changing Name in FileDaemon to bacula2-fd.

It does not have any effect on the pid file name.

Looking into the source...
The pid file name seems to be hardcoded in bacula-2.4.4/src/filed/filed.c@203:

   create_pid_file(me->pid_directory, "bacula-fd", get_first_port_host_order(me->FDaddrs));

... which makes me think whether you really fixed a bug in f17, or rather introduced one. Did anyone test if the service starts in f17 after the update?
Comment 16 Jon Ciesla 2012-03-12 13:40:11 EDT
Yes, please verify that, if someone has an f17 instance.  I may need to patch that file, and apply to all branches.
Comment 17 Fedora Update System 2012-03-13 14:44:46 EDT
bacula2-2.4.4-10.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 18 Michal Schmidt 2012-03-18 05:20:50 EDT
Testing in F17:

bacula2-2.4.4-9.fc17 starts fine.
bacula2-2.4.4-10.fc17 is broken.

# ls /run/bacu*
/run/bacula-fd.9102.pid
Comment 19 Jon Ciesla 2012-03-19 11:43:51 EDT
Ok, a patched update is coming.
Comment 20 Fedora Update System 2012-03-19 11:44:36 EDT
bacula2-2.4.4-11.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/bacula2-2.4.4-11.fc17
Comment 21 Fedora Update System 2012-03-20 02:04:59 EDT
Package bacula2-2.4.4-11.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing bacula2-2.4.4-11.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-4221/bacula2-2.4.4-11.fc17
then log in and leave karma (feedback).
Comment 22 Fedora Update System 2012-04-11 22:44:08 EDT
bacula2-2.4.4-11.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.