Red Hat Bugzilla – Bug 809360
sd_notifyf doesn't work as expected for httpd
Last modified: 2013-01-15 17:49:45 EST
Created attachment 574773 [details]
Description of problem:
I'm trying to use sd_notifyf in httpd to overcome issues with the PID file creation (see Bug #728465) and possibly start using some more advanced systemd features.
To achieve it, I'm trying to write mod_systemd httpd module (attached). It works like this:
1. It calls getpid() to get and store the PID in main httpd process. The pid_file is not written that time yet.
2. When the Apache forks and creates the children processes, it calls sd_notifyf with the PID of main process from the child process. The reason why to do it here is that pid_file is already written that time.
The problem is that if I run "httpd" command without systemd, I see the following in the httpd log meaning that it got to my systemd related code (please ignore logging debug messages as "error", it's only for my convenience):
[Mon Apr 02 15:16:05.678043 2012] [core:notice] [pid 24112:tid 140255251499008] SELinux policy enabled; httpd running as context unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[Mon Apr 02 15:16:05.679088 2012] [suexec:notice] [pid 24112:tid 140255251499008] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Mon Apr 02 15:16:06.000579 2012] [ssl:notice] [pid 24112:tid 140255251499008] AH01886: SSL FIPS mode disabled
[Mon Apr 02 15:16:06.030958 2012] [:error] [pid 24112:tid 140255251499008] AH00521: init_module
[Mon Apr 02 15:16:06.051324 2012] [auth_digest:notice] [pid 24113:tid 140255251499008] AH01757: generating secret for digest authentication ...
[Mon Apr 02 15:16:07.000118 2012] [lbmethod_heartbeat:notice] [pid 24113:tid 140255251499008] AH02282: No slotmem from mod_heartmonitor
[Mon Apr 02 15:16:07.000315 2012] [ssl:notice] [pid 24113:tid 140255251499008] AH01886: SSL FIPS mode disabled
[Mon Apr 02 15:16:07.043374 2012] [:error] [pid 24113:tid 140255251499008] AH00521: systemd_post_config
[Mon Apr 02 15:16:07.043424 2012] [:error] [pid 24113:tid 140255251499008] AH00521: got pid 24113
[Mon Apr 02 15:16:07.045062 2012] [:error] [pid 24115:tid 140255251499008] AH00521: using pid 24113
But if I start it using systemd, it looks like httpd is stopped/killed before it even tries to call sd_notify:
[Mon Apr 02 15:22:33.448722 2012] [core:notice] [pid 24437:tid 139748586375168] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Mon Apr 02 15:22:33.449712 2012] [suexec:notice] [pid 24437:tid 139748586375168] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Mon Apr 02 15:22:34.000216 2012] [ssl:notice] [pid 24437:tid 139748586375168] AH01886: SSL FIPS mode disabled
[Mon Apr 02 15:22:34.035757 2012] [:error] [pid 24437:tid 139748586375168] AH00521: init_module
[Mon Apr 02 15:22:34.055171 2012] [auth_digest:notice] [pid 24438:tid 139748586375168] AH01757: generating secret for digest authentication ...
Service file I currently use looks like this:
> Description=The Apache HTTP Server
> After=syslog.target network.target remote-fs.target nss-lookup.target
> ExecStart=/usr/sbin/httpd $OPTIONS
> ExecReload=/usr/sbin/httpd $OPTIONS -t
> ExecReload=/usr/sbin/httpd -HUP $MAINPID
> ExecStop=/usr/sbin/httpd $OPTIONS -k graceful-stop
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Compile and install attached mod_systemd and enable it in httpd's configuration file.
2. Start httpd using systemd
Maybe it's caused by my misconfiguration or misunderstanding of the way how it's supposed to work. But I was trying to configure it differently for long time, so I think it's worth reporting.
Adding reproducer. I believe this should work with sd_notifyf, shouldn't it?
Created attachment 610232 [details]
There is also variation of this reproducer showing different problem. If I keep parent process running and try to update STATUS from child process, it is not reflected in systemctl status output. I still see "Status: "Processing requests..." there.
From the documentation I would expect it will work even called from the child:
If the unset_environment parameter is non-zero sd_notify() will unset the $NOTIFY_SOCKET environment variable before returning (regardless whether the function call itself succeeded or not). Further calls to sd_notify() will then fail, but the variable is no longer inherited by child processes.
Created attachment 610248 [details]
reproducer for second problem
For the first reproducer I would expect that systemd will wait some time for sd_notify call and it will not care from which children process it comes. I thought that's exactly why sd_notify exists. Currently it kills children when main process terminates.
Hmm, you set Type=notify, but you actually have a forking process, right?
You need to set Type=forking in that case, otherwise systemd will assume your main process is dead and terminate you immediately. You also need to set NotifyAccess=all if it is not the original process that sends off the message
Hm, I think we could close this bug. I remember having discussion about this with someone from systemd community and find out the current systemd behaviour is good.
In httpd we've fixed it by using -DFOREGROUND httpd option, so httpd does not fork and Type=notify can be used.
Feel free to close this bug.