Red Hat Bugzilla – Bug 809360
sd_notifyf doesn't work as expected for httpd
Last modified: 2013-01-15 17:49:45 EST
Created attachment 574773 [details] mod_systemd Description of problem: I'm trying to use sd_notifyf in httpd to overcome issues with the PID file creation (see Bug #728465) and possibly start using some more advanced systemd features. To achieve it, I'm trying to write mod_systemd httpd module (attached). It works like this: 1. It calls getpid() to get and store the PID in main httpd process. The pid_file is not written that time yet. 2. When the Apache forks and creates the children processes, it calls sd_notifyf with the PID of main process from the child process. The reason why to do it here is that pid_file is already written that time. The problem is that if I run "httpd" command without systemd, I see the following in the httpd log meaning that it got to my systemd related code (please ignore logging debug messages as "error", it's only for my convenience): [Mon Apr 02 15:16:05.678043 2012] [core:notice] [pid 24112:tid 140255251499008] SELinux policy enabled; httpd running as context unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 [Mon Apr 02 15:16:05.679088 2012] [suexec:notice] [pid 24112:tid 140255251499008] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Mon Apr 02 15:16:06.000579 2012] [ssl:notice] [pid 24112:tid 140255251499008] AH01886: SSL FIPS mode disabled [Mon Apr 02 15:16:06.030958 2012] [:error] [pid 24112:tid 140255251499008] AH00521: init_module [Mon Apr 02 15:16:06.051324 2012] [auth_digest:notice] [pid 24113:tid 140255251499008] AH01757: generating secret for digest authentication ... [Mon Apr 02 15:16:07.000118 2012] [lbmethod_heartbeat:notice] [pid 24113:tid 140255251499008] AH02282: No slotmem from mod_heartmonitor [Mon Apr 02 15:16:07.000315 2012] [ssl:notice] [pid 24113:tid 140255251499008] AH01886: SSL FIPS mode disabled [Mon Apr 02 15:16:07.043374 2012] [:error] [pid 24113:tid 140255251499008] AH00521: systemd_post_config [Mon Apr 02 15:16:07.043424 2012] [:error] [pid 24113:tid 140255251499008] AH00521: got pid 24113 [Mon Apr 02 15:16:07.045062 2012] [:error] [pid 24115:tid 140255251499008] AH00521: using pid 24113 But if I start it using systemd, it looks like httpd is stopped/killed before it even tries to call sd_notify: [Mon Apr 02 15:22:33.448722 2012] [core:notice] [pid 24437:tid 139748586375168] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0 [Mon Apr 02 15:22:33.449712 2012] [suexec:notice] [pid 24437:tid 139748586375168] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Mon Apr 02 15:22:34.000216 2012] [ssl:notice] [pid 24437:tid 139748586375168] AH01886: SSL FIPS mode disabled [Mon Apr 02 15:22:34.035757 2012] [:error] [pid 24437:tid 139748586375168] AH00521: init_module [Mon Apr 02 15:22:34.055171 2012] [auth_digest:notice] [pid 24438:tid 139748586375168] AH01757: generating secret for digest authentication ... Service file I currently use looks like this: > [Unit] > Description=The Apache HTTP Server > After=syslog.target network.target remote-fs.target nss-lookup.target > > [Service] > Type=notify > PIDFile=/var/run/httpd/httpd.pid > EnvironmentFile=/etc/sysconfig/httpd > ExecStart=/usr/sbin/httpd $OPTIONS > ExecReload=/usr/sbin/httpd $OPTIONS -t > ExecReload=/usr/sbin/httpd -HUP $MAINPID > ExecStop=/usr/sbin/httpd $OPTIONS -k graceful-stop > PrivateTmp=true > NotifyAccess=all > > [Install] > WantedBy=multi-user.target Version-Release number of selected component (if applicable): systemd 37-17.fc16 Steps to Reproduce: 1. Compile and install attached mod_systemd and enable it in httpd's configuration file. 2. Start httpd using systemd Maybe it's caused by my misconfiguration or misunderstanding of the way how it's supposed to work. But I was trying to configure it differently for long time, so I think it's worth reporting.
Adding reproducer. I believe this should work with sd_notifyf, shouldn't it?
Created attachment 610232 [details] simple reproducer
There is also variation of this reproducer showing different problem. If I keep parent process running and try to update STATUS from child process, it is not reflected in systemctl status output. I still see "Status: "Processing requests..." there. From the documentation I would expect it will work even called from the child: If the unset_environment parameter is non-zero sd_notify() will unset the $NOTIFY_SOCKET environment variable before returning (regardless whether the function call itself succeeded or not). Further calls to sd_notify() will then fail, but the variable is no longer inherited by child processes.
Created attachment 610248 [details] reproducer for second problem
For the first reproducer I would expect that systemd will wait some time for sd_notify call and it will not care from which children process it comes. I thought that's exactly why sd_notify exists. Currently it kills children when main process terminates.
Hmm, you set Type=notify, but you actually have a forking process, right? You need to set Type=forking in that case, otherwise systemd will assume your main process is dead and terminate you immediately. You also need to set NotifyAccess=all if it is not the original process that sends off the message
Hm, I think we could close this bug. I remember having discussion about this with someone from systemd community and find out the current systemd behaviour is good. In httpd we've fixed it by using -DFOREGROUND httpd option, so httpd does not fork and Type=notify can be used. Feel free to close this bug.
Cool, thanks!