Description of problem: I was testing the community ami "ami-2abf4443" on amazon and while performing the httpd related tests, I got following: [root@localhost ec2-user]# git clone git://fedorapeople.org/home/fedora/athmane/public_git/testcases-scripts.git Cloning into testcases-scripts... remote: Counting objects: 17, done. remote: Compressing objects: 100% (14/14), done. remote: Total 17 (delta 4), reused 0 (delta 0) Receiving objects: 100% (17/17), done. Resolving deltas: 100% (4/4), done. [root@localhost ec2-user]# cd testcases-scripts/httpd/ [root@localhost httpd]# ./runtests.sh Starting httpd (via systemctl): [ OK ] Running ./httpd_basic_auth.sh - httpd basic authentication test. Adding password for user test Reloading httpd configuration (via systemctl): Job failed. See system logs and 'systemctl status' for details. [FAILED] ==> [ PASS ] Running ./httpd_php.sh - httpd handle PHP test ==> [ PASS ] Running ./httpd_servehtml.sh - httpd serve html page test. ==> [ PASS ] Running ./httpd_servehtml_ssl.sh - httpd serve html page test over SSL. ==> [ PASS ] Running ./httpd_vhost.sh - httpd virtual host test. Reloading httpd configuration (via systemctl): [ OK ] ==> [ PASS ] tail /var/log/messages says: =============================== Aug 5 07:34:20 localhost yum[1297]: Installed: php-common-5.3.6-2.fc15.x86_64 Aug 5 07:34:20 localhost yum[1297]: Installed: apr-1.4.5-1.fc15.x86_64 Aug 5 07:34:20 localhost yum[1297]: Installed: apr-util-1.3.12-1.fc15.x86_64 Aug 5 07:34:20 localhost yum[1297]: Installed: httpd-tools-2.2.17-10.fc15.1.x86_64 Aug 5 07:34:20 localhost yum[1297]: Installed: apr-util-ldap-1.3.12-1.fc15.x86_64 Aug 5 07:34:22 localhost systemd[1]: Reloading. Aug 5 07:34:22 localhost systemd-getty-generator[1322]: Failed to create symlink from /lib/systemd/system/serial-getty@.service to /run/systemd/generator/getty.target.wants/serial-getty: File exists Aug 5 07:34:22 localhost systemd[1]: /lib/systemd/system-generators/systemd-getty-generator exited with exit status 1. Aug 5 07:34:22 localhost yum[1297]: Installed: httpd-2.2.17-10.fc15.1.x86_64 Aug 5 07:34:23 localhost yum[1297]: Installed: php-cli-5.3.6-2.fc15.x86_64 Aug 5 07:34:23 localhost yum[1297]: Installed: php-pdo-5.3.6-2.fc15.x86_64 Aug 5 07:34:24 localhost yum[1297]: Installed: mysql-libs-5.5.14-2.fc15.x86_64 Aug 5 07:34:24 localhost yum[1297]: Installed: php-mysql-5.3.6-2.fc15.x86_64 Aug 5 07:34:24 localhost yum[1297]: Installed: php-5.3.6-2.fc15.x86_64 Aug 5 07:34:24 localhost yum[1297]: Installed: 1:mod_ssl-2.2.17-10.fc15.1.x86_64 Aug 5 07:34:26 localhost systemd[1]: Reloading. Aug 5 07:34:26 localhost systemd-getty-generator[1350]: Failed to create symlink from /lib/systemd/system/serial-getty@.service to /run/systemd/generator/getty.target.wants/serial-getty: File exists Aug 5 07:34:26 localhost systemd[1]: /lib/systemd/system-generators/systemd-getty-generator exited with exit status 1. Aug 5 07:34:27 localhost systemd[1]: httpd.service: control process exited, code=exited status=7 [root@localhost httpd]# systemctl status systemd.service systemd.service Loaded: error Active: inactive (dead) Version-Release number of selected component (if applicable): "ami-2abf4443" How reproducible: always Steps to Reproduce: 1. git clone git://fedorapeople.org/home/fedora/athmane/public_git/testcases-scripts.git 2. Execute runtest.sh 3. Actual results: Running ./httpd_basic_auth.sh - httpd basic authentication test. Adding password for user test Reloading httpd configuration (via systemctl): Job failed. See system logs and 'systemctl status' for details. Expected results: All tests should pass. Additional info:
So, the systemd-getty-generator output is just noise and can safely be ignored (fixed now in git). It appears as if reloading of apache via the init script failed with exit code 7. That might not be a bug at all. Reassigning to Apache.
"reload" exits with exit code 7 if httpd is not running (httpd process with the pid stored in httpd pidfile does not exist). I have checked your tests and the problem is that httpd needs some time to fork and write its pid to pidfile. If you put "sleep 2" after "service httpd start" in "0-install_httpd.sh", it will *probably* work as expected ("probably", because 2 seconds don't have to be enough everytime... proper solution is to use some loop to check if httpd started completely and is ready to serve pages). I will close this as NOTABUG, because it doesn't seem to be httpd problem, but test problem.
(In reply to comment #2) > I have checked your tests and the problem is that httpd needs some time to fork > and write its pid to pidfile. If that's so, it is a bug. A forking daemon must exit the original process _after_ the pidfile has been written. For all the details, see: http://0pointer.de/public/systemd-man/daemon.html
Do many daemons actually behave like that? I picked exim and mysqld at random and it looks like they both behave similarly to httpd - fork(), exit() in the parent, write pid file in the child.
I have to admit httpd really acts like Michal pointed out. I'm not so familiar with code so far, but I'm trying to find out particular place where it writes PID and if it's really in child process.
Hm, forget the first sentence. I wanted to say it does *not* act like it should according to http://0pointer.de/public/systemd-man/daemon.html.
Yes; httpd is not unusual in how it daemonizes, and I am fairly sceptical when I see people: a) inventing new "rules" which are so far removed from existing practice, and then b) declaring that existing practice is "buggy" It might be better to have httpd run with -DFOREGROUND -DNO_DETACH so the parent does not fork at all - I believe systemd will cope with this, or possibly need a tweak to the service file, and then it avoids the race with the exit + the new parent writing the pid file.
(In reply to comment #7) > Yes; httpd is not unusual in how it daemonizes, and I am fairly sceptical when > I see people: > > a) inventing new "rules" which are so far removed from existing practice, and > then > b) declaring that existing practice is "buggy" The problem with not following the rule is that the system gets false information about when the daemon is started and ready. Even before systemd came, wasn't the initscript susceptible to the very same problem when "start" and "reload" were called in a quick succession? Unfortunately, "existing practice" and "buggy" are not mutually exclusive. > It might be better to have httpd run with -DFOREGROUND -DNO_DETACH so the > parent does not fork at all - I believe systemd will cope with this, or > possibly need a tweak to the service file, and then it avoids the race with the > exit + the new parent writing the pid file. Yes, systemd can handle non-forking services. For correctness there still should be a way to tell when the service is started and ready. There are two possibilities: 1. The service gets patched to add socket activation. Then it can be assumed to be ready (to accept connections) at all times and it can use "Type=simple". 2. The service gets patched to call sd_notify() when it's ready. The unit file will use "Type=notify". Socket activation is optional.
(In reply to comment #0) > [root@localhost httpd]# systemctl status systemd.service > systemd.service > Loaded: error > Active: inactive (dead) There's no such thing as "systemd.service". You probably meant to check the status of "httpd.service".
(In reply to comment #8) > The problem with not following the rule is that the system gets false > information about when the daemon is started and ready. There is no "false information", that is silly. All you are saying is that systemd's (entirely arbitrary) expectation about what the exit code means is completely different from the actual behaviour. Jan is looking into moving the service to the non-forked type.
Or maybe writing module to support sd_notify(). I will try both and see.
(In reply to comment #10) > (In reply to comment #8) > > The problem with not following the rule is that the system gets false > > information about when the daemon is started and ready. > > There is no "false information", that is silly. All you are saying is that > systemd's (entirely arbitrary) expectation about what the exit code means is > completely different from the actual behaviour. The expectation is not arbitrary. It is the only reasonable interpretation of the exit code. Otherwise the exit code would be meaningless. I already mentioned the problem is not caused by systemd. The problem shows up even when systemd is not involved. For a proof, try this: [root@alice ~]# cd /etc/init.d [root@alice init.d]# export SYSTEMCTL_SKIP_REDIRECT=1 [root@alice init.d]# ./httpd start; ./httpd reload Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using ::1 for ServerName [ OK ] Reloading httpd: [FAILED] [root@alice init.d]# echo $? 7
Well, for systemd, it can be partly fixed by mod_systemd I'm attaching. It's compromise between patching httpd and doing nothing and it fixes "start;reload" sequence using systemctl. -DFOREGROUND -DNO_DETACH didn't work correctly for me and I think there would be problem with graceful restarts. Unfortunately I can't say if this can be fixed upstream or not.
Created attachment 521848 [details] mod_systemd.c
This message is a notice that Fedora 15 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 15. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At this time, all open bugs with a Fedora 'version' of '15' have been closed as WONTFIX. (Please note: Our normal process is to give advanced warning of this occurring, but we forgot to do that. A thousand apologies.) Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, feel free to reopen this bug and simply change the 'version' to a later Fedora version. Bug Reporter: Thank you for reporting this issue and we are sorry that we were unable to fix it before Fedora 15 reached end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged to click on "Clone This Bug" (top right of this page) and open it against that version of Fedora. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping