Description of problem: systemd stop/restart httpd needs ~90 secs There are no pending requests (web server is quiescent). Version-Release number of selected component (if applicable): 2.4 How reproducible: systemctl stop httpd (or systemctl restart httpd) Steps to Reproduce: 1. systemctl stop httpd & 2. 'systemctl status httpd' or 'pgrep httpd' show some httpd orphans 3. Actual results: 'systemctl stop httpd' hang for ~90 secs (default TimeoutStopSec=90 secs reached) Expected results: Apache should stop after few secs even on a graceful stop/restart. Additional info: http://httpd.apache.org/docs/2.4/stopping.html "...you shouldn't ever need to send signals to any process except the parent." but the default KillMode for systemd is "control-group". workaround: Add KillMode=none to /usr/lib/systemd/system/httpd.service An alternative workaround could be to set TimeoutStopSec=10 or so but KillMode=none seems to me the polite way.
Hmmmm, interesting. This isn't trivially reproducible here. Does the error_log say anything about why the daemon is slow to shut down? "systemctl stop httpd" *will* first send attempt to terminate (only) the httpd parent, not the whole process group. The question is the correct behaviour in the case of a parent which is slow to terminate. We are using graceful-stop here which complicates things.
With KillMode=none Feb 14 11:02:28 webmail1 systemd[1]: Stopping The Apache HTTP Server... Feb 14 11:02:28 webmail1 systemd[1]: Stopped The Apache HTTP Server. With default KillMode=control-group Feb 14 11:07:11 webmail1 systemd[1]: Stopping The Apache HTTP Server... Feb 14 11:08:42 webmail1 systemd[1]: httpd.service stopping timed out (2). Killing. Feb 14 11:08:42 webmail1 systemd[1]: Stopped The Apache HTTP Server. Feb 14 11:08:42 webmail1 systemd[1]: Unit httpd.service entered failed state /var/log/http/error_log With KillMode=none [Thu Feb 14 11:02:28.717716 2013] [mpm_prefork:notice] [pid 10224] AH00170: caught SIGWINCH, shutting down gracefully With default KillMode=control-group there's a stack dump Thu Feb 14 11:07:11.924371 2013] [mpm_prefork:notice] [pid 10267] AH00170: caught SIGWINCH, shutting down gracefully *** glibc detected *** /usr/sbin/httpd: free(): invalid pointer: 0x00007f99df359e90 *** *** glibc detected *** /usr/sbin/httpd: free(): invalid pointer: 0x00007f99df359e90 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x359707ca8e)[0x7f99dc160a8e] /lib64/libapr-1.so.0(apr_pool_destroy+0x19c)[0x7f99dc8d585c] /etc/httpd/modules/mod_mpm_prefork.so(+0x31be)[0x7f99d37d81be] /etc/httpd/modules/mod_mpm_prefork.so(+0x39fb)[0x7f99d37d89fb] ======= Backtrace: ========= /lib64/libpthread.so.0(+0x3597c0f000)[0x7f99dc6af000] /etc/httpd/modules/mod_authn_file.so(+0xbc0)[0x7f99da64cbc0] ======= Memory map: ======== [...] I did a 'yum upgrade' and kernel was updated from 3.7.4-204.fc18.x86_64 to 3.7.6-201.fc18.x86_64 (systemd and httpd are already the latest) Now I don't get the stack dump anymore. Other details: # systemctl start httpd.service # ps -def|grep http root 1752 1151 0 12:18 pts/0 00:00:00 tail -f /var/log/httpd/error_log root 1847 1 4 12:22 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 1849 1847 0 12:22 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 1850 1847 0 12:22 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 1851 1847 0 12:22 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 1852 1847 0 12:22 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 1853 1847 0 12:22 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND root 1855 1151 0 12:22 pts/0 00:00:00 grep --color=auto http # systemctl status httpd.service httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled) Active: active (running) since Thu 2013-02-14 12:22:43 CET; 14s ago Process: 1833 ExecStop=/usr/sbin/httpd $OPTIONS -k graceful-stop (code=exited, status=0/SUCCESS) Main PID: 1847 (httpd) Status: "Total requests: 0; Current requests/sec: 0; Current traffic: 0 B/sec" CGroup: name=systemd:/system/httpd.service ├─1847 /usr/sbin/httpd -DFOREGROUND ├─1849 /usr/sbin/httpd -DFOREGROUND ├─1850 /usr/sbin/httpd -DFOREGROUND ├─1851 /usr/sbin/httpd -DFOREGROUND ├─1852 /usr/sbin/httpd -DFOREGROUND └─1853 /usr/sbin/httpd -DFOREGROUND Feb 14 12:22:41 webmail1 systemd[1]: Starting The Apache HTTP Server... Feb 14 12:22:43 webmail1 systemd[1]: Started The Apache HTTP Server. # systemctl stop httpd.service ..zzz... The error_log instantaneously: [Thu Feb 14 12:23:09.204078 2013] [mpm_prefork:notice] [pid 1847] AH00170: caught SIGWINCH, shutting down gracefully In the meantime: root 1752 1151 0 12:18 pts/0 00:00:00 tail -f /var/log/httpd/error_log apache 1851 1 0 12:22 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 1852 1 0 12:22 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND root 1857 1151 0 12:23 pts/0 00:00:00 systemctl stop httpd.service root 1858 1857 0 12:23 pts/0 00:00:00 /usr/bin/systemd-tty-ask-password-agent --watch Before you ask : yes there is SSL virtual host but the certificate is not password encrypted. Anyway why systemd-tty-ask-password-agent should ask for a password stopping httpd :-) ?
For what it's worth I have the same problem on latest F18 (Kernel 3.8.4). I put the workaround in (KillMode=none) and problem goes away. [Sun Mar 31 15:34:43.215636 2013] [mpm_prefork:notice] [pid 676] AH00170: caught SIGWINCH, shutting down gracefully *** glibc detected *** /usr/sbin/httpd: free(): invalid pointer: 0xb9306548 *** ======= Backtrace: ========= /lib/libc.so.6(+0x47b17ff9)[0xb7295ff9] /lib/libapr-1.so.0(apr_pool_destroy+0x1ca)[0xb740d10a] /etc/httpd/modules/mod_mpm_prefork.so(+0x27cf)[0xb70367cf] /etc/httpd/modules/mod_mpm_prefork.so(+0x318b)[0xb703718b] [0xb7705400] /etc/httpd/modules/mod_authn_dbd.so(+0x9f9)[0xb71ac9f9] /lib/ld-linux.so.2(+0x47a89d04)[0xb7715d04] /lib/libc.so.6(+0x47ad3f01)[0xb7251f01]
httpd-2.4.4-4.fc19 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/httpd-2.4.4-4.fc19
Package httpd-2.4.4-4.fc19: * should fix your issue, * was pushed to the Fedora 19 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing httpd-2.4.4-4.fc19' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2013-5797/httpd-2.4.4-4.fc19 then log in and leave karma (feedback).
httpd-2.4.4-4.fc19 has been pushed to the Fedora 19 stable repository. If problems still persist, please make note of it in this bug report.
httpd-2.4.4-3.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/httpd-2.4.4-3.fc18
httpd-2.4.4-3.fc18 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report.