Description of problem: Every week in the night from saturday to sunday httpd runs in an endless crash loop. It is restarted by systemd and then crashes again immediately. This happens every second. Version-Release number of selected component (if applicable): httpd-2.4.37-5.fc29.x86_64 How reproducible: Every week. Steps to Reproduce: 1. don't know, happens automatically 2. 3. Actual results: Sun 2018-12-16 03:34:05 CET 4945 0 0 11 missing /usr/sbin/httpd Sun 2018-12-16 03:34:07 CET 4944 0 0 11 missing /usr/sbin/httpd Sun 2018-12-16 03:34:09 CET 4980 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:09 CET 5158 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:09 CET 4985 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:09 CET 4992 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:09 CET 4977 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:09 CET 4998 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:09 CET 5155 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:11 CET 5413 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:11 CET 5535 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:11 CET 5538 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:11 CET 5415 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:11 CET 5699 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:12 CET 5698 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:13 CET 5949 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:13 CET 5828 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:13 CET 5827 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:14 CET 5950 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:14 CET 6084 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:15 CET 6086 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:15 CET 6085 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:17 CET 6420 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:19 CET 6275 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:19 CET 6418 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:19 CET 6425 0 0 4 missing /usr/sbin/httpd Sun 2018-12-16 03:34:20 CET 6798 0 0 4 missing /usr/sbin/httpd .... (gdb) bt #0 0x00007fc9468286b2 in ?? () from /usr/lib64/php/modules/intl.so #1 0x000055bfa4cdcca0 in ?? () #2 0x00007fc94935dd98 in apr_proc_fork () from /lib64/libapr-1.so.0 #3 0x00007fc947fe46b9 in ?? () from /etc/httpd/modules/mod_wsgi.so #4 0x00007fc947fe7a34 in ?? () from /etc/httpd/modules/mod_wsgi.so #5 0x00007fc949355d5f in apr_proc_other_child_alert () from /lib64/libapr-1.so.0 #6 0x00007fc9489ea816 in ?? () from /etc/httpd/modules/mod_mpm_prefork.so #7 0x000055bfa2d23bde in ap_run_mpm () #8 0x000055bfa2d1c2d3 in main () Expected results: No crash at all Additional info: It loops until I run "systemctl restart httpd.service".
Can you double-check this system has updated to httpd-2.4.37-5? There was a fix for an issue which would manifest like this in -5.
Yes, it's httpd-2.4.37-5 indeed.
The bug can be triggered by systemctl reload httpd.service which is executed regularly by logrotate, for example. Whereas an systemctl restart httpd.service does not trigger the error. Hope this helps.
Hi Erik, thanks for your report! Could you please share with us, what's the value of your SSLProtocol option, if you are using mod_ssl?
mod_ssl is not used in my setup.
Do you mean it's not configured or the package is not installed?
The package is not installed.
Please provide output from root running both: httpd -M rpm -qf /etc/httpd/modules/*.so | sort -u wondering if this has the same root-cause as bug 1676842 which was a non-Fedora php package.
Created attachment 1536268 [details] output of httpd -M The output requested.
# rpm -qf /etc/httpd/modules/*.so | sort -u httpd-2.4.38-2.fc29.x86_64 mod_http2-1.11.1-1.fc29.x86_64 php-7.2.14-1.fc29.x86_64 python2-mod_wsgi-4.6.4-2.fc29.x86_64
Have you modified /etc/httpd/conf.modules.d/15-php.conf or is it stock? Please provide the config. Remi, this looks like bug 1676842 but with prefork MPM & mod_php loaded.
No, /etc/httpd/conf.modules.d/15-php.conf is stock. # less 15-php.conf # # PHP is an HTML-embedded scripting language which attempts to make it # easy for developers to write dynamically generated webpages. # # Cannot load both php5 and php7 modules <IfModule !mod_php5.c> <IfModule prefork.c> LoadModule php7_module modules/libphp7.so </IfModule> </IfModule> <IfModule !mod_php5.c> <IfModule !prefork.c> # ZTS module is not supported, so FPM is preferred LoadModule php7_module modules/libphp7-zts.so </IfModule> </IfModule>
Are you sure it's the parent apache that is crashing and systemd is restarting it... or... is it the apache children crashing and the parent apache restarting them? What does journalctl say (assuming it's handling coredumps)? What does apache error.log show when the endless crashes are occurring? What does: systemctl status httpd say is the start time in the Active: line? Is the start time static or increasing (i.e. parent really is dying)? If it's the children dying, not the parent, you may have the same bug as me bz#1680481 If parent is dying, it's a different bug.
Cross-checking with the other bug - can you confirm whether you have php-imap installed?
Yes, php-imap is installed. You might want to have a closer look at the stack backtrace I provied. This helps more than guessing into blue.
Thanks. Yeah, the backtraces all show the crash is in a bogus pointer in an pthread_atfork handler, the actual memory address may show up in some php .so but it's a stale pointer from an unloaded DSO, most likely. I'm working on it.
Let's just track this once, it's clearly a common bug at this ponit. *** This bug has been marked as a duplicate of bug 1680481 ***