Red Hat Bugzilla – Bug 492753
Logrotate kills httpd after a "yum update"
Last modified: 2009-08-18 08:04:21 EDT
Today I realized that apache httpd that is always running on my machine, has mysteriously died. The last thing on the log was:
[Sun Mar 29 04:07:16 2009] [notice] SIGHUP received. Attempting to restart
httpd: Syntax error on line 196 of /etc/httpd/conf/httpd.conf: Cannot load /etc/
httpd/modules/mod_proxy.so into server: /etc/httpd/modules/mod_proxy.so: undefin
ed symbol: ap_timeout_parameter_parse
After that, apache just died. So it was logrotate that killed my Apache using the /etc/logrotate.d/httpd script - which sends a SIGHUP to the running apache.
But why did SIGHUP kill my apache instead of just restarting? After some investigation, I figured out what happened, I think. What I am guessting happened is that sometime in the past, "yum updated" upgraded my httpd, and mod_proxy.so in particular, but didn't restart the server. When the (old) running httpd received a signal, it tried to reload the (new) modules, and that failed.
I'm not sure exactly what should be done to fix this problem. Should logrotate's script be changed to do something different (e.g., restart the httpd service completely, but contrast with bug 221073)? should "yum update" have done something different (like restart the httpd service immedately?) Should httpd have done something different (not reload a module which has already been loaded in the running process)?
But the end result is that, to make a long story short, logrotate killed my httpd. Which is why I am filing this bug for "logrotate".
signalling Apache to reset log files by SIGHUP is the right thing documented in Apache
so logrotate did nothing wrong.
reassigning to httpd
Just to add that our webserver had exactly the same problem following a recent logrotate:
[Sun Mar 29 04:14:10 2009] [notice] SIGHUP received. Attempting to restart
httpd: Syntax error on line 188 of /etc/httpd/conf/httpd.conf: Cannot load /etc/httpd/modules/mod_proxy.so into server: /etc/httpd/modules/mod_proxy.so: undefined symbol: ap_timeout_parameter_parse
Manually restarting it brought apache back up without a complaint, but I can't see why it didn't work when logrotate tried to restart it.
We experienced the same issue as well.
[Sun May 24 05:09:15 2009] [notice] SIGHUP received. Attempting to restart
httpd: Syntax error on line 199 of /etc/httpd/conf/httpd.conf: Cannot load /etc/httpd/modules/mod_proxy.so into server: /etc/httpd/modules/mod_proxy.so: undefined symbol: ap_timeout_parameter_parse
Further, this is running x86_64 version, which may or may not be related. I have 3 other servers running FC10's apache, one running x86_64, two running i386, but they have not experienced this problem, only this one (which is the one that receives probably 90% of the traffic load)
Versions of httpd in question:
Linux version 126.96.36.199-170.2.56.fc10.x86_64 - httpd-2.2.11-2.fc10.x86_64 - this is the server that failed
Linux version 188.8.131.52-78.2.50.fc9.i686 - httpd-2.2.11-2.fc10.i386 - running fine
(this was updated to FC10 via yum, the fc10 kernel was older than the current fc9 kernel which is why it retained the fc9 kernel)
Linux version 184.108.40.206-78.2.50.fc9.i586 - httpd-2.2.11-2.fc10.i386 - running fine (same comment about the kernel as above)
Linux version 220.127.116.11-170.2.35.fc10.x86_64 - httpd-2.2.11-2.fc10.x86_64 - this is our x86_64 machine that is running fine.
Looks like an update has cured it for me, not sure which one but the logrotate script was re-written on an update (I had commented out the HUP line) and these are the modules that were updated at that time:
*** This bug has been marked as a duplicate of bug 491567 ***