Red Hat Bugzilla – Bug 78537
httpd intermittent failure (at 2GB limit?)
Last modified: 2007-04-18 12:48:41 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20021003
Description of problem:
> Confirmed that with SIGXFSZ set to ignore, new log entries are
> simply dropped once the 2gb limit is passed, so marking this closed.
Unfortunately this isn't what I'm seeing on my Red Hat Linux 8.0 web server. My
site gets very heavy traffic and often hits the 2GB limit with access log. A
few times I noticed my site stopped responding and when I look at the processes
I see this:
PID TTY STAT TIME COMMAND
1 ? S 0:39 init 
1836 ? S 1:10 syslogd -m 0
1879 ? S 1:39 sendmail: accepting connections
1888 ? S 0:00 sendmail: Queue runner@01:00:00 for /var/spool/client
1920 ? S 0:01 crond
9754 ? S 1:06 /usr/sbin/httpd
30692 ? Z 0:31 [httpd <defunct>]
21284 ? Z 0:32 [httpd <defunct>]
8589 ? Z 0:32 [httpd <defunct>]
28034 ? Z 0:34 [httpd <defunct>]
28154 ? Z 0:32 [httpd <defunct>]
32331 ? Z 0:37 [httpd <defunct>]
6490 ? Z 0:31 [httpd <defunct>]
1844 ? Z 0:34 [httpd <defunct>]
23605 ? Z 0:30 [httpd <defunct>]
Strangely it sits in this state for a few minutes, sometimes up to an hour, and
sometimes recovers on its own with all httpd processes happily serving again.
Several hours later it fails again in this manner and repeats.
Steps to Reproduce:
1. Fill up 2GB log file with Apache 2.0.
2. Pound it with constant moderate-heavy traffic.
httpd failure with bunch of child zombie processes. Somehow recovers on its
own. Fails and recovers repeatedly.
Shouldn't fail at all.
Are you getting any error_log entries?
Just to be clear, are all the httpd child processes zombies in this situation? A
"ps axf | grep httpd" might be useful; better yet, enabling the /server-status
page and seeing what that gives.
[Fri Nov 29 16:21:11 2002] [notice] child pid 8714 exit signal Segmentation faul
[Fri Nov 29 16:23:48 2002] [notice] child pid 8922 exit signal Segmentation faul
These error_log entries occurred a few hours _before_ the logs overflowed last
night. These temporary zombie failure occurr intermittently so it is difficult
to see it happening. The above processes are ALL the processes running within
that security context (using Linux Virtual Server with security contexts kernel
Hmm, I am no longer certain this failure is caused by full logs.
Next time it fails, what can I do to help diagnose this?
Seeing problems with httpd failing in two instances.
1. 2GB limit of log files
2. During log rotate
These issues may or may not be related.
Another, related issue is that apache doesn't currently (up to RH9 ) support
*transferring* files over 2GB either. Just noticed it, trying to download a
custom RH9 DVD image... Simply rebuilding httpd with LFS (-DFILE_OFFSET_BITS=-64
etc...) doesn't work either since it makes it segfault on startup but I guess
that's to be expected having seen the comments in in bug 69520.
Thanks for the report. This is a mass bug update; since this release
of Red Hat Linux is no longer supported, please either:
a) try and reproduce the bug with a supported version of Red Hat
Enterprise Linux or Fedora Core, and re-open this bug as appropriate
after changing the Product field, or,
b) if relevant, try and reproduce this bug using the current version
of the upstream package, and report the bug upstream.
c) report the bug to the Fedora Legacy project who may wish to
continue maintenance of this package.