Bug 78537 - httpd intermittent failure (at 2GB limit?)
Summary: httpd intermittent failure (at 2GB limit?)
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: httpd
Version: 8.0
Hardware: All
OS: Linux
Target Milestone: ---
Assignee: Joe Orton
QA Contact: Brian Brock
Depends On:
TreeView+ depends on / blocked
Reported: 2002-11-25 10:44 UTC by Warren Togami
Modified: 2007-04-18 16:48 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2005-01-25 16:46:08 UTC

Attachments (Terms of Use)

Description Warren Togami 2002-11-25 10:44:39 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20021003

Description of problem:
> Confirmed that with SIGXFSZ set to ignore, new log entries are 
> simply dropped once the 2gb limit is passed, so marking this closed.

Unfortunately this isn't what I'm seeing on my Red Hat Linux 8.0 web server.  My
site gets very heavy traffic and often hits the 2GB limit with access log.  A
few times I noticed my site stopped responding and when I look at the processes
I see this:

    1 ?        S      0:39 init [3]
 1836 ?        S      1:10 syslogd -m 0
 1879 ?        S      1:39 sendmail: accepting connections
 1888 ?        S      0:00 sendmail: Queue runner@01:00:00 for /var/spool/client
1920 ?        S      0:01 crond
 9754 ?        S      1:06 /usr/sbin/httpd
30692 ?        Z      0:31 [httpd <defunct>]
21284 ?        Z      0:32 [httpd <defunct>]
 8589 ?        Z      0:32 [httpd <defunct>]
28034 ?        Z      0:34 [httpd <defunct>]
28154 ?        Z      0:32 [httpd <defunct>]
32331 ?        Z      0:37 [httpd <defunct>]
 6490 ?        Z      0:31 [httpd <defunct>]
 1844 ?        Z      0:34 [httpd <defunct>]
23605 ?        Z      0:30 [httpd <defunct>] 

Strangely it sits in this state for a few minutes, sometimes up to an hour, and
sometimes recovers on its own with all httpd processes happily serving again. 
Several hours later it fails again in this manner and repeats.

How reproducible:

Steps to Reproduce:
1. Fill up 2GB log file with Apache 2.0.
2. Pound it with constant moderate-heavy traffic.

Actual Results:
httpd failure with bunch of child zombie processes.  Somehow recovers on its
own.  Fails and recovers repeatedly.

Expected Results:
Shouldn't fail at all.

Comment 1 Joe Orton 2002-11-25 14:39:05 UTC
Are you getting any error_log entries?

Just to be clear, are all the httpd child processes zombies in this situation? A
"ps axf | grep httpd" might be useful; better yet, enabling the /server-status
page and seeing what that gives.

Comment 2 Warren Togami 2002-11-30 22:29:59 UTC
[Fri Nov 29 16:21:11 2002] [notice] child pid 8714 exit signal Segmentation faul
t (11)
[Fri Nov 29 16:23:48 2002] [notice] child pid 8922 exit signal Segmentation faul
t (11)

These error_log entries occurred a few hours _before_ the logs overflowed last
night.  These temporary zombie failure occurr intermittently so it is difficult
to see it happening.  The above processes are ALL the processes running within
that security context (using Linux Virtual Server with security contexts kernel
patch http://www.solucorp.qc.ca/miscprj/s_context.hc).

Hmm, I am no longer certain this failure is caused by full logs.

Next time it fails, what can I do to help diagnose this?

Comment 3 Warren Togami 2002-12-24 05:15:31 UTC
Seeing problems with httpd failing in two instances.
1. 2GB limit of log files
2. During log rotate

These issues may or may not be related.

Comment 4 Panu Matilainen 2003-06-10 08:09:23 UTC
Another, related issue is that apache doesn't currently (up to RH9 ) support
*transferring* files over 2GB either. Just noticed it, trying to download a
custom RH9 DVD image... Simply rebuilding httpd with LFS (-DFILE_OFFSET_BITS=-64
etc...) doesn't work either since it makes it segfault on startup but I guess
that's to be expected having seen the comments in in bug 69520.

Comment 5 Joe Orton 2005-01-25 16:46:08 UTC
Thanks for the report.  This is a mass bug update; since this release
of Red Hat Linux is no longer supported, please either:

a) try and reproduce the bug with a supported version of Red Hat
Enterprise Linux or Fedora Core, and re-open this bug as appropriate
after changing the Product field, or,

b) if relevant, try and reproduce this bug using the current version
of the upstream package, and report the bug upstream.

c) report the bug to the Fedora Legacy project who may wish to
continue maintenance of this package.

Note You need to log in before you can comment on or make changes to this bug.