Bug 76967

Summary: httpd exits (crashes?) after 10-20 hours uptime
Product: [Retired] Red Hat Linux Reporter: Joe Bayes <jbayes>
Component: httpdAssignee: Joe Orton <jorton>
Status: CLOSED WONTFIX QA Contact: Brian Brock <bbrock>
Severity: high Docs Contact:
Priority: medium    
Version: 8.0CC: chris.ricker, jabapi
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2005-01-25 16:36:35 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
error log
none
access log none

Description Joe Bayes 2002-10-30 10:59:30 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020827

Description of problem:
I start httpd (from /etc/init.d) and it appears to work properly. A day or two
later, I notice that httpd is no longer running. Error logs show 5 lines like:

[Mon Oct 28 10:47:56 2002] [warn] child process 18591 still did not exit,
sending a SIGTERM

and then one line:

[Mon Oct 28 10:48:00 2002] [notice] caught SIGTERM, shutting down

This has happened twice so far; I have not been able to keep httpd up for a full
24 hours yet. 





Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. start apache
2. wait 10-20 hours, perhaps hitting the site several times in that period
3. Observe that eventually, apache shuts down/crashes/quits. 
	

Additional info:

Comment 1 Joe Bayes 2002-10-30 11:04:03 UTC
Created attachment 82665 [details]
error log

Comment 2 Joe Bayes 2002-10-30 11:05:17 UTC
Created attachment 82666 [details]
access log

Comment 3 Joe Bayes 2002-10-30 11:08:15 UTC
Oops, I forgot to say:
httpd-2.0.40-8


Comment 4 Joe Orton 2002-11-01 14:04:10 UTC
It looks like something on your system has sent a SIGTERM signal to the httpd
process; are you sure nobody else has run "service httpd stop" or anything on
this system?

Comment 5 Joe Bayes 2002-11-01 20:12:57 UTC
Yes, fairly sure. I'm the only one with root here, and I figure a script kiddie
would probably have found something more interesting to do, had they gotten in. 

If nobody else has this problem, then it's probably something screwey with my
setup, but I figured I'd post the bug report and see if anybody else was having
the same problem. I'll wait another couple of weeks before going back to 1.3.

Thanks.

Comment 6 Chris Ricker 2002-12-08 23:09:48 UTC
I've been seeing Apache crash similarly (though less frequently -- maybe twice
in the past month) on my relatively non-busy personal web server with the same
SIGTERM message in the log

Comment 7 Joe Orton 2002-12-10 10:12:44 UTC
Could be the httpd parent is calling kill(0, SIGTERM) by accident on some edge
case - I'll add some extra logging to try and track this down.

Comment 8 Janne Pikkarainen 2002-12-10 10:16:54 UTC
I have three test servers running RH 8.0 and Apache 2.0. One of them has some
stability issues with Apache, it crashes once a week. The only difference I know
is that the crashy server is also running X (since it's also my desktop computer
running 24/7), the others are not.

I think this has to be related with logrotate (it's rotating the logs once a
week for me), since I'm not running any special cron jobs but Apache still
crashes every single weekend. Other than that it's stable as a rock.

Comment 9 Joe Orton 2002-12-10 10:21:09 UTC
To clarify: you're saying it consistently crashes at the weekend, and that you
have set logrotate to only run at the weekend?  (I wouldn't call this a "crash"
as such since it's really a controlled shutdown, except we're not sure who is in
control)


Comment 10 Joe Orton 2002-12-10 10:23:10 UTC
(logrotate shouldn't be SIGTERM'ing httpd so that would be a weird correlation;
logrotate sends SIGHUP to httpd)

Comment 11 Janne Pikkarainen 2002-12-10 11:56:11 UTC
Could be a controlled shutdown, but the last line in Apache's error_log is 

---
[notice] SIGHUP received.  Attempting to restart
---

After that Apache dies, for some reason SIGHUP kills it. httpd.conf, files in
/etc/httpd/conf.d and php.ini are in their original state.

Yes, logrotate runs itself weekly, only at the weekend.

Since this box is a not-so-often used testbox and I can easily resolve this
Apache-hickup by restarting it every Monday, I haven't paid too much attention
to this.

But what bugs me most is that those another two servers running without X are
not showing up any sign of instability. I will shutdown X this weekend and see
if Apache dies without a running X session. I also increased Apache's debug
level so we'll see what's going under its hood. Maybe. 

Any suggestions how to further debug this are welcome.

Comment 12 Joe Orton 2002-12-10 12:03:58 UTC
Jaba: that looks like a different problem to that being tracked here; here,
there is no evidence of a SIGHUP in the error log, just a shutdown due to a
SIGTERM.  Can you open another bug "SIGHUP causes server shutdown" and attach
your error_log to that?

Comment 13 Janne Pikkarainen 2002-12-10 12:04:19 UTC
Just investigated the logs bit more and spotted something strange:

---
[Sun Dec 8 04:02:09 2002] [warn] child process 600 still did not exit, sending a
SIGTERM
[Sun Dec 8 04:02:10 2002] [warn] child process 601 still did not exit, sending a
SIGTERM
[Sun Dec 8 04:02:10 2002] [warn] child process 602 still did not exit, sending a
SIGTERM
[Sun Dec 8 04:02:10 2002] [warn] child process 603 still did not exit, sending a
SIGTERM
[Sun Dec 8 04:02:10 2002] [warn] child process 604 still did not exit, sending a
SIGTERM
[Sun Dec 8 04:02:10 2002] [warn] child process 605 still did not exit, sending a
SIGTERM
[Sun Dec 8 04:02:10 2002] [warn] child process 606 still did not exit, sending a
SIGTERM
[Sun Dec 8 04:02:10 2002] [warn] child process 607 still did not exit, sending a
SIGTERM
[Sun Dec 8 04:01:54 2002] [notice] SIGHUP received.  Attempting to restart
---

And yes, this is the exact order of events in error_log. SIGHUP was received
about 15 seconds before child processes started to die, but this SIGHUP line is
the last line in error_log... strange.

Comment 14 Joe Orton 2005-01-25 16:36:35 UTC
Thanks for the report.  This is a mass bug update; since this release
of Red Hat Linux is no longer supported, please either:

a) try and reproduce the bug with a supported version of Red Hat
Enterprise Linux or Fedora Core, and re-open this bug as appropriate
after changing the Product field, or,

b) if relevant, try and reproduce this bug using the current version
of the upstream package, and report the bug upstream, or,

c) report the bug to the Fedora Legacy project who may wish to
continue maintenance of this package.  http://www.fedoralegacy.org/