Bug 521743

Summary: -HUP httpd with mod_python cause Virt/Res to increase
Product: Red Hat Enterprise Linux 5 Reporter: masanari iida <masanari_iida>
Component: mod_pythonAssignee: Joe Orton <jorton>
Status: CLOSED CANTFIX QA Contact: BaseOS QE <qe-baseos-auto>
Severity: medium Docs Contact:
Priority: low    
Version: 5.4   
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-10-13 14:11:18 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description masanari iida 2009-09-08 04:47:40 UTC 
Description of problem:
An appache server with mod_python loaded cause memory leak
when I do "service httpd reload".

Version-Release number of selected component (if applicable):
httpd-2.2.3-31.el5
python-2.4.3-27.el5
mod_python-3.2.8-3.1

How reproducible:
Always

Steps to Reproduce:
1. Install httpd, python and mod_python RPMs.
2. LoadModule python_module is enabled in /etc/httpd/conf.d/python.conf
3. Start httpd service.
4. top -bn1 |grep httpd
5. service httpd reload

Continue step 4 and 5 many times.
  
Actual results:
top(1) shows Virt and Res size of the httpd processes are
increasing.  

Just after started httpd.
# top -bn1 |grep httpd
14847 root      18   0 17996 9368 3972 S  0.0  0.9   0:00.34 httpd
14923 apache    15   0 17996 6248  832 S  0.0  0.6   0:00.00 httpd
14924 apache    15   0 17996 6248  832 S  0.0  0.6   0:00.00 httpd
14925 apache    15   0 17996 6248  832 S  0.0  0.6   0:00.00 httpd
14926 apache    19   0 17996 6248  832 S  0.0  0.6   0:00.00 httpd
14927 apache    19   0 17996 6248  832 S  0.0  0.6   0:00.00 httpd
14928 apache    19   0 17996 6248  832 S  0.0  0.6   0:00.00 httpd
14929 apache    19   0 17996 6248  832 S  0.0  0.6   0:00.00 httpd
14930 apache    24   0 17996 6248  832 S  0.0  0.6   0:00.00 httpd

After 50 times of httpd reload.
# top -bn1 |grep httpd
14847 root      18   0 59124  39m 3980 S  0.0  4.0   0:08.25 httpd
15823 apache    17   0 59124  36m  832 S  0.0  3.7   0:00.00 httpd
15824 apache    17   0 59124  36m  832 S  0.0  3.7   0:00.00 httpd
15825 apache    17   0 59124  36m  832 S  0.0  3.7   0:00.00 httpd
15826 apache    19   0 59124  36m  832 S  0.0  3.7   0:00.00 httpd
15827 apache    19   0 59124  36m  832 S  0.0  3.7   0:00.00 httpd
15828 apache    21   0 59124  36m  832 S  0.0  3.7   0:00.00 httpd
15829 apache    21   0 59124  36m  832 S  0.0  3.7   0:00.00 httpd
15830 apache    23   0 59124  36m  832 S  0.0  3.7   0:00.00 httpd
15836 root      15   0  5476 1268 1032 S  0.0  0.1   0:00.00 httpd
15841 root      20   0 12380 3844 2584 R  0.0  0.4   0:00.00 httpd

After 150 times of httpd reload.
# top -bn1 |grep httpd
14847 root      18   0  142m 104m 3984 S  0.0 10.5   0:48.49 httpd
17722 apache    17   0  142m 101m  832 S  0.0 10.2   0:00.00 httpd
17723 apache    18   0  142m 101m  832 S  0.0 10.2   0:00.00 httpd
17724 apache    19   0  142m 101m  832 S  0.0 10.2   0:00.00 httpd
17725 apache    19   0  142m 101m  832 S  0.0 10.2   0:00.00 httpd
17726 apache    21   0  142m 101m  832 S  0.0 10.2   0:00.00 httpd
17727 apache    22   0  142m 101m  832 S  0.0 10.2   0:00.00 httpd
17728 apache    24   0  142m 101m  832 S  0.0 10.2   0:00.00 httpd
17729 apache    24   0  142m 101m  832 S  0.0 10.2   0:00.00 httpd

Expected results:
The Virt/Res size should be keep smaller size even after many httpd reload.

Additional info:
This system is NOT using mod_python, but mod_python and python were
installed as a default. So mod_python was enabled without my attention.
This symptom was found during "512 reload cause httpd + mod_perl to crash" reproduce test on RHEL5.
The symptom I was testing was not happened on RHEL5, though I found
the other one.
Comment 1 Joe Orton 2009-10-13 14:11:18 UTC 
I'm sorry to say that there seems to be little we can do to fix this.

The Python interpreter is designed with certain parts of process state which cannot be cleaned up, causing memory leaks over time if the server is frequently SIGHUPed (which in turn, causes the Python interpreter to be reloaded).

I would recommend, in order of preference:

1) do not load mod_python in configurations in which it is not used.

2) occassionally perform a "service httpd restart" to completely stop and start the server.

3) execute python scripts via an out-of-process interpreter, e.g. via CGI.

I hope these answers are of some use.