From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040510 Description of problem: LLNL has run into what appears to be a memory leak in python, or at least an extremely poor garbage collector in the version we ship. Here's their initial report, which is Issue Tracker #43734 "I have some more information about the Python memory leak problem. I was able to track down at least one python memory leak that appears to be a problem for xwatch-lustre. There is a bug in the pickling module, and I found a bug report about it here: http://sourceforge.net/tracker/index.php?func=detail&aid=770997&group_id=5470&atid=105470 Unfortunately I don't see any action taken on the bug. The reproducer listed there causes a memory leak on our CHAOS 2.0 clusters." Version-Release number of selected component (if applicable): python-2.2.3-5 How reproducible: Always Steps to Reproduce: 1. run attached test case on a RHEL3 system 2. 3. Actual Results: running test case leaks memory Expected Results: memory doesn't leak Additional info: this same test case running on RHL 7.3, 8.0, and FC2 doesn't exhibit this problem.
Created attachment 101981 [details] test case
some output from running the attached test case under RHEL3: [dave@sideshowbob dave]$ ./pickle.py & [1] 6655 [dave@sideshowbob dave]$ ps aux | grep pickle dave 6655 99.9 1.1 5452 4372 pts/1 R 16:09 0:03 /usr/bin/python ./pickle.py dave 6657 0.0 0.1 3692 704 pts/1 S+ 16:09 0:00 grep pickle [dave@sideshowbob dave]$ ps aux | grep pickle dave 6655 99.9 1.9 8580 7472 pts/1 R 16:09 0:09 /usr/bin/python ./pickle.py dave 6659 0.0 0.1 3684 688 pts/1 R+ 16:09 0:00 grep pickle [dave@sideshowbob dave]$ ps aux | grep pickle dave 6655 99.9 2.6 11232 10156 pts/1 R 16:09 0:12 /usr/bin/python ./pickle.py dave 6661 0.0 0.1 3688 700 pts/1 S+ 16:09 0:00 grep pickle [dave@sideshowbob dave]$ ps aux | grep pickle dave 6655 99.9 2.9 12348 11196 pts/1 R 16:09 0:14 /usr/bin/python ./pickle.py dave 6663 0.0 0.1 3680 692 pts/1 R+ 16:09 0:00 grep pickle [dave@sideshowbob dave]$ ps aux | grep pickle dave 6655 99.9 3.2 13340 12304 pts/1 R 16:09 0:17 /usr/bin/python ./pickle.py dave 6665 0.0 0.1 3688 688 pts/1 R+ 16:09 0:00 grep pickle [dave@sideshowbob dave]$
Upstream fixed this problem. The next version after the one we ship doesn't have this problem. LLNL's work around for the moment has been to install that version of python rather than RH's
I have backported cPickle.py from python 2.3.4, with as little modifications as possible (had to change some things to make it compile). Packages are available at: ftp://people.redhat.com/misa/python-fixes/cPickle/ Please make sure LLNL is aware this is not something supported by the python community. Availability of this package in mainstream Python is something we have to decide on.
May I ask why not use the workaround posted in the upstream bug report? (and that is, use the binary flag for duping the pickle)
Because modifying code that works fine on other releases, newer or older, is not regarded as the best way to fix the problem?
I'm going to close this because nothing happened in the last 3 years, and I can't see anything happening for RHEL-3 at this point.