Red Hat Bugzilla – Bug 460692
m2crypto leaks fds into GC
Last modified: 2009-09-02 07:19:56 EDT
Description of problem: This is basically a copy and paste of the problem from urllib2: http://bugs.python.org/issue3066 ...in m2urllib2, you do: r.recv = r.read ...which creates a circular reference, I've tested the fix I posted in m2crypto and it seems to work (although I'm not convinced it's the best fix). This would be nice to fix for 5.3.
Thanks for your report. Does the leak have any noticeable user impact?
Yes; using reposync (from yum-utils) it is possible to overrun the 1024 FDs allowed by default ulimit settings and crash the process.
It's worth noting that the generic fix has gone into the 5.3 errata, so putting it into m2crypto wouldn't be a deviation from core python urllib2 in RHEL-5.
Reproducer for M2Crypto: import gc import M2Crypto o = M2Crypto.m2urllib2.build_opener() r = o.open('https://addons.mozilla.org') s = [r.fp._sock.fp._sock] r.close() print gc.get_referrers(s[0]) ... should print only [[<M2Crypto.SSL.Connection.Connection instance ... >]], not including a <socket._fileobject> instance Upstream urllib2 solves this by ignoring the circular reference, and explicitly disconnecting r.fp from r on r.close(): see http://svn.python.org/view/python/trunk/Lib/urllib2.py?rev=53511&r1=52481&r2=53511 . This drops other references and eventually closes the underlying socket - but only if the user of "r" calls r.close() on it. AFAICS reposync uses urlgrab, which always calls r.close().
Created attachment 330905 [details] Patch against 0.19.1 - applies against the RHEL5 version as well The attached patch will be available in rawhide m2crypto-0.19.1-5. Can you test the patch in the scenario where the circular reference actually causes problems, please?
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2009-1351.html