Hide Forgot
Description of problem: Python application based on python-krbV is leaking fds in /var/tmp/ directory named HTTP_XXX, until it eventually reaches 1024 limit and stops working. After attaching gdb I could see that these are opened by ./src/lib/krb5/rcache/rc_io.c:krb5_rc_io_open_internal() After digging a bit further I found following code at ./src/lib/krb5/krb/rd_req_dec.c:rd_req_decoded_opt() which seems to be responsible for opening this fd, when KRB5_AUTH_CONTEXT_DO_TIME flag is present: ------------------------------------------------------------------------------ if (((*auth_context)->rcache == NULL) && ((*auth_context)->auth_context_flags & KRB5_AUTH_CONTEXT_DO_TIME) && server) { if ((retval = krb5_get_server_rcache(context, krb5_princ_component(context,server,0), &(*auth_context)->rcache))) ------------------------------------------------------------------------------ fd opened in this way ^^ doesn't seem to be ever closed. One workaround I found is to create/attach rcache to AuthContext explicitly, then problem goes away. Version-Release number of selected component (if applicable): krb5-libs-1.10.3-42.el6.x86_64 krb5-libs-1.10.3-42.el6.i686 krb5-devel-1.10.3-42.el6.x86_64 krb5-debuginfo-1.10.3-42.el6.x86_64 krb5-workstation-1.10.3-42.el6.x86_64 python-krbV-1.0.90-3.el6.x86_64 How reproducible: 100% Steps to Reproduce: Make an rd_req() call where AuthContext has KRB5_AUTH_CONTEXT_DO_TIME flag set and at same time it doesn't set any rcache via krb5_auth_con_setrcache(). (I'm sorry, but I don't have a reproducer in C.) Actual results: rcache fd is opened every time rd_req() is called when KRB5_AUTH_CONTEXT_DO_TIME flag is present and there's no rcache attached to AuthContext. Expected results: rcache fds should not pile up Additional info: https://git.fedorahosted.org/cgit/python-krbV.git This is what python source on server side looks like: ------------------------------------------------------------------------------ def login_krbv(self, krb_request, remote_addr): context = krbV.default_context() principal = self.krb_auth_principal keytabfile = self.krb_auth_keytabfile server_principal = krbV.Principal(name = principal, context = context) server_keytab = krbV.Keytab(name = keytabfile, context = context) auth_context = krbV.AuthContext(context = context) auth_context.flags = (krbV.KRB5_AUTH_CONTEXT_DO_SEQUENCE | krbV.KRB5_AUTH_CONTEXT_DO_TIME) hostname = principal.split("/")[1] auth_context.addrs = (socket.gethostbyname(hostname), 0, socket.gethostbyname(remote_addr), 0) # decode and read the authentication request decoded_request = base64.decodestring(krb_request) auth_tuple = context.rd_req(decoded_request, server = server_principal, keytab = server_keytab, auth_context = auth_context, options = krbV.AP_OPTS_MUTUAL_REQUIRED) auth_context, opts, server_principal, cache_credentials = auth_tuple cprinc = cache_credentials[2] # remove @REALM and return username return cprinc.name.split("@")[0] ------------------------------------------------------------------------------ And the workaround is: ------------------------------------------------------------------------------ diff --git a/server/autobkr/server/auth.py b/server/autobkr/server/auth.py index 9ec9f203ba9e..c18027a0ab25 100644 --- a/server/autobkr/server/auth.py +++ b/server/autobkr/server/auth.py @@ -38,6 +38,8 @@ class KrbAuth(): auth_context = krbV.AuthContext(context = context) auth_context.flags = (krbV.KRB5_AUTH_CONTEXT_DO_SEQUENCE | krbV.KRB5_AUTH_CONTEXT_DO_TIME) + auth_context.rcache = context.default_rcache() + hostname = principal.split("/")[1] ------------------------------------------------------------------------------
I just realized, that maybe this is because python doesn't release auth_context objects immediately. And "context.default_rcache()" works around that because it keeps single instance around through entire life-time.
(In reply to Jan Stancek from comment #2) > I just realized, that maybe this is because python doesn't release > auth_context objects immediately. And "context.default_rcache()" works > around that because it keeps single instance around through entire life-time. That makes sense to me. Adding in python-krbV to hopefully provide more information.
After call rd_req, auth_context never gets released - gdb never triggers on krb5_auth_con_free(). I added following to end of login_krbv() ... import gc import sys gc.collect() print 'auth_context', auth_context print str(sys.getrefcount(auth_context)) print str(auth_context) print str(gc.get_referrers(auth_context)) And this is what I get: auth_context <krbV.AuthContext instance at 0x2c2ec20> 2 <krbV.AuthContext instance at 0x2c2ec20> [<frame object at 0x7f766400a250>] I don't get the first referrer. It looks like it's referencing itself. Perhaps it's coming from this code? static PyObject* Context_rd_req(PyObject *unself, PyObject *args, PyObject *kw) ... if(auth_context) { Py_INCREF(auth_context); }
Here's a reproducer that should demonstrate the problem: -------------------------------------------------------------------------- import krbV import sys import gc def login_krbv(): context = krbV.default_context() auth_context = krbV.AuthContext(context = context) for i in range(0, 3): print sys.getrefcount(auth_context._ac) print gc.get_referrers(auth_context._ac) auth_context.flags = (krbV.KRB5_AUTH_CONTEXT_DO_SEQUENCE | krbV.KRB5_AUTH_CONTEXT_DO_TIME) login_krbv() -------------------------------------------------------------------------- python reproducer.py 2 [{'_ac': <PyCObject object at 0x7f9c882ec378>, 'context': <krbV.Context instance at 0x7f9c88236758>}] 3 [{'_ac': <PyCObject object at 0x7f9c882ec378>, 'context': <krbV.Context instance at 0x7f9c88236758>}] 4 [{'_ac': <PyCObject object at 0x7f9c882ec378>, 'context': <krbV.Context instance at 0x7f9c88236758>}] -------------------------------------------------------------------------- If I run this via gdb with breakpoint set on destroy_ac(), it never triggers. If I comment the 2 lines with "auth_context.flags = ...", breakpoint triggers. Or with ltrace: # ltrace --dl python reproducer.py 2>&1 | grep destroy_ac # # # After commenting out flags line # ltrace --dl python reproducer.py 2>&1 | grep destroy_ac (11597) destroy_ac I'm guessing this is PyObject_GetAttrString() in Context_setattr() grabbing new reference that is never Py_DECREF-ed. If I add that, reproducer above no longer works: AuthContext_setattr(PyObject *unself __UNUSED, PyObject *args) ... tmp = PyObject_GetAttrString(self, "_ac"); if(tmp) { ac = PyCObject_AsVoidPtr(tmp); Py_DECREF(tmp); }
Red Hat Enterprise Linux 6 is in the Production 3 Phase. During the Production 3 Phase, Critical impact Security Advisories (RHSAs) and selected Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available. The official life cycle policy can be reviewed here: http://redhat.com/rhel/lifecycle This issue does not meet the inclusion criteria for the Production 3 Phase and will be marked as CLOSED/WONTFIX. If this remains a critical requirement, please contact Red Hat Customer Support to request a re-evaluation of the issue, citing a clear business justification. Note that a strong business justification will be required for re-evaluation. Red Hat Customer Support can be contacted via the Red Hat Customer Portal at the following URL: https://access.redhat.com/