| Summary: | rcache leaks fd when using AuthContext with KRB5_AUTH_CONTEXT_DO_TIME | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Jan Stancek <jstancek> |
| Component: | python-krbV | Assignee: | John Dennis <jdennis> |
| Status: | CLOSED WONTFIX | QA Contact: | Namita Soman <nsoman> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.7 | CC: | dcallagh, dpal, jburke, jplans, nalin, pkis, rharwood |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-12-06 10:31:17 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
I just realized, that maybe this is because python doesn't release auth_context objects immediately. And "context.default_rcache()" works around that because it keeps single instance around through entire life-time. (In reply to Jan Stancek from comment #2) > I just realized, that maybe this is because python doesn't release > auth_context objects immediately. And "context.default_rcache()" works > around that because it keeps single instance around through entire life-time. That makes sense to me. Adding in python-krbV to hopefully provide more information. After call rd_req, auth_context never gets released - gdb never triggers on krb5_auth_con_free().
I added following to end of login_krbv()
...
import gc
import sys
gc.collect()
print 'auth_context', auth_context
print str(sys.getrefcount(auth_context))
print str(auth_context)
print str(gc.get_referrers(auth_context))
And this is what I get:
auth_context <krbV.AuthContext instance at 0x2c2ec20>
2
<krbV.AuthContext instance at 0x2c2ec20>
[<frame object at 0x7f766400a250>]
I don't get the first referrer. It looks like it's referencing itself. Perhaps it's coming from this code?
static PyObject*
Context_rd_req(PyObject *unself, PyObject *args, PyObject *kw)
...
if(auth_context)
{
Py_INCREF(auth_context);
}
Here's a reproducer that should demonstrate the problem:
--------------------------------------------------------------------------
import krbV
import sys
import gc
def login_krbv():
context = krbV.default_context()
auth_context = krbV.AuthContext(context = context)
for i in range(0, 3):
print sys.getrefcount(auth_context._ac)
print gc.get_referrers(auth_context._ac)
auth_context.flags = (krbV.KRB5_AUTH_CONTEXT_DO_SEQUENCE |
krbV.KRB5_AUTH_CONTEXT_DO_TIME)
login_krbv()
--------------------------------------------------------------------------
python reproducer.py
2
[{'_ac': <PyCObject object at 0x7f9c882ec378>, 'context': <krbV.Context instance at 0x7f9c88236758>}]
3
[{'_ac': <PyCObject object at 0x7f9c882ec378>, 'context': <krbV.Context instance at 0x7f9c88236758>}]
4
[{'_ac': <PyCObject object at 0x7f9c882ec378>, 'context': <krbV.Context instance at 0x7f9c88236758>}]
--------------------------------------------------------------------------
If I run this via gdb with breakpoint set on destroy_ac(), it never triggers.
If I comment the 2 lines with "auth_context.flags = ...", breakpoint triggers.
Or with ltrace:
# ltrace --dl python reproducer.py 2>&1 | grep destroy_ac
#
# # After commenting out flags line
# ltrace --dl python reproducer.py 2>&1 | grep destroy_ac
(11597) destroy_ac
I'm guessing this is PyObject_GetAttrString() in Context_setattr() grabbing new reference that is never Py_DECREF-ed. If I add that, reproducer above no longer works:
AuthContext_setattr(PyObject *unself __UNUSED, PyObject *args)
...
tmp = PyObject_GetAttrString(self, "_ac");
if(tmp) {
ac = PyCObject_AsVoidPtr(tmp);
Py_DECREF(tmp);
}
Red Hat Enterprise Linux 6 is in the Production 3 Phase. During the Production 3 Phase, Critical impact Security Advisories (RHSAs) and selected Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available. The official life cycle policy can be reviewed here: http://redhat.com/rhel/lifecycle This issue does not meet the inclusion criteria for the Production 3 Phase and will be marked as CLOSED/WONTFIX. If this remains a critical requirement, please contact Red Hat Customer Support to request a re-evaluation of the issue, citing a clear business justification. Note that a strong business justification will be required for re-evaluation. Red Hat Customer Support can be contacted via the Red Hat Customer Portal at the following URL: https://access.redhat.com/ |
Description of problem: Python application based on python-krbV is leaking fds in /var/tmp/ directory named HTTP_XXX, until it eventually reaches 1024 limit and stops working. After attaching gdb I could see that these are opened by ./src/lib/krb5/rcache/rc_io.c:krb5_rc_io_open_internal() After digging a bit further I found following code at ./src/lib/krb5/krb/rd_req_dec.c:rd_req_decoded_opt() which seems to be responsible for opening this fd, when KRB5_AUTH_CONTEXT_DO_TIME flag is present: ------------------------------------------------------------------------------ if (((*auth_context)->rcache == NULL) && ((*auth_context)->auth_context_flags & KRB5_AUTH_CONTEXT_DO_TIME) && server) { if ((retval = krb5_get_server_rcache(context, krb5_princ_component(context,server,0), &(*auth_context)->rcache))) ------------------------------------------------------------------------------ fd opened in this way ^^ doesn't seem to be ever closed. One workaround I found is to create/attach rcache to AuthContext explicitly, then problem goes away. Version-Release number of selected component (if applicable): krb5-libs-1.10.3-42.el6.x86_64 krb5-libs-1.10.3-42.el6.i686 krb5-devel-1.10.3-42.el6.x86_64 krb5-debuginfo-1.10.3-42.el6.x86_64 krb5-workstation-1.10.3-42.el6.x86_64 python-krbV-1.0.90-3.el6.x86_64 How reproducible: 100% Steps to Reproduce: Make an rd_req() call where AuthContext has KRB5_AUTH_CONTEXT_DO_TIME flag set and at same time it doesn't set any rcache via krb5_auth_con_setrcache(). (I'm sorry, but I don't have a reproducer in C.) Actual results: rcache fd is opened every time rd_req() is called when KRB5_AUTH_CONTEXT_DO_TIME flag is present and there's no rcache attached to AuthContext. Expected results: rcache fds should not pile up Additional info: https://git.fedorahosted.org/cgit/python-krbV.git This is what python source on server side looks like: ------------------------------------------------------------------------------ def login_krbv(self, krb_request, remote_addr): context = krbV.default_context() principal = self.krb_auth_principal keytabfile = self.krb_auth_keytabfile server_principal = krbV.Principal(name = principal, context = context) server_keytab = krbV.Keytab(name = keytabfile, context = context) auth_context = krbV.AuthContext(context = context) auth_context.flags = (krbV.KRB5_AUTH_CONTEXT_DO_SEQUENCE | krbV.KRB5_AUTH_CONTEXT_DO_TIME) hostname = principal.split("/")[1] auth_context.addrs = (socket.gethostbyname(hostname), 0, socket.gethostbyname(remote_addr), 0) # decode and read the authentication request decoded_request = base64.decodestring(krb_request) auth_tuple = context.rd_req(decoded_request, server = server_principal, keytab = server_keytab, auth_context = auth_context, options = krbV.AP_OPTS_MUTUAL_REQUIRED) auth_context, opts, server_principal, cache_credentials = auth_tuple cprinc = cache_credentials[2] # remove @REALM and return username return cprinc.name.split("@")[0] ------------------------------------------------------------------------------ And the workaround is: ------------------------------------------------------------------------------ diff --git a/server/autobkr/server/auth.py b/server/autobkr/server/auth.py index 9ec9f203ba9e..c18027a0ab25 100644 --- a/server/autobkr/server/auth.py +++ b/server/autobkr/server/auth.py @@ -38,6 +38,8 @@ class KrbAuth(): auth_context = krbV.AuthContext(context = context) auth_context.flags = (krbV.KRB5_AUTH_CONTEXT_DO_SEQUENCE | krbV.KRB5_AUTH_CONTEXT_DO_TIME) + auth_context.rcache = context.default_rcache() + hostname = principal.split("/")[1] ------------------------------------------------------------------------------