Bug 1267176 - Memory leak / possible DoS with krb auth. [rhel 7.2]
Memory leak / possible DoS with krb auth. [rhel 7.2]
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: sssd (Show other bugs)
7.1
Unspecified Unspecified
urgent Severity urgent
: rc
: 7.2
Assigned To: SSSD Maintainers
Kaushik Banerjee
: Security, SecurityTracking
Depends On:
Blocks: CVE-2015-5292
  Show dependency treegraph
 
Reported: 2015-09-29 04:50 EDT by Anthony Russell
Modified: 2015-11-19 06:40 EST (History)
18 users (show)

See Also:
Fixed In Version: sssd-1.13.0-38.el7
Doc Type: Bug Fix
Doc Text:
Applications performing Kerberos authentication previously increased the memory footprint of the Kerberos plug-in that parses the Privilege Attribute Certificate (PAC) information. The plug-in has been updated to free the memory it allocates, thus fixing this bug.
Story Points: ---
Clone Of: 1266404
Environment:
Last Closed: 2015-11-19 06:40:48 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
FedoraHosted SSSD 2803 None None None Never

  None (edit)
Description Anthony Russell 2015-09-29 04:50:51 EDT
Description of problem:

Back port request for upstream fix at https://fedorahosted.org/sssd/ticket/2803#comment:3 to RHEL7

From there:
We believe there is a memory leak in the sssd_pac_plugin (sssd_pac_plugin.so library) distributed with the sssd_client package. 

The issue was revealed in one of our daemons using kerberos 5 authentication and leaking gigabytes of memory after several days (1GB/day on average for 10 authentications per second). Putting it in valgrind, we found the following :

==27734== 1,344 (+1,344) (32 (+32) direct, 1,312 (+1,312) indirect) bytes in 1 (+1) blocks are definitely lost in loss record 484 of 540
==27734==    at 0x4A06A2E: malloc (vg_replace_malloc.c:270)
==27734==    by 0x50D96C2: krb5_pac_init (pac.c:231)
==27734==    by 0x50D9D2D: krb5_pac_parse (pac.c:332)
==27734==    by 0x19C70CA6: sssdpac_verify (sssd_pac.c:144)
==27734==    by 0x50C1E52: krb5int_authdata_verify (authdata.c:617)
==27734==    by 0x50E19CF: rd_req_decoded_opt (rd_req_dec.c:437)
==27734==    by 0x50E1A59: krb5_rd_req_decoded (rd_req_dec.c:599)
==27734==    by 0x50E0A88: krb5_rd_req (rd_req.c:87)
==27734==    by 0x4E49770: kg_accept_krb5 (accept_sec_context.c:643)
==27734==    by 0x4E4B0A9: krb5_gss_accept_sec_context_ext (accept_sec_context.c:1338)
==27734==    by 0x4E4B208: krb5_gss_accept_sec_context (accept_sec_context.c:1367)
==27734==    by 0x4E3AC22: gss_accept_sec_context (g_accept_sec_context.c:203)
==27734==    by 0x19A6C728: Csec_server_establish_context_ext_KRB5 (Csec_plugin_GSS.c:482)
==27734==    by 0x834EBC2: Csec_server_establish_context_ext_caller (Csec_plugin.h:129)
==27734==    by 0x835416A: Csec_server_establish_context_ext (Csec_api.c:277)
==27734==    by 0x8353FE4: Csec_server_establishContext (Csec_api.c:218)
==27734==    by 0x406407: doit (Cns_main.c:747)
==27734==    by 0x8129420: _Cpool_starter (Cpool.c:377)
==27734==    by 0x8124914: _Cthread_start_pthread (Cthread.c:209)
==27734==    by 0x30CCA07A50: start_thread (pthread_create.c:301)
==27734==    by 0x30CC6E89AC: clone (clone.S:115)

The study of the corresponding code clearly shows a genuine bug introduced in sssd_client via commit 92af6f25 in May 2013. Here are the details :

    in krb5int_authdata_verify (authdata.c:617), so krb5 code, the plugin is called :

       code = (*module->ftable->verify)(kcontext,
                                        context,
                                        module->plugin_context,
                                        *(module->request_context_pp),
                                        auth_context,
                                        key,
                                        ap_req);

    this calls sssdpac_verify (sssd_pac.c:144) and the code looks like this (I've dropped irrelevant lines and commented) :

         krb5_pac pac;   // krb5_pac is actually a pointer type
         ...
         kerr = krb5_pac_parse(kcontext, sssdctx->data.data,    // allocation of the pac pointer from input data
                               sssdctx->data.length, &pac);
         ...
         kerr = krb5_pac_verify(kcontext, pac,                  // verification of the pac content
                                req->ticket->enc_part2->times.authtime,
                                req->ticket->enc_part2->client, key, NULL);
         ...           // no reference to pac
         return 0;     // memory leak!

    I did check the code of parse and verify, and they are doing what you expect, so no hidden ref count or anything special. 

So according to all this, the patch to the leak is the following :

diff --git a/src/sss_client/sssd_pac.c b/src/sss_client/sssd_pac.c
index 1939f61..1d1fe8a 100644
--- a/src/sss_client/sssd_pac.c
+++ b/src/sss_client/sssd_pac.c
@@ -150,6 +150,10 @@ static krb5_error_code sssdpac_verify(krb5_context kcontext,
     kerr = krb5_pac_verify(kcontext, pac,
                            req->ticket->enc_part2->times.authtime,
                            req->ticket->enc_part2->client, key, NULL);
+    // deallocate pac - fixes memory leak reported in bug ...
+    krb5_pac_free(kcontext, pac);
+    pac = NULL;
+    // check result of the verification
     if (kerr != 0) {
         /* The krb5 documentation says:
          * A checksum mismatch can occur if the PAC was copied from a


Version-Release number of selected component (if applicable):

This has been found on a machine running the following versions of kerberos and sssd :
   krb5-workstation-1.12.2-15.el7_1.x86_64
   krb5-libs-1.12.2-15.el7_1.x86_64
   krb5-libs-1.12.2-15.el7_1.i686
   sssd-client-1.12.2-58.el7_1.17.x86_64
Comment 1 Jakub Hrozek 2015-09-30 08:38:14 EDT
Upstream ticket:
https://fedorahosted.org/sssd/ticket/2803
Comment 2 Jakub Hrozek 2015-09-30 08:53:56 EDT
 master: b4c44ebb8997d3debb33607c123ccfd9926e0cba
Comment 6 Steeve Goveas 2015-10-12 12:31:47 EDT
Verified sanity only in sssd version

[root@ipaclient02 ~]# rpm -q sssd
sssd-1.13.0-39.el7.x86_64

[root@ipaclient02 ~]# service sssd stop; rm -f /var/lib/sss/{mc,db}/*; service sssd start
Redirecting to /bin/systemctl stop  sssd.service
Redirecting to /bin/systemctl start  sssd.service

[root@ipaclient02 ~]# klist
klist: Credentials cache keyring 'persistent:0:0' not found

[root@ipaclient02 ~]# echo Secret123 | kinit aduser@ADLABS.COM
Password for aduser@ADLABS.COM: 

[root@ipaclient02 ~]# ssh -vvv -K -l aduser@adlabs.com `hostname`
OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy -p 22 ipaclient02.labs02.test
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: permanently_drop_suid: 0
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug2: fd 5 setting O_NONBLOCK
debug2: fd 4 setting O_NONBLOCK
debug3: load_hostkeys: loading entries for host "ipaclient02.labs02.test" from file "/root/.ssh/known_hosts"
.
.
.
debug3: Ignored env _
debug3: Ignored env OLDPWD
debug2: channel 0: request shell confirm 1
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Last login: Mon Oct 12 21:54:25 2015 from ipaclient02.labs01.test
Could not chdir to home directory /home/adlabs.com/aduser: No such file or directory

-sh-4.2$ id
uid=1436801930(aduser@adlabs.com) gid=1436801930(aduser@adlabs.com) groups=1436801930(aduser@adlabs.com),1436800513(domain users@adlabs.com) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
Comment 7 errata-xmlrpc 2015-11-19 06:40:48 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-2355.html

Note You need to log in before you can comment on or make changes to this bug.