Bug 659881

Summary: Memory leak in m2crypto-0.16/SWIG/_aes.i: AES_crypt
Product: Red Hat Enterprise Linux 5 Reporter: Dave Malcolm <dmalcolm>
Component: m2cryptoAssignee: Miloslav Trmač <mitr>
Status: CLOSED ERRATA QA Contact: BaseOS QE Security Team <qe-baseos-security>
Severity: low Docs Contact:
Priority: low    
Version: 5.6CC: jrieden, raud
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: m2crypto-0.16-8.el5 Doc Type: Bug Fix
Doc Text:
Prior to this update, the AES_crypt() function did not free a temporary buffer. This caused a memory leak when the function was called repeatedly. This problem has been fixed and the AES_crypt() function now frees memory correctly.
Story Points: ---
Clone Of:
: 803520 (view as bug list) Environment:
Last Closed: 2011-07-21 11:23:01 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 803520, 1009995    

Description Dave Malcolm 2010-12-03 23:06:03 UTC
Description of problem:

In the following code in m2crypto-0.16/SWIG/_aes.i
    59  /* 
    60  // op == 0: decrypt
    61  // otherwise: encrypt (Python code will supply the value 1.)
    62  */
    63  PyObject *AES_crypt(const AES_KEY *key, PyObject *in, int outlen, int op) {
    64      const void *buf;
    65      int len;
    66      unsigned char *out;
    67  
    68      if (PyObject_AsReadBuffer(in, &buf, &len) == -1)
    69          return NULL;
    70  
    71      if (!(out=(unsigned char *)PyMem_Malloc(outlen))) {
    72          PyErr_SetString(PyExc_MemoryError, "AES_crypt");
    73          return NULL;
    74      }
    75      if (op == 0)
    76          AES_encrypt((const unsigned char *)in, out, key);
    77      else
    78          AES_decrypt((const unsigned char *)in, out, key);
    79      return PyString_FromStringAndSize(out, outlen);
    80  }

A buffer of "outlen" bytes is claimed as "out" at line 71:

    71      if (!(out=(unsigned char *)PyMem_Malloc(outlen))) {

but it is never freed (assuming I'm reading this correctly), thus leaking "outlen" bytes each time.



Version-Release number of selected component (if applicable):
m2crypto-0.16-6.el5.6

Comment 1 Dave Malcolm 2010-12-03 23:42:53 UTC
Leak still present in latest upstream code:
  http://svn.osafoundation.org/m2crypto/trunk/SWIG/_aes.i

Comment 2 Miloslav Trmač 2010-12-13 22:32:38 UTC
Thanks for your report, filed upstream as https://bugzilla.osafoundation.org/show_bug.cgi?id=12999 .

Comment 7 Eliska Slobodova 2011-06-24 14:25:44 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Prior to this update, the AES_crypt() function did not free a temporary buffer. This caused a memory leak when the function was called repeatedly. This problem has been fixed and the AES_crypt() function now frees memory correctly.

Comment 8 errata-xmlrpc 2011-07-21 11:23:01 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-1058.html