Bug 1009995

Summary:	Memory leak in FIPS_selftest_aes_ccm and FIPS_selftest_aes_gcm
Product:	Red Hat Enterprise Linux 6	Reporter:	Alicja Kario <hkario>
Component:	openssl	Assignee:	Tomas Mraz <tmraz>
Status:	CLOSED ERRATA	QA Contact:	Alicja Kario <hkario>
Severity:	low	Docs Contact:
Priority:	low
Version:	6.5	CC:	dmalcolm, ebenes, jrieden, pvrabec, raud, sforsber, tlavigne, zmraz
Target Milestone:	rc	Keywords:	Regression
Target Release:	6.5
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openssl-1.0.1e-12.el6	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	803520
Clones:	1014653 (view as bug list)		Environment:
Last Closed:	2013-11-21 09:25:35 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	659881, 803520
Bug Blocks:	993793, 1014653

Comment 1 Alicja Kario 2013-09-19 16:28:20 UTC

Using m2crypto-0.20.2-9.el6.ppc64

Run:

cat > aescryptleak.py<<_EOF_
import M2Crypto

key = M2Crypto.__m2crypto.aes_new()
M2Crypto.__m2crypto.AES_set_key(key, '\x00' * 16, 16 * 8, 1)
M2Crypto.__m2crypto.AES_set_key(key, '\x00' * 16, 16 * 8, 1)
M2Crypto.__m2crypto.AES_set_key(key, '\x00' * 16, 16 * 8, 1)
M2Crypto.__m2crypto.AES_set_key(key, '\x00' * 16, 16 * 8, 1)
M2Crypto.__m2crypto.AES_set_key(key, '\x00' * 16, 16 * 8, 1)
M2Crypto.__m2crypto.AES_set_key(key, '\x00' * 16, 16 * 8, 1)

M2Crypto.__m2crypto.AES_crypt(key, '\x00' * 16, 16, 1)
M2Crypto.__m2crypto.AES_crypt(key, '\x00' * 16, 16, 1)
M2Crypto.__m2crypto.AES_crypt(key, '\x00' * 16, 16, 1)
M2Crypto.__m2crypto.AES_crypt(key, '\x00' * 16, 16, 1)
M2Crypto.__m2crypto.AES_crypt(key, '\x00' * 16, 16, 1)

M2Crypto.__m2crypto.AES_free(key)
_EOF_

Under valgrind:
valgrind python aescryptleak.py

Result:

==10027== Memcheck, a memory error detector
==10027== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==10027== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==10027== Command: python aescryptleak.py
==10027== 
==10027== 
==10027== HEAP SUMMARY:
==10027==     in use at exit: 2,244,174 bytes in 13,682 blocks
==10027==   total heap usage: 83,263 allocs, 69,581 frees, 12,665,548 bytes allocated
==10027== 
==10027== LEAK SUMMARY:
==10027==    definitely lost: 1,016 bytes in 2 blocks
==10027==    indirectly lost: 0 bytes in 0 blocks
==10027==      possibly lost: 17,328 bytes in 38 blocks
==10027==    still reachable: 2,225,830 bytes in 13,642 blocks
==10027==         suppressed: 0 bytes in 0 blocks
==10027== Rerun with --leak-check=full to see details of leaked memory
==10027== 
==10027== For counts of detected and suppressed errors, rerun with: -v
==10027== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 110 from 8)

Expected:
No definitely lost bytes.

Comment 3 Alicja Kario 2013-09-20 12:18:01 UTC

The bug is not present in RHEL6.5-20130905.1 compose while it's present in RHEL6.5-20130913.0 compose. Between those composes m2crypto didn't change, both of them have m2crypto-0.20.2-9.el6 package.

The package that did change was openssl. Trace from valgrind confirms it:

==1670== 680 bytes in 1 blocks are definitely lost in loss record 2,384 of 2,711
==1670==    at 0x4A069EE: malloc (vg_replace_malloc.c:270)
==1670==    by 0xB7EEA2D: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB86E6D8: EVP_CipherInit_ex (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB8D91BF: FIPS_selftest_aes_gcm (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB8DC1B1: FIPS_selftest (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB8DAD04: FIPS_module_mode_set (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB7F11A9: FIPS_mode_set (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB7F123C: OPENSSL_init_library (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB8E0875: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB7ED3B7: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0x7FF000679: ???
==1670==    by 0x6C74707972637364: ???


==1670== 336 bytes in 1 blocks are definitely lost in loss record 2,248 of 2,711
==1670==    at 0x4A069EE: malloc (vg_replace_malloc.c:270)
==1670==    by 0xB7EEA2D: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB86E6D8: EVP_CipherInit_ex (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB8D948F: FIPS_selftest_aes_ccm (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB8DC1A7: FIPS_selftest (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB8DAD04: FIPS_module_mode_set (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB7F11A9: FIPS_mode_set (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB7F123C: OPENSSL_init_library (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB8E0875: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0xB7ED3B7: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==1670==    by 0x7FF000679: ???
==1670==    by 0x6C74707972637364: ???

Comment 4 Alicja Kario 2013-09-20 12:22:16 UTC

above traces were obtained using openssl-1.0.1e-11.el6.x86_64 package

Comment 5 Alicja Kario 2013-09-20 12:33:22 UTC

Got the same leaks by just running openssl speed:

==1720== HEAP SUMMARY:
==1720==     in use at exit: 1,464 bytes in 9 blocks
==1720==   total heap usage: 4,442,373 allocs, 4,442,364 frees, 392,701,289 bytes allocated
==1720== 
==1720== 336 bytes in 1 blocks are definitely lost in loss record 8 of 9
==1720==    at 0x4A069EE: malloc (vg_replace_malloc.c:270)
==1720==    by 0x4EEEA2D: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4F6E6D8: EVP_CipherInit_ex (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4FD948F: FIPS_selftest_aes_ccm (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4FDC1A7: FIPS_selftest (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4FDAD04: FIPS_module_mode_set (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4EF11A9: FIPS_mode_set (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4EF123C: OPENSSL_init_library (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4FE0875: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4EED3B7: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x7FF000682: ???
==1720==    by 0x4F48006465657072: ???
==1720== 
==1720== 680 bytes in 1 blocks are definitely lost in loss record 9 of 9
==1720==    at 0x4A069EE: malloc (vg_replace_malloc.c:270)
==1720==    by 0x4EEEA2D: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4F6E6D8: EVP_CipherInit_ex (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4FD91BF: FIPS_selftest_aes_gcm (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4FDC1B1: FIPS_selftest (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4FDAD04: FIPS_module_mode_set (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4EF11A9: FIPS_mode_set (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4EF123C: OPENSSL_init_library (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4FE0875: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x4EED3B7: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==1720==    by 0x7FF000682: ???
==1720==    by 0x4F48006465657072: ???
==1720== 
==1720== LEAK SUMMARY:
==1720==    definitely lost: 1,016 bytes in 2 blocks
==1720==    indirectly lost: 0 bytes in 0 blocks
==1720==      possibly lost: 0 bytes in 0 blocks
==1720==    still reachable: 448 bytes in 7 blocks
==1720==         suppressed: 0 bytes in 0 blocks

Comment 8 errata-xmlrpc 2013-11-21 09:25:35 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1585.html