Bug 622003 - Calling program crashes when loading OpenSSL CHIL Engine twice
Calling program crashes when loading OpenSSL CHIL Engine twice
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: openssl (Show other bugs)
All Linux
low Severity medium
: rc
: ---
Assigned To: Tomas Mraz
BaseOS QE Security Team
Depends On:
  Show dependency treegraph
Reported: 2010-08-06 14:18 EDT by Sander Temme
Modified: 2011-07-21 03:40 EDT (History)
7 users (show)

See Also:
Fixed In Version: openssl-0.9.8e-19.el5
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2011-07-21 03:40:41 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Sander Temme 2010-08-06 14:18:44 EDT
Description of problem:

The CHIL Engine (used to access Thales/nCipher cryptographic hardware) sets an ex_data table entry in OpenSSL, with a function pointer to the cleanup function located in the CHIL engine binary.

When a calling program unloads the Engine, the cleanup function pointer is not cleared.  When the calling program loads the Engine again, a second function pointer is added to the ex_data cleanup stack, leaving the first one in place but likely pointing to invalid memory.  This crashes the calling program as soon as it attempts to clean up the ex_data  entry.  

Version-Release number of selected component (if applicable):
Any OpenSSL 0.9.8 RPM.
Any OpenSSL version from openssl.org

How reproducible:

Apache 2.2.x, as supplied by Red Hat or downloaded from apache.org, does this double library load and triggers this issue any time the Engine library text does not land in the same memory location second time around, which is most of the time.

Steps to Reproduce:
1. Install Thales nCipher HSM
2. Install nCSS Software
3. Add /opt/nfast/toolkits/hwcrhk to /etc/ld.so.conf.d/nfast, run ldconfig -l (if I recall)
4. Edit /etc/httpd/conf/httpd.conf (if I recall) to add SSLCryptoDevice chil anywhere
5. /etc/init.d/httpd start.  
Actual results:

httpd server segfaults.

Expected results:

httpd server does not segfault.

Additional info:

This patch to the openssl upstream fixes the issue:


--- e_chil.c	2010/03/24 23:42:30
+++ e_chil.c	2010/05/26 16:16:49
@@ -111,11 +111,10 @@
 /* RSA stuff */
 static int hwcrhk_rsa_mod_exp(BIGNUM *r, const BIGNUM *I, RSA *rsa, BN_CTX *ctx);
 /* This function is aliased to mod_exp (with the mont stuff dropped). */
 static int hwcrhk_mod_exp_mont(BIGNUM *r, const BIGNUM *a, const BIGNUM *p,
 		const BIGNUM *m, BN_CTX *ctx, BN_MONT_CTX *m_ctx);
+static int hwcrhk_rsa_finish(RSA *rsa);
 #ifndef OPENSSL_NO_DH
@@ -135,10 +134,6 @@
 	UI_METHOD *ui_method, void *callback_data);
 static EVP_PKEY *hwcrhk_load_pubkey(ENGINE *eng, const char *key_id,
 	UI_METHOD *ui_method, void *callback_data);
-static void hwcrhk_ex_free(void *obj, void *item, CRYPTO_EX_DATA *ad,
-	int ind,long argl, void *argp);
 /* Interaction stuff */
 static int hwcrhk_insert_card(const char *prompt_info,
@@ -193,7 +188,7 @@
+	hwcrhk_rsa_finish,
@@ -603,7 +598,7 @@
 	if (hndidx_rsa == -1)
 		hndidx_rsa = RSA_get_ex_new_index(0,
 			"nFast HWCryptoHook RSA key handle",
-			NULL, NULL, hwcrhk_ex_free);
 	return 1;
@@ -1081,6 +1076,21 @@
 	return hwcrhk_mod_exp(r, a, p, m, ctx);
+static int hwcrhk_rsa_finish(RSA *rsa)
+	{
+	HWCryptoHook_RSAKeyHandle *hptr;
+	int ret;
+	hptr = RSA_get_ex_data(rsa, hndidx_rsa);
+	if (hptr)
+                {
+                ret = p_hwcrhk_RSAUnloadKey(*hptr, NULL);
+                OPENSSL_free(hptr);
+		RSA_set_ex_data(rsa, hndidx_rsa, NULL);
+                }
+	return 1;
+	}
 #ifndef OPENSSL_NO_DH
@@ -1139,34 +1149,6 @@
 	return 1;
-/* This cleans up an RSA KM key, called when ex_data is freed */
-static void hwcrhk_ex_free(void *obj, void *item, CRYPTO_EX_DATA *ad,
-	int ind,long argl, void *argp)
-	char tempbuf[1024];
-	HWCryptoHook_ErrMsgBuf rmsg;
-	HWCryptoHook_RSAKeyHandle *hptr;
-#if !defined(OPENSSL_NO_RSA)
-	int ret;
-	rmsg.buf = tempbuf;
-	rmsg.size = sizeof(tempbuf);
-	hptr = (HWCryptoHook_RSAKeyHandle *) item;
-	if(hptr)
-                {
-                ret = p_hwcrhk_RSAUnloadKey(*hptr, NULL);
-                OPENSSL_free(hptr);
-                }
 /* Mutex calls: since the HWCryptoHook model closely follows the POSIX model
  * these just wrap the POSIX functions and add some logging.
Comment 2 Sander Temme 2010-10-26 13:30:23 EDT
What are your plans regarding this issue?  I would like to do a product Integration Guide using the base OS OpenSSL and Apache, but this is a blocking issue for that.  --Thanks
Comment 3 Tomas Mraz 2010-10-26 13:49:19 EDT
The plan is to get the fix into a future Red Hat Enterprise Linux 5 update release.
Comment 7 RHEL Product and Program Management 2011-01-11 15:01:53 EST
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.
Comment 8 Sander Temme 2011-01-11 17:30:20 EST
(In reply to comment #7)

When you say "next release", do you mean six, seven, or a future update to five?
Comment 9 RHEL Product and Program Management 2011-01-11 18:18:07 EST
This request was erroneously denied for the current release of
Red Hat Enterprise Linux.  The error has been fixed and this
request has been re-proposed for the current release.
Comment 10 Sander Temme 2011-01-11 18:55:31 EST
(In reply to comment #9)

Thank you, I am happy to hear that.
Comment 16 Mark J. Cox (Product Security) 2011-05-11 09:09:39 EDT
Confirmed,  openssl-0.9.8e-20.el5 fixes this issue.  I simply used 'SSLCryptoDevice chil' in conf.d/ssl.conf and started server with 
LD_LIBRARY_PATH=/opt/nfast/toolkits/hwcrhk/ /usr/sbin/httpd

Before -20 the server would segfault and not start up, with -20 the server starts up and SSL requests get handled by nCipher hardware (verified by watching hardserver.log)
Comment 18 errata-xmlrpc 2011-07-21 03:40:41 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.