Bug 622003 - Calling program crashes when loading OpenSSL CHIL Engine twice
Summary: Calling program crashes when loading OpenSSL CHIL Engine twice
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: openssl   
(Show other bugs)
Version: 5.4
Hardware: All
OS: Linux
Target Milestone: rc
: ---
Assignee: Tomas Mraz
QA Contact: BaseOS QE Security Team
Depends On:
TreeView+ depends on / blocked
Reported: 2010-08-06 18:18 UTC by Sander Temme
Modified: 2018-11-14 17:31 UTC (History)
7 users (show)

Fixed In Version: openssl-0.9.8e-19.el5
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2011-07-21 07:40:41 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:1010 normal SHIPPED_LIVE openssl bug fix and enhancement update 2011-07-20 15:44:40 UTC

Description Sander Temme 2010-08-06 18:18:44 UTC
Description of problem:

The CHIL Engine (used to access Thales/nCipher cryptographic hardware) sets an ex_data table entry in OpenSSL, with a function pointer to the cleanup function located in the CHIL engine binary.

When a calling program unloads the Engine, the cleanup function pointer is not cleared.  When the calling program loads the Engine again, a second function pointer is added to the ex_data cleanup stack, leaving the first one in place but likely pointing to invalid memory.  This crashes the calling program as soon as it attempts to clean up the ex_data  entry.  

Version-Release number of selected component (if applicable):
Any OpenSSL 0.9.8 RPM.
Any OpenSSL version from openssl.org

How reproducible:

Apache 2.2.x, as supplied by Red Hat or downloaded from apache.org, does this double library load and triggers this issue any time the Engine library text does not land in the same memory location second time around, which is most of the time.

Steps to Reproduce:
1. Install Thales nCipher HSM
2. Install nCSS Software
3. Add /opt/nfast/toolkits/hwcrhk to /etc/ld.so.conf.d/nfast, run ldconfig -l (if I recall)
4. Edit /etc/httpd/conf/httpd.conf (if I recall) to add SSLCryptoDevice chil anywhere
5. /etc/init.d/httpd start.  
Actual results:

httpd server segfaults.

Expected results:

httpd server does not segfault.

Additional info:

This patch to the openssl upstream fixes the issue:


--- e_chil.c	2010/03/24 23:42:30
+++ e_chil.c	2010/05/26 16:16:49
@@ -111,11 +111,10 @@
 /* RSA stuff */
 static int hwcrhk_rsa_mod_exp(BIGNUM *r, const BIGNUM *I, RSA *rsa, BN_CTX *ctx);
 /* This function is aliased to mod_exp (with the mont stuff dropped). */
 static int hwcrhk_mod_exp_mont(BIGNUM *r, const BIGNUM *a, const BIGNUM *p,
 		const BIGNUM *m, BN_CTX *ctx, BN_MONT_CTX *m_ctx);
+static int hwcrhk_rsa_finish(RSA *rsa);
 #ifndef OPENSSL_NO_DH
@@ -135,10 +134,6 @@
 	UI_METHOD *ui_method, void *callback_data);
 static EVP_PKEY *hwcrhk_load_pubkey(ENGINE *eng, const char *key_id,
 	UI_METHOD *ui_method, void *callback_data);
-static void hwcrhk_ex_free(void *obj, void *item, CRYPTO_EX_DATA *ad,
-	int ind,long argl, void *argp);
 /* Interaction stuff */
 static int hwcrhk_insert_card(const char *prompt_info,
@@ -193,7 +188,7 @@
+	hwcrhk_rsa_finish,
@@ -603,7 +598,7 @@
 	if (hndidx_rsa == -1)
 		hndidx_rsa = RSA_get_ex_new_index(0,
 			"nFast HWCryptoHook RSA key handle",
-			NULL, NULL, hwcrhk_ex_free);
 	return 1;
@@ -1081,6 +1076,21 @@
 	return hwcrhk_mod_exp(r, a, p, m, ctx);
+static int hwcrhk_rsa_finish(RSA *rsa)
+	{
+	HWCryptoHook_RSAKeyHandle *hptr;
+	int ret;
+	hptr = RSA_get_ex_data(rsa, hndidx_rsa);
+	if (hptr)
+                {
+                ret = p_hwcrhk_RSAUnloadKey(*hptr, NULL);
+                OPENSSL_free(hptr);
+		RSA_set_ex_data(rsa, hndidx_rsa, NULL);
+                }
+	return 1;
+	}
 #ifndef OPENSSL_NO_DH
@@ -1139,34 +1149,6 @@
 	return 1;
-/* This cleans up an RSA KM key, called when ex_data is freed */
-static void hwcrhk_ex_free(void *obj, void *item, CRYPTO_EX_DATA *ad,
-	int ind,long argl, void *argp)
-	char tempbuf[1024];
-	HWCryptoHook_ErrMsgBuf rmsg;
-	HWCryptoHook_RSAKeyHandle *hptr;
-#if !defined(OPENSSL_NO_RSA)
-	int ret;
-	rmsg.buf = tempbuf;
-	rmsg.size = sizeof(tempbuf);
-	hptr = (HWCryptoHook_RSAKeyHandle *) item;
-	if(hptr)
-                {
-                ret = p_hwcrhk_RSAUnloadKey(*hptr, NULL);
-                OPENSSL_free(hptr);
-                }
 /* Mutex calls: since the HWCryptoHook model closely follows the POSIX model
  * these just wrap the POSIX functions and add some logging.

Comment 2 Sander Temme 2010-10-26 17:30:23 UTC
What are your plans regarding this issue?  I would like to do a product Integration Guide using the base OS OpenSSL and Apache, but this is a blocking issue for that.  --Thanks

Comment 3 Tomas Mraz 2010-10-26 17:49:19 UTC
The plan is to get the fix into a future Red Hat Enterprise Linux 5 update release.

Comment 7 RHEL Product and Program Management 2011-01-11 20:01:53 UTC
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.

Comment 8 Sander Temme 2011-01-11 22:30:20 UTC
(In reply to comment #7)

When you say "next release", do you mean six, seven, or a future update to five?

Comment 9 RHEL Product and Program Management 2011-01-11 23:18:07 UTC
This request was erroneously denied for the current release of
Red Hat Enterprise Linux.  The error has been fixed and this
request has been re-proposed for the current release.

Comment 10 Sander Temme 2011-01-11 23:55:31 UTC
(In reply to comment #9)

Thank you, I am happy to hear that.

Comment 16 Mark J. Cox 2011-05-11 13:09:39 UTC
Confirmed,  openssl-0.9.8e-20.el5 fixes this issue.  I simply used 'SSLCryptoDevice chil' in conf.d/ssl.conf and started server with 
LD_LIBRARY_PATH=/opt/nfast/toolkits/hwcrhk/ /usr/sbin/httpd

Before -20 the server would segfault and not start up, with -20 the server starts up and SSL requests get handled by nCipher hardware (verified by watching hardserver.log)

Comment 18 errata-xmlrpc 2011-07-21 07:40:41 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.