Description of problem: When softhsm is used via openssl->engine(pkcs11) there is a double free segfault on EC keys Version-Release number of selected component (if applicable): softhsm-2.3.0-4.fc28.x86_64 openssl-pkcs11-0.4.8-1.fc28.x86_64 p11-kit-0.23.12-1.fc28.x86_64 How reproducible: Always Steps to Reproduce: 1. create softhsm token with an EC key pair 2. openssl req -new -engine pkcs11 -keyform ENGINE -key 'pkcs11:token=MyToken;object=MyECKey' -subj /CN=Testing Actual results: engine "pkcs11" set. -----BEGIN CERTIFICATE REQUEST----- MIHMMHQCAQAwEjEQMA4GA1UEAwwHVGVzdGluZzBZMBMGByqGSM49AgEGCCqGSM49 AwEHA0IABNeUPwRKsBqva4vOvEtFvTNZNdm7g0zW/QqwdKDuZJGSQM3JdiQdfwEP PT2UljVLEVHhk9v+wyI3dvFXMaERiMSgADAKBggqhkjOPQQDAgNIADBFAiEAy80L 9LdEJ98qFQnJrpEDJnWiuJtQxJ+wLzol782yOawCIEzpdYz+ynOPWdv4Rx9KZ2Oy GYTc0ENS2a2edvDERVuB -----END CERTIFICATE REQUEST----- double free or corruption (out) Aborted (core dumped) Expected results: Req is generated without core dump Additional info: The command is completed by the core dump occurs in the cleanup #0 0x00007f992bb36feb in raise () from /lib64/libc.so.6 #1 0x00007f992bb215c1 in abort () from /lib64/libc.so.6 #2 0x00007f992bb799d7 in __libc_message () from /lib64/libc.so.6 #3 0x00007f992bb7feac in malloc_printerr () from /lib64/libc.so.6 #4 0x00007f992bb81910 in _int_free () from /lib64/libc.so.6 #5 0x00007f992b16c056 in HandleManager::sessionClosed(unsigned long) () from /usr/lib64/pkcs11/libsofthsm2.so #6 0x00007f992b129851 in SoftHSM::C_CloseSession(unsigned long) () from /usr/lib64/pkcs11/libsofthsm2.so #7 0x00007f992b10feb8 in C_CloseSession () from /usr/lib64/pkcs11/libsofthsm2.so #8 0x00007f992b5f5531 in managed_C_CloseSession () from /usr/lib64/p11-kit-proxy.so #9 0x00007f992b60ed24 in binding_C_CloseSession () from /usr/lib64/p11-kit-proxy.so #10 0x00007f992b3b5e35 in ffi_closure_unix64_inner () from /lib64/libffi.so.6 #11 0x00007f992b3b61a6 in ffi_closure_unix64 () from /lib64/libffi.so.6 #12 0x00007f992b5e54f4 in proxy_C_CloseSession () from /usr/lib64/p11-kit-proxy.so #13 0x00007f992b5e5680 in proxy_C_CloseAllSessions () from /usr/lib64/p11-kit-proxy.so #14 0x00007f992b60ed44 in binding_C_CloseAllSessions () from /usr/lib64/p11-kit-proxy.so #15 0x00007f992b3b5e35 in ffi_closure_unix64_inner () from /lib64/libffi.so.6 #16 0x00007f992b3b61a6 in ffi_closure_unix64 () from /lib64/libffi.so.6 #17 0x00007f992b8f8fac in pkcs11_release_slot () from /usr/lib64/engines-1.1/pkcs11.so #18 0x00007f992b8f93df in pkcs11_release_all_slots () from /usr/lib64/engines-1.1/pkcs11.so #19 0x00007f992b8f2332 in ctx_finish () from /usr/lib64/engines-1.1/pkcs11.so #20 0x00007f992b8f04dc in engine_finish () from /usr/lib64/engines-1.1/pkcs11.so #21 0x00007f992c62f17b in engine_unlocked_finish (e=0x55a4b1bc5080, unlock_for_handlers=<optimized out>) at crypto/engine/eng_init.c:60 #22 0x00007f992c631016 in int_cleanup_cb_doall (p=0x55a4b1bc4660) at crypto/engine/eng_table.c:176 #23 0x00007f992c659c09 in doall_util_fn (arg=0x0, func_arg=0x0, func=0x7f992c630ff0 <int_cleanup_cb_doall>, use_arg=0, lh=0x55a4b1bc4570) at crypto/lhash/lhash.c:192 #24 OPENSSL_LH_doall (lh=0x55a4b1bc4570, func=func@entry=0x7f992c630ff0 <int_cleanup_cb_doall>) at crypto/lhash/lhash.c:200 #25 0x00007f992c631341 in lh_ENGINE_PILE_doall ( doall=0x7f992c630ff0 <int_cleanup_cb_doall>, lh=<optimized out>) at crypto/engine/eng_int.h:177 #26 engine_table_cleanup (table=0x7f992c980f98 <rsa_table>) at crypto/engine/eng_table.c:184 #27 0x00007f992c62f3aa in engine_cleanup_cb_free (item=0x55a4b1bc4b40) at crypto/engine/eng_lib.c:153 #28 0x00007f992c6b0c60 in OPENSSL_sk_pop_free (st=0x55a4b1bb9380, func=0x7f992c62f3a0 <engine_cleanup_cb_free>) at crypto/stack/stack.c:265 #29 0x00007f992c62f6e0 in sk_ENGINE_CLEANUP_ITEM_pop_free ( freefunc=0x7f992c62f3a0 <engine_cleanup_cb_free>, sk=<optimized out>) at crypto/engine/eng_int.h:62 #30 engine_cleanup_int () at crypto/engine/eng_lib.c:160 #31 0x00007f992c657cfe in OPENSSL_cleanup () at crypto/init.c:490 #32 0x00007f992bb3972c in __run_exit_handlers () from /lib64/libc.so.6 #33 0x00007f992bb3985c in exit () from /lib64/libc.so.6 #34 0x000055a4afeebe06 in main (argc=<optimized out>, argv=<optimized out>)
RSA keys are ok: $ openssl req -config openssl.cnf -new -engine pkcs11 -keyform ENGINE -key 'pkcs11:token=MyToken;object=MyRSAKey' -subj /CN=Testing engine "pkcs11" set. -----BEGIN CERTIFICATE REQUEST----- MIICVzCCAT8CAQAwEjEQMA4GA1UEAwwHVGVzdGluZzCCASIwDQYJKoZIhvcNAQEB BQADggEPADCCAQoCggEBAMQCx8lrIvTxD7YQ/bwAxAuBOvit+sVTzUd32dAyeRSD 4pbL0m6cab9sqGWJcPfHj2DVeI3/YtjUVj8riXsZLDy8on4JXO21Jl4SbQ0/mkbG uTXBIHSrRWeZge2O8M3LjALEBJh+pV/los3a0o66Q+YUVVlVvzuUjUQXUXVzUzwR SddG+Ny2fo5H0Mt5R15La2MnrTg5+vXNWCsvqjnyvb6WfmGmevcnpw3i6eqKJa+I Tms4ixX1nZyDipUCPifjqsEFpLXixiwBpzyPDJAApWVelcruJwrO8d3cq2EXqitd cEVKmhKrx7/9M++2LzVF/Rnh2qsYTVK01q5PqsDStMECAwEAAaAAMA0GCSqGSIb3 DQEBCwUAA4IBAQCgHvTetV924UqhAwEvx+eJUZnTTTpPUnzsR7mZDIbP4c5XNqyT 4gw2S2ra0ji9T693zi14J9JJ8ZvL4qP7e/mR4h41xM66PfpPCrz2xSdPDXzCmf/p ka0ftp2McdNknG14G/xTz7VISsF1CpZpeAceck4ivJXS4rlzlsjBJ04zYVJSyKAC FBjXRXsWDzdt9HQGkL6Ks95Cq2N4cFX84/YFLWr63kAAoq4YQECTomxdnx4s/exl u5pipZuOW8uMuN9q7yQSKat/WJCjHFSsmAKbvDYsrZAcGUNodVdGvyFdqjLgPTEB FnWv8sjBJex0d5NdAqy/hGel1KM5w3BEaQWr -----END CERTIFICATE REQUEST-----
Another test: use an EC key from another HSM but keep softhsm configured; openssl req -config openssl.cnf -new -engine pkcs11 -keyform ENGINE -key 'pkcs11:token=AnotherHSM;object=AnotherECKey' -subj /CN=Testing engine "pkcs11" set. Missing CKA_ALWAYS_AUTHENTICATE attribute -----BEGIN CERTIFICATE REQUEST----- MIHLMHQCAQAwEjEQMA4GA1UEAwwHVGVzdGluZzBZMBMGByqGSM49AgEGCCqGSM49 AwEHA0IABOpqCdmKYRNh8Ep7btTU83xFG7NDp9Ra3OAnoUR+8dCWV1yN3Db/m+OH u4DdwEtgOLngEcRRWYLImyvZBb++G1ugADAKBggqhkjOPQQDAgNHADBEAiBt0vVj XjOQnTkm3kfbyaNOANg4b7T8RFDW0JCBbJn+bQIgE3TIhSmYBdU4c0TJUxWlPwYv sZc2+Q0VUqmmBv0wv60= -----END CERTIFICATE REQUEST----- Segmentation fault (core dumped) #0 0x0000000000000080 in ?? () #1 0x00007f90497c4744 in SoftHSM::C_Finalize(void*) () from /usr/lib64/pkcs11/libsofthsm2.so #2 0x00007f90497abbe8 in C_Finalize () from /usr/lib64/pkcs11/libsofthsm2.so #3 0x00007f9049c91027 in finalize_module_inlock_reentrant () from /usr/lib64/p11-kit-proxy.so #4 0x00007f9049c911d9 in managed_C_Finalize () from /usr/lib64/p11-kit-proxy.so #5 0x00007f9049caaaf4 in binding_C_Finalize () from /usr/lib64/p11-kit-proxy.so #6 0x00007f9049a51e35 in ffi_closure_unix64_inner () from /lib64/libffi.so.6 #7 0x00007f9049a521a6 in ffi_closure_unix64 () from /lib64/libffi.so.6 #8 0x00007f9049c93521 in p11_kit_modules_finalize () from /usr/lib64/p11-kit-proxy.so #9 0x00007f9049c8073d in proxy_free () from /usr/lib64/p11-kit-proxy.so #10 0x00007f9049c80825 in proxy_C_Finalize () from /usr/lib64/p11-kit-proxy.so #11 0x00007f9049caaaf4 in binding_C_Finalize () from /usr/lib64/p11-kit-proxy.so #12 0x00007f9049a51e35 in ffi_closure_unix64_inner () from /lib64/libffi.so.6 #13 0x00007f9049a521a6 in ffi_closure_unix64 () from /lib64/libffi.so.6 #14 0x00007f9049f922bf in pkcs11_CTX_unload () from /usr/lib64/engines-1.1/pkcs11.so #15 0x00007f9049f8e34f in ctx_finish () from /usr/lib64/engines-1.1/pkcs11.so #16 0x00007f9049f8c4dc in engine_finish () from /usr/lib64/engines-1.1/pkcs11.so #17 0x00007f904accb17b in engine_unlocked_finish (e=0x560e3c41f090, unlock_for_handlers=<optimized out>) at crypto/engine/eng_init.c:60 #18 0x00007f904accd016 in int_cleanup_cb_doall (p=0x560e3c41e690) at crypto/engine/eng_table.c:176 #19 0x00007f904acf5c09 in doall_util_fn (arg=0x0, func_arg=0x0, func=0x7f904acccff0 <int_cleanup_cb_doall>, use_arg=0, lh=0x560e3c41e570) at crypto/lhash/lhash.c:192 #20 OPENSSL_LH_doall (lh=0x560e3c41e570, func=func@entry=0x7f904acccff0 <int_cleanup_cb_doall>) at crypto/lhash/lhash.c:200 #21 0x00007f904accd341 in lh_ENGINE_PILE_doall ( doall=0x7f904acccff0 <int_cleanup_cb_doall>, lh=<optimized out>) at crypto/engine/eng_int.h:177 #22 engine_table_cleanup (table=0x7f904b01cf98 <rsa_table>) at crypto/engine/eng_table.c:184 #23 0x00007f904accb3aa in engine_cleanup_cb_free (item=0x560e3c41eb40) at crypto/engine/eng_lib.c:153 #24 0x00007f904ad4cc60 in OPENSSL_sk_pop_free (st=0x560e3c413380, func=0x7f904accb3a0 <engine_cleanup_cb_free>) at crypto/stack/stack.c:265 #25 0x00007f904accb6e0 in sk_ENGINE_CLEANUP_ITEM_pop_free ( freefunc=0x7f904accb3a0 <engine_cleanup_cb_free>, ---Type <return> to continue, or q <return> to quit--- sk=<optimized out>) at crypto/engine/eng_int.h:62 #26 engine_cleanup_int () at crypto/engine/eng_lib.c:160 #27 0x00007f904acf3cfe in OPENSSL_cleanup () at crypto/init.c:490 #28 0x00007f904a1d572c in __run_exit_handlers () from /lib64/libc.so.6 #29 0x00007f904a1d585c in exit () from /lib64/libc.so.6 #30 0x0000560e3be89e06 in main (argc=<optimized out>, argv=<optimized out>) at apps/openssl.c:266
It's apparently because of the order of atexit() cleanups: the engine's finish() method vs SoftHSM's C++ destructors for static variables. If you tweak it somehow e.g., by setting LD_PRELOAD like this or recompiling softhsm with -fno-use-cxa-atexit, you could bypass the issue: LD_PRELOAD=/usr/lib64/pkcs11/libsofthsm2.so openssl req -new -engine pkcs11 -keyform ENGINE -key 'pkcs11:token=MyToken;object=MyECKey' -subj /CN=Testing My C++-fu is too weak to provide a proper fix though.
I went deeper in the issue and found a bug in the engine (openssl-pkcs11). When ENGINE_load_private_key() is called, it calls EVP_PKEY_set1_engine() to add a engine reference to the EVP_PKEY, but does not check its return value. In fact, for EC keys, EVP_PKEY_set1_engine() fails, but it is ignored. The EVP_PKEY is returned containing all necessary bits to perform a signature, but does not contain a reference to the engine. When ENGINE_finish() is called, the engine reference counter reaches zero, causing the structures to be freed. Finally, when the application exits, the destructors are called, and they try to free the engine structures again, causing the double free and core dump. I opened an issue in libp11 upstream project: https://github.com/OpenSC/libp11/issues/243
openssl-pkcs11-0.4.8-2.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-6e46af6a08
openssl-pkcs11-0.4.8-2.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-6e46af6a08
openssl-pkcs11-0.4.8-2.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.