RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1834909 - softhsm use-after-free on process exit
Summary: softhsm use-after-free on process exit
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: softhsm
Version: 8.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.0
Assignee: Alexander Bokovoy
QA Contact: Kaleem
URL:
Whiteboard:
Depends On: 1831086
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-05-12 16:09 UTC by Alexander Bokovoy
Modified: 2020-11-04 02:51 UTC (History)
7 users (show)

Fixed In Version: softhsm-2.6.0-3
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1831086
Environment:
Last Closed: 2020-11-04 02:50:41 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+

Attachments (Terms of Use)

Description Alexander Bokovoy 2020-05-12 16:09:26 UTC 
+++ This bug was initially created as a clone of Bug #1831086 +++

My OpenConnect builds are failing on rawhide. It turns out that when softhsm is used, it prevents the process from exiting cleanly.

Reproduce by just rebuilding the openconnect RPM. Or checkout out from git://git.infradead.org/users/dwmw2/openconnect.git and running 'make check'...


make  check-TESTS
make[2]: Entering directory '/home/dwmw2/git/openconnect/gtls/tests'
make[3]: Entering directory '/home/dwmw2/git/openconnect/gtls/tests'
PASS: autocompletion
PASS: auth-username-pass
PASS: auth-certificate
PASS: auth-nonascii
PASS: cert-fingerprint
PASS: id-test


The test it's stuck on, and hasn't printed yet, is auth-pkcs11. If I connect to the stuck openconnect process with gdb, I see this backtrace:

919	      if ((atomic_load_acquire (&rwlock->__data.__readers)
(gdb) bt
#0  0x00007f3be4ed14b7 in __pthread_rwlock_wrlock_full (abstime=0x0, rwlock=0x134f7b0) at pthread_rwlock_common.c:919
#1  __GI___pthread_rwlock_wrlock (rwlock=0x134f7b0) at pthread_rwlock_wrlock.c:27
#2  0x00007f3be4bfbe7d in CRYPTO_THREAD_write_lock (lock=<optimized out>) at crypto/threads_pthread.c:78
#3  0x00007f3be4b61745 in ENGINE_finish (e=0x134f7f0) at crypto/engine/eng_init.c:101
#4  0x00007f3bd4b55e8b in OSSLCryptoFactory::~OSSLCryptoFactory() () from /usr/lib64/pkcs11/libsofthsm2.so
#5  0x00007f3bd4b55f1d in OSSLCryptoFactory::~OSSLCryptoFactory() () from /usr/lib64/pkcs11/libsofthsm2.so
#6  0x00007f3be4d38680 in __run_exit_handlers (status=status@entry=0, listp=0x7f3be4ebe738 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:108
#7  0x00007f3be4d387c0 in __GI_exit (status=status@entry=0) at exit.c:139
#8  0x0000000000403f4c in main (argc=<optimized out>, argv=<optimized out>) at ../main.c:1906

--- Additional comment from David Woodhouse on 2020-05-04 19:22:36 EEST ---

==228434== Invalid read of size 4
==228434==    at 0x54D9316: __pthread_rwlock_wrlock_full (pthread_rwlock_common.c:581)
==228434==    by 0x54D9316: pthread_rwlock_wrlock (pthread_rwlock_wrlock.c:27)
==228434==    by 0x58ADE7C: CRYPTO_THREAD_write_lock (threads_pthread.c:78)
==228434==    by 0x5813744: ENGINE_finish (eng_init.c:101)
==228434==    by 0x134794D6: OSSLCryptoFactory::~OSSLCryptoFactory() (OSSLCryptoFactory.cpp:240)
==228434==    by 0x13479558: OSSLCryptoFactory::~OSSLCryptoFactory() (OSSLCryptoFactory.cpp:259)
==228434==    by 0x552767F: __run_exit_handlers (exit.c:108)
==228434==    by 0x55277BF: exit (exit.c:139)
==228434==    by 0x404043: main (main.c:1553)
==228434==  Address 0x14bb74c8 is 24 bytes inside a block of size 56 free'd
==228434==    at 0x4839A0C: free (vg_replace_malloc.c:540)
==228434==    by 0x583FBE1: OPENSSL_cleanup (init.c:601)
==228434==    by 0x583FBE1: OPENSSL_cleanup (init.c:497)
==228434==    by 0x552767F: __run_exit_handlers (exit.c:108)
==228434==    by 0x55277BF: exit (exit.c:139)
==228434==    by 0x404043: main (main.c:1553)
==228434==  Block was alloc'd at
==228434==    at 0x483880B: malloc (vg_replace_malloc.c:309)
==228434==    by 0x584859D: CRYPTO_zalloc (mem.c:230)
==228434==    by 0x58ADE0B: CRYPTO_THREAD_lock_new (threads_pthread.c:29)
==228434==    by 0x58137B9: do_engine_lock_init (eng_lib.c:25)
==228434==    by 0x58137B9: do_engine_lock_init_ossl_ (eng_lib.c:21)
==228434==    by 0x54DBD7E: __pthread_once_slow (pthread_once.c:116)
==228434==    by 0x58ADEFC: CRYPTO_THREAD_run_once (threads_pthread.c:118)
==228434==    by 0x5813878: ENGINE_new (eng_lib.c:33)
==228434==    by 0x58152FD: ENGINE_rdrand (eng_rdrand.c:70)
==228434==    by 0x58152FD: engine_load_rdrand_int (eng_rdrand.c:85)
==228434==    by 0x583F8BC: ossl_init_engine_rdrand (init.c:353)
==228434==    by 0x583F8BC: ossl_init_engine_rdrand_ossl_ (init.c:347)
==228434==    by 0x54DBD7E: __pthread_once_slow (pthread_once.c:116)
==228434==    by 0x58ADEFC: CRYPTO_THREAD_run_once (threads_pthread.c:118)
==228434==    by 0x584004C: OPENSSL_init_crypto (init.c:723)
==228434==    by 0x584004C: OPENSSL_init_crypto (init.c:620)

--- Additional comment from David Woodhouse on 2020-05-07 17:22:32 EEST ---

This is because SoftHSM's atexit hander (well, the destructor for the OSSLCryptoFactory object) is trying to clean up the rdrand engine after OpenSSL's own atexit handler has run and cleaned up *everything*.

Proof of concept fix at https://github.com/dwmw2/SoftHSMv2/commit/f9c24e04f21e241ee93dd862f5ebb56dd54461c5

I've worked around it for now in my OpenConnect builds by exporting OPENSSL_ia32cap=~0x4000000000000000 when running the tests, to disable RDRAND.

--- Additional comment from David Woodhouse on 2020-05-12 17:06:02 EEST ---

Fix merged: https://github.com/opendnssec/SoftHSMv2/commit/c2cc0652b4c4829fc6ba186469f4e324af77dfe8

--- Additional comment from Fedora Update System on 2020-05-12 18:50:01 EEST ---

FEDORA-2020-3735365352 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-3735365352
Comment 13 errata-xmlrpc 2020-11-04 02:50:41 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: idm:DL1 and idm:client security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4670
Note You need to log in before you can comment on or make changes to this bug.