Bug 2388236

Summary: [abrt] Crash under gnutls_x509_trust_list_verify_crt2() (accessing GnuTLS internals from multiple threads at the same time)
Product: [Fedora] Fedora Reporter: Loris Santamaria <loris.santamaria>
Component: gnutlsAssignee: Red Hat Crypto Team <crypto-team>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 42CC: ansasaki, crypto-team, dueno, fkrenzel, gnome-sig, loris.santamaria, mcatanza, mclasen, mcrha, rstrode, tm, zfridric
Target Milestone: ---Flags: fedora-admin-xmlrpc: mirror+
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
URL: https://retrace.fedoraproject.org/faf/reports/bthash/b8c14cc3dd48c62b44a10c227d2f3d27bf39032
Whiteboard: abrt_hash:50aad4babab71183dc8131178d1060bc32adc659;VARIANT_ID=workstation;
Fixed In Version: gnutls-3.8.11-1.fc43 Doc Type: ---
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2025-11-23 00:56:20 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1937513    
Bug Blocks:    
Attachments:
Description Flags
File: proc_pid_status
none
File: maps
none
File: limits
none
File: os_info
none
File: cpuinfo
none
File: core_backtrace
none
File: exploitable
none
File: dso_list
none
File: mountinfo
none
File: backtrace
none
File: open_fds
none
File: environ none

Description Loris Santamaria 2025-08-13 14:49:12 UTC
Version-Release number of selected component:
evolution-data-server-3.56.2-1.fc42

Additional info:
reporter:       libreport-2.17.15
type:           CCpp
reason:         evolution-calendar-factory killed by SIGSEGV
journald_cursor: s=5c07d2b76c634327bd03e0830805b202;i=177446;b=bbe5ed35eca9468ea6c1e84fa15a30c3;m=1e2e4f04d2;t=63c401a75ec8a;x=ba877b7b0f229409
executable:     /usr/libexec/evolution-calendar-factory
cmdline:        /usr/libexec/evolution-calendar-factory
cgroup:         0::/user.slice/user-1000.slice/user/app.slice/evolution-calendar-factory.service
rootdir:        /
uid:            1000
kernel:         6.15.9-201.fc42.x86_64
package:        evolution-data-server-3.56.2-1.fc42
runlevel:       N 5
backtrace_rating: 4
crash_function: object_find_init

Truncated backtrace:
Thread no. 1 (34 frames)
 #0 object_find_init at src/lib/object.c:338
 #1 C_FindObjectsInit at src/pkcs11.c:490
 #2 find_cert_cb at ../../lib/pkcs11.c:4173
 #3 _pkcs11_traverse_tokens at ../../lib/pkcs11.c:1618
 #4 _gnutls_pkcs11_get_distrust_after at ../../lib/pkcs11.c:4855
 #5 _gnutls_pkcs11_verify_crt_status at ../../../lib/x509/verify.c:1343
 #6 gnutls_x509_trust_list_verify_crt2 at ../../../lib/x509/verify-high.c:1590
 #7 _gnutls_x509_cert_verify_peers at ../../lib/cert-session.c:598
 #8 gnutls_certificate_verify_peers at ../../lib/cert-session.c:770
 #9 gnutls_certificate_verify_peers3 at ../../lib/cert-session.c:704
 #10 g_tls_connection_gnutls_verify_chain at ../tls/gnutls/gtlsconnection-gnutls.c:1061
 #11 verify_peer_certificate at ../tls/base/gtlsconnection-base.c:1355
 #12 accept_or_reject_peer_certificate at ../tls/base/gtlsconnection-base.c:1397
 #15 g_main_context_dispatch_unlocked at ../glib/gmain.c:4249
 #16 g_main_context_iterate_unlocked at ../glib/gmain.c:4314
 #17 g_main_context_iteration at ../glib/gmain.c:4379
 #18 crank_sync_handshake_context at ../tls/base/gtlsconnection-base.c:1690
 #19 g_tls_connection_base_handshake at ../tls/base/gtlsconnection-base.c:1815
 #20 soup_connection_connect at ../libsoup/soup-connection.c:873
 #21 soup_session_ensure_item_connection at ../libsoup/soup-session.c:1783
 #22 soup_session_process_queue_item at ../libsoup/soup-session.c:1805
 #23 soup_session_send at ../libsoup/soup-session.c:3264
 #24 e_soup_session_send_message_sync at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/libedataserver/e-soup-session.c:1948
 #25 e_soup_session_send_message_simple_sync at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/libedataserver/e-soup-session.c:2095
 #26 e_webdav_session_propfind_internal_sync at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/libedataserver/e-webdav-session.c:1432
 #27 e_webdav_session_propfind_sync at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/libedataserver/e-webdav-session.c:1488
 #28 e_webdav_session_getctag_sync at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/libedataserver/e-webdav-session.c:3309
 #29 ecb_caldav_get_changes_sync at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/calendar/backends/caldav/e-cal-backend-caldav.c:891
 #30 e_cal_meta_backend_get_changes_sync at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/calendar/libedata-cal/e-cal-meta-backend.c:5149
 #31 ecmb_refresh_internal_sync at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/calendar/libedata-cal/e-cal-meta-backend.c:821
 #32 e_cal_backend_custom_operation_thread at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/calendar/libedata-cal/e-cal-backend.c:4651
 #34 cal_backend_dispatch_thread at /usr/src/debug/evolution-data-server-3.56.2-1.fc42.x86_64/src/calendar/libedata-cal/e-cal-backend.c:430
 #36 g_thread_proxy at ../glib/gthread.c:893
 #38 __clone3 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Comment 1 Loris Santamaria 2025-08-13 14:49:16 UTC
Created attachment 2103509 [details]
File: proc_pid_status

Comment 2 Loris Santamaria 2025-08-13 14:49:18 UTC
Created attachment 2103510 [details]
File: maps

Comment 3 Loris Santamaria 2025-08-13 14:49:19 UTC
Created attachment 2103511 [details]
File: limits

Comment 4 Loris Santamaria 2025-08-13 14:49:20 UTC
Created attachment 2103512 [details]
File: os_info

Comment 5 Loris Santamaria 2025-08-13 14:49:22 UTC
Created attachment 2103513 [details]
File: cpuinfo

Comment 6 Loris Santamaria 2025-08-13 14:49:23 UTC
Created attachment 2103514 [details]
File: core_backtrace

Comment 7 Loris Santamaria 2025-08-13 14:49:24 UTC
Created attachment 2103515 [details]
File: exploitable

Comment 8 Loris Santamaria 2025-08-13 14:49:26 UTC
Created attachment 2103516 [details]
File: dso_list

Comment 9 Loris Santamaria 2025-08-13 14:49:27 UTC
Created attachment 2103517 [details]
File: mountinfo

Comment 10 Loris Santamaria 2025-08-13 14:49:29 UTC
Created attachment 2103518 [details]
File: backtrace

Comment 11 Loris Santamaria 2025-08-13 14:49:30 UTC
Created attachment 2103519 [details]
File: open_fds

Comment 12 Loris Santamaria 2025-08-13 14:49:31 UTC
Created attachment 2103520 [details]
File: environ

Comment 13 Milan Crha 2025-08-13 15:22:14 UTC
Thanks for a bug report. This looks similar as bug #1937513.

From what I can see, several of your CalDAV calendars had been connecting to their server, while multiple threads had been accessing GnuTLS internals. That's something what had been supposed to be fixed within that old bug, if I recall correctly.

I move this to the glib-networking for further investigation. I see some threads do wait on some condition, but not all, not from all the places.

Comment 14 Michael Catanzaro 2025-08-13 21:26:29 UTC
Good analysis, thanks. I see we are verifying the same gnutls_x509_trust_list 0x7fd319089520 on thread 1 (the crash thread) and thread 12 at the same time.

This is indeed very similar to bug #1937513, but it's not the same because in that bug GTlsDatabase was performing the verification, whereas in this bug, it's handled by GTlsConnection instead. That changed in https://gitlab.gnome.org/GNOME/glib-networking/-/commit/d94c33138ecb12905fde400b1db41a3829ce7be9 so the original fix here didn't even last for very long.

The underlying issue is a bug in GnuTLS, https://gitlab.com/gnutls/gnutls/-/issues/1212. Probably best to just fix it. Alternatively, it can probably be solved with extra mutexes in glib-networking, but that really shouldn't be required.

Comment 15 Daiki Ueno 2025-08-29 06:04:56 UTC
This does not look like the same issue as bug #1937513, where I put a comment saying that the original issue was fixed in:
https://gitlab.com/gnutls/gnutls/-/issues/1277

Here, the crash is happening in a PKCS#11 module itself:

> crash_function: object_find_init
> 
> Truncated backtrace:
> Thread no. 1 (34 frames)
>  #0 object_find_init at src/lib/object.c:338
>  #1 C_FindObjectsInit at src/pkcs11.c:490
>  #2 find_cert_cb at ../../lib/pkcs11.c:4173

I suspect that the object_find_init function is from tpm2-pkcs11:
https://github.com/tpm2-software/tpm2-pkcs11/blob/1e5b798e2f1b0b674a97bf1149d28874779a2680/src/lib/object.c#L313

which has a locking mechanism, but it is only enabled under certain conditions (CKF_OS_LOCKING_OK is set and/or locking functions are specified):
https://github.com/tpm2-software/tpm2-pkcs11/blob/1e5b798e2f1b0b674a97bf1149d28874779a2680/src/lib/general.c#L173

I guess the fix would be to simply set the CKF_OS_LOCKING_OK flag in the GnuTLS; this is done for p11-kit-trust.so, but not for external modules:
https://gitlab.com/gnutls/gnutls/-/blob/935505fa10e96e92f3f2d69ca6f64f0ac2726d15/lib/pkcs11.c#L412

Comment 16 Fedora Update System 2025-11-21 02:29:29 UTC
FEDORA-2025-45b1844342 (gnutls-3.8.11-1.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-45b1844342

Comment 17 Fedora Update System 2025-11-22 02:23:23 UTC
FEDORA-2025-45b1844342 has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-45b1844342`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-45b1844342

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 18 Fedora Update System 2025-11-23 00:56:20 UTC
FEDORA-2025-45b1844342 (gnutls-3.8.11-1.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.