RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1147659 - cyrus-sasl client library (client.c) is not thread safe
Summary: cyrus-sasl client library (client.c) is not thread safe
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: cyrus-sasl
Version: 7.1
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: rc
: ---
Assignee: Jakub Jelen
QA Contact: Stanislav Zidek
URL:
Whiteboard:
: 1095249 (view as bug list)
Depends On:
Blocks: 1191021 1205796
TreeView+ depends on / blocked
 
Reported: 2014-09-29 18:56 UTC by Noriko Hosoi
Modified: 2015-11-19 09:14 UTC (History)
4 users (show)

Fixed In Version: cyrus-sasl-2.1.26-18.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-19 09:14:55 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
git patch file for cyrus-sasl-2.1.26 (1.76 KB, patch)
2014-10-01 22:08 UTC, Noriko Hosoi
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2127 0 normal SHIPPED_LIVE cyrus-sasl bug fix update 2015-11-19 09:23:15 UTC

Description Noriko Hosoi 2014-09-29 18:56:02 UTC
A utility ldclt in the 389-ds-base package acts as a cyrus-sasl client to get authenticated to access the Directory Server.  The utility crashes if it runs with 4 or more threads.  It turned out the cyrus-sasl client library (client.c) is not thread safe (or contains a bug). 

tested cyrus-sasl packages:

    cyrus-sasl-lib-2.1.26-17.el7.x86_64
    cyrus-sasl-gssapi-2.1.26-17.el7.x86_64
    cyrus-sasl-2.1.26-17.el7.x86_64
    cyrus-sasl-scram-2.1.26-17.el7.x86_64
    cyrus-sasl-debuginfo-2.1.26-17.el7.x86_64
    cyrus-sasl-devel-2.1.26-17.el7.x86_64
    cyrus-sasl-md5-2.1.26-17.el7.x86_64
    cyrus-sasl-plain-2.1.26-17.el7.x86_64

Description:
This cmechlist is a global variable shared among threads.
static cmech_list_t *cmechlist; /* global var which holds the list */

This is the contents of cmechlist when it's still healthy
(gdb) p cmechlist->mech_list
$2 = (cmechanism_t *) 0x7ffff001bfb0
(gdb) p *cmechlist->mech_list
$3 = {m = {version = 4, plugname = 0x7ffff001b8d0 "scram", plug = 0x7fffcc5d1160 <scram_client_plugins>}, next = 0x7ffff001aaf0}
(gdb) p *cmechlist->mech_list->next
$4 = {m = {version = 4, plugname = 0x7ffff001ab20 "gssapiv2", plug = 0x7fffcf005220 <gssapi_client_plugins+96>},
  next = 0x7ffff001aaa0}
(gdb) p *cmechlist->mech_list->next->next
$5 = {m = {version = 4, plugname = 0x7ffff001aad0 "gssapiv2", plug = 0x7fffcf0051c0 <gssapi_client_plugins>}, next = 0x7ffff0016920}
(gdb) p *cmechlist->mech_list->next->next->next
$6 = {m = {version = 4, plugname = 0x7ffff0016950 "digestmd5", plug = 0x7fffcf5f81a0 <digestmd5_client_plugins>},
  next = 0x7ffff000ab70}
(gdb) p *cmechlist->mech_list->next->next->next->next
$7 = {m = {version = 4, plugname = 0x7ffff000aba0 "EXTERNAL", plug = 0x7ffff6da65c0 <external_client_plugins>}, next = 0x7ffff00157f0}
(gdb) p *cmechlist->mech_list->next->next->next->next->next
$8 = {m = {version = 4, plugname = 0x7ffff00151a0 "crammd5", plug = 0x7fffcf7fe120 <crammd5_client_plugins>}, next = 0x7ffff001b1c0}
(gdb) p *cmechlist->mech_list->next->next->next->next->next->next
$9 = {m = {version = 4, plugname = 0x7ffff001ab70 "login", plug = 0x7fffcc9db100 <login_client_plugins>}, next = 0x7ffff001b870}
(gdb) p *cmechlist->mech_list->next->next->next->next->next->next->next
$10 = {m = {version = 4, plugname = 0x7ffff001b220 "plain", plug = 0x7fffcc7d6100 <plain_client_plugins>}, next = 0x7ffff0014470}
(gdb) p *cmechlist->mech_list->next->next->next->next->next->next->next->next
$11 = {m = {version = 4, plugname = 0x7ffff00144a0 "anonymous", plug = 0x7fffec7f80e0 <anonymous_client_plugins>}, next = 0x0}

This is a mech_list which is held in a context.  The variable c_conn is passed from ldap_int_sasl_close as lc->lconn_sasl_authctx or lc->lconn_sasl_sockctx.  Note that the address of c_conn->mech_list appears as cmechlist->mech_list->next->next->next in the global mech_list.
(gdb) p c_conn->mech_list
$12 = (cmechanism_t *) 0x7ffff0016920
(gdb) p *c_conn->mech_list
$13 = {m = {version = 4, plugname = 0x7ffff0016950 "digestmd5", plug = 0x7fffcf5f81a0 <digestmd5_client_plugins>},
  next = 0x7ffff000ab70}
(gdb) p *c_conn->mech_list->next
$14 = {m = {version = 4, plugname = 0x7ffff000aba0 "EXTERNAL", plug = 0x7ffff6da65c0 <external_client_plugins>},
  next = 0x7ffff00157f0}
...
$18 = {m = {version = 4, plugname = 0x7ffff00144a0 "anonymous", plug = 0x7fffec7f80e0 <anonymous_client_plugins>}, next = 0x0}
(gdb) p *c_conn->mech_list->next->next->next->next->next->next
Cannot access memory at address 0x0

In client_dispose (client.c in cyrus-sasl lib), it has this code:
 326
 327 static void client_dispose(sasl_conn_t *pconn)
 328 {
...
 346   if (c_conn->mech_list != cmechlist->mech_list) {
 347       /* free connection-specific mech_list */
 348       cmechanism_t *m, *prevm;
 349
 350       m = c_conn->mech_list; /* m point to beginning of the list */
 351
 352       while (m) {
 353       prevm = m;
 354       m = m->next;
 355       sasl_FREE(prevm);
 356       }
 357   }
That is, c_conn->mech_list is freed even if it is part of the global cmechlist.

Once client_dispose is called, the global mach_list is corrupted and it crashes the sasl client program, in this case ldclt.

(gdb) p cmechlist->mech_list
$36 = (cmechanism_t *) 0x7ffff001bfb0
(gdb) p *cmechlist->mech_list
$37 = {m = {version = 4, plugname = 0x7ffff001b8d0 "scram", plug = 0x7fffcc5d1160 <scram_client_plugins>}, next = 0x7ffff001aaf0}
(gdb) p *cmechlist->mech_list->next
$38 = {m = {version = 4, plugname = 0x7ffff001ab20 "gssapiv2", plug = 0x7fffcf005220 <gssapi_client_plugins+96>},
  next = 0x7ffff001aaa0}
(gdb) p *cmechlist->mech_list->next->next
$39 = {m = {version = 4, plugname = 0x7ffff001aad0 "gssapiv2", plug = 0x7fffcf0051c0 <gssapi_client_plugins>}, next = 0x7ffff0016920}
(gdb) p *cmechlist->mech_list->next->next->next
$40 = {m = {version = -268352416, plugname = 0x7ffff0016950 "digestmd5", plug = 0x7fffcf5f81a0 <digestmd5_client_plugins>},  next = 0x7ffff000ab70}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(gdb) p *cmechlist->mech_list->next->next->next->next
$41 = {m = {version = -268343024, plugname = 0x7ffff000aba0 "EXTERNAL", plug = 0x7ffff6da65c0 <external_client_plugins>},  next = 0x7ffff00157f0}

The check at the line 346 is not enough to determine whether the mech object should be freed or not.

    346   if (c_conn->mech_list != cmechlist->mech_list) {

Instead of checking the head mech_list, individual mech must be checked if it exists in the global list cmechlist->mech_list or not.  The mech object should be freed only if it is not found in the global list.

Comment 1 Noriko Hosoi 2014-09-29 18:57:03 UTC
*** Bug 1095249 has been marked as a duplicate of this bug. ***

Comment 2 Noriko Hosoi 2014-09-29 18:58:49 UTC
Please see also Bug 1095249 - SASL+ldclt gives segmentation fault.

The bug contains the stacktraces from gdb and valgrind.

Comment 3 Noriko Hosoi 2014-09-30 17:20:30 UTC
Hello Petr,

If possible, we'd like to make try changes in the code (client.c) and submit a patch if it works.  So far, I found the dist-git for cyrus-sasl, where I got the src.rpm.  But I have no luck to find the repository for the source code itself.  Could you please share the info with us?

Thanks,
--noriko

Comment 5 Noriko Hosoi 2014-10-01 22:08:20 UTC
Created attachment 943208 [details]
git patch file for cyrus-sasl-2.1.26

By applying the attached patch, no more crash of ldclt is observed.
> Bug 1095249 - SASL+ldclt gives segmentation fault.

valgrind reports these leaks which are not related to the mech object.
==11247== 2,720 bytes in 10 blocks are possibly lost in loss record 137 of 150
==11247==    at 0x4C2A1D4: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==11247==    by 0x4011EB4: _dl_allocate_tls (dl-tls.c:318)
==11247==    by 0x608C9A0: pthread_create@@GLIBC_2.2.5 (allocatestack.c:570)
==11247==    by 0x11425A: runThem (ldclt.c:653)
==11247==    by 0x10C58A: main (ldclt.c:3318)
==11247== 
==11247== 15,960 bytes in 399 blocks are definitely lost in loss record 146 of 150
==11247==    at 0x4C2845D: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==11247==    by 0x5E7C25E: _plug_strdup (plugin_common.c:261)
==11247==    by 0xD99B1AE: ask_user_info (digestmd5.c:4195)
==11247==    by 0xD99FAC7: digestmd5_client_mech_step (digestmd5.c:4246)
==11247==    by 0x5E707F1: sasl_client_step (client.c:1000)
==11247==    by 0x5E70B75: sasl_client_start (client.c:946)
==11247==    by 0x5A19832: ldap_int_sasl_bind (cyrus.c:523)
==11247==    by 0x5A1CE51: ldap_sasl_interactive_bind (sasl.c:471)
==11247==    by 0x5A1D07C: ldap_sasl_interactive_bind_s (sasl.c:505)
==11247==    by 0x10ECC1: connectToLDAP (ldapfct.c:1007)
==11247==    by 0x10F4D4: connectToServer (ldapfct.c:1207)
==11247==    by 0x113414: doExactSearch (ldapfct.c:3819)
==11247==    by 0x11B853: threadMain (threadMain.c:1190)
==11247==    by 0x608BDF2: start_thread (pthread_create.c:308)
==11247==    by 0x63963DC: clone (clone.S:113)

Comment 6 Noriko Hosoi 2014-10-01 22:09:39 UTC
Scratch build is found here:
https://brewweb.devel.redhat.com/taskinfo?taskID=8052523

Comment 12 errata-xmlrpc 2015-11-19 09:14:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2127.html


Note You need to log in before you can comment on or make changes to this bug.