RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 680367 - sssd not thread-safe
Summary: sssd not thread-safe
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: sssd
Version: 6.0
Hardware: Unspecified
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: Stephen Gallagher
QA Contact: Chandrasekar Kannan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-02-25 09:34 UTC by Todd Lipcon
Modified: 2017-02-06 14:35 UTC (History)
8 users (show)

Fixed In Version: sssd-1.5.1-10.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-05-19 11:39:49 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2011:0560 0 normal SHIPPED_LIVE Low: sssd security, bug fix, and enhancement update 2011-05-19 11:38:17 UTC

Description Todd Lipcon 2011-02-25 09:34:15 UTC
Description of problem:

The version of the sssd client included in RHEL 6 "Santiago" is not thread safe for some calls. In particular, under heavy concurrency of getpwuid_r on a system slaved to LDAP, the process making the calls eventually crashes with an invalid free.

The relevant Fedora bug is here:
https://fedorahosted.org/sssd/ticket/640
fixed in SSSD 1.5.0.

Version-Release number of selected component (if applicable):

RHEL6.0 includes sssd-client 1.2.1

How reproducible:

Very.

Steps to Reproduce:
1. Call getpwuid_r at a high rate from several threads
2. Eventually process will crash

I can reproduce this reliably when load testing a piece of software that relies on these calls. If necessary I can probably write a simple C test case to reproduce the problem, but the upstream bug should be sufficient.
  
Actual results:

getpwuid_r returns strange error codes, and soon memory corruption results, for example due to invalid frees:
*** glibc detected *** /mnt/toolchain/JDK6u20-64bit/bin/java: free(): invalid pointer: 0x0000003575741d23 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3575675676]
/lib64/libnss_sss.so.2(_nss_sss_getpwuid_r+0x11b)[0x7fe716cb42cb]
/lib64/libc.so.6(getpwuid_r+0xdd)[0x35756a5dfd]

Expected results:

Should function correctly, since getpwuid_r is a reentrant-safe call.

Comment 4 Kaushik Banerjee 2011-03-18 11:32:46 UTC
1. Configure sssd 1.2 running on RHEL6.0
2. run ./nss-thread-test (compiled as "gcc -lpthread nss-thread-test.c -o nss-thread-test")

The script doesn't finish executing and crashes with following error:
<snip>

7f9b317fd000-7f9b321fd000 rw-p 00000000 00:00 0 
7f9b321fd000-7f9b321fe000 ---p 00000000 00:00 0 
7f9b321fe000-7f9b32bfe000 rw-p 00000000 00:00 0 
7f9b32bfe000-7f9b32bff000 ---p 00000000 00:00 0 
7f9b32bff000-7f9b335ff000 rw-p 00000000 00:00 0 
7f9b335ff000-7f9b33600000 ---p 00000000 00:00 0 Aborted (core dumped)

</snip>

3. On another RHEL6.1 machine with sssd 1.5 installed and configured.
4. Repeat step 2.

Result:
The multi-threaded script executes successfully.

<snip>

Completed join with thread 43 status= 0
Completed join with thread 44 status= 0
Completed join with thread 45 status= 0
Completed join with thread 46 status= 0
Completed join with thread 47 status= 0
Completed join with thread 48 status= 0
Completed join with thread 49 status= 0

</snip>

Version: rpm -qi sssd | head
Name        : sssd                         Relocations: (not relocatable)
Version     : 1.5.1                             Vendor: Red Hat, Inc.
Release     : 14.el6                        Build Date: Thu 10 Mar 2011 01:00:12 AM IST
Install Date: Fri 11 Mar 2011 02:36:04 AM IST      Build Host: x86-009.build.bos.redhat.com
Group       : Applications/System           Source RPM: sssd-1.5.1-14.el6.src.rpm
Size        : 3418526                          License: GPLv3+
Signature   : (none)
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
URL         : http://fedorahosted.org/sssd/
Summary     : System Security Services Daemon


Marking bug as 'VERIFIED'


----------nss-thread-test.c-----------

#define _GNU_SOURCE

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <pwd.h>

#define NUM_THREADS 50
#define BUFLEN 4096

struct thread_data {
    int  tid;
    int  min;
    int  max;
};

struct thread_data td[NUM_THREADS];

void *thread_main(void *arg)
{
    struct thread_data *data = (struct thread_data *) arg;
    int i;
    struct passwd pw, *pwp;
    char buf[BUFLEN];
    int num = 0;
    unsigned long long uid;

    for (i=0; i<NUM_THREADS; i++) {
        uid = random() % (data->max - data->min) + data->min;
        printf("calling getpwuid from %d for %llu (%d/%d)\n",
                data->tid, uid, i, NUM_THREADS);
        getpwuid_r(uid, &pw, buf, BUFLEN, &pwp);
    }

    printf("Thread %d done\n", data->tid);
    pthread_exit((void *) NULL);
}


int main(int argc, char *argv[])
{
    pthread_t thread[NUM_THREADS];
    pthread_attr_t attr;
    int ret, t;
    void *status;

    pthread_attr_init(&attr);
    pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);

    srandom(time(NULL));

    for(t=0; t < NUM_THREADS; t++) {
        printf("Creating thread %d\n", t);

        td[t].tid = t;
        td[t].min = 501;
        td[t].max = 9999;

        ret = pthread_create(&thread[t], &attr, thread_main, &td[t]);
        if (ret) {
            printf("ERROR; return code from pthread_create() is %d\n", ret);
            goto done;
        }
    }

    for(t=0; t<NUM_THREADS; t++) {
        ret = pthread_join(thread[t], &status);
        if (ret) {
            printf("ERROR return code from pthread_join() is %d\n", ret);
            goto done;
        }
        printf("Completed join with thread %d status= %ld\n", t, (long) status);
    }


    ret = 0;
done:
    pthread_attr_destroy(&attr);
    pthread_exit(NULL);
    return ret;
}

Comment 7 errata-xmlrpc 2011-05-19 11:39:49 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0560.html

Comment 8 errata-xmlrpc 2011-05-19 13:09:42 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0560.html


Note You need to log in before you can comment on or make changes to this bug.