RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 617623 - SSSD suffers from serious performance issues on initgroups calls
Summary: SSSD suffers from serious performance issues on initgroups calls
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: sssd
Version: 6.0
Hardware: All
OS: Linux
low
high
Target Milestone: rc
: ---
Assignee: Stephen Gallagher
QA Contact: Chandrasekar Kannan
URL:
Whiteboard:
Depends On:
Blocks: 579775
TreeView+ depends on / blocked
 
Reported: 2010-07-23 14:59 UTC by Stephen Gallagher
Modified: 2015-01-04 23:43 UTC (History)
7 users (show)

Fixed In Version: sssd-1.2.1-23.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-11-10 21:40:05 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Stephen Gallagher 2010-07-23 14:59:23 UTC 
Description of problem:
Users who are a member of one or more groups containing large numbers of members end up with unacceptable wait times (or timeouts) performing initgroups calls.

Since initgroups calls are always performed on login, this is causing unacceptably long login waits.


Version-Release number of selected component (if applicable):
sssd-1.2.1-21.el6

How reproducible:
Every time

Steps to Reproduce:
1. Create 10,000 users in LDAP.
2. Create a group that contains all of these users
3. Attempt to log in as one of these users.

  
Actual results:
There is a very long wait time to log in (measured in seconds, if not tens of seconds depending on system I/O performance)


Expected results:
Login time should be reasonable ( < 5 seconds)

Additional info:
This is happening because we tried to be too clever with the initgroups calls. When a user logs in, we attempt to completely refresh all entries associated with all groups the user is a member of. This worked fine in our small test environments, but we didn't adequately test this for scalability. Our customers are now doing this for us and are not pleased with the results.
Comment 1 Stephen Gallagher 2010-07-30 20:01:44 UTC 
Testers: you can compile and run the following C program and run it with the 'time' command

getgroups.c:

#include <sys/types.h>
#include <unistd.h>
#include <grp.h>

int main(int argc, char **argv)
{
    initgroups(argv[1], 0);

    return 0;
}



Compile and run:
gcc -o initgroups.test getgroups.c
time ./initgroups.test <username>


Prior to this patch, time would have reported 3-5s for users in groups of ~300-500 users. After this patch it will average < 1s.

Please test this on users of varying sized groups and numbers of groups.
Comment 3 Gowrishankar Rajaiyan 2010-08-30 12:50:25 UTC 
Setup:

1. Created a group bosgroup with 500 users, Group1 with 5000 users and mumgroup with 10000 users.
2. bos 500 is a member of Group1 and mumgroup.
3. pnq5000 is a member of mumgroup.
4. mum9999 is a member on mumgroup.

[root@rhel6snap11 ~]# time ./initgroups.test bos500

real	0m0.173s
user	0m0.000s
sys	0m0.001s
[root@rhel6snap11 ~]# time ./initgroups.test pnq5000

real	0m0.226s
user	0m0.001s
sys	0m0.000s
[root@rhel6snap11 ~]# time ./initgroups.test mum9999

real	0m0.272s
user	0m0.000s
sys	0m0.001s

Verified: sssd-1.2.1-26.el6.
Comment 4 Petter Reinholdtsen 2010-09-11 06:03:37 UTC 
Where can I find the patch used to fix this issue?
Comment 5 Stephen Gallagher 2010-09-13 10:59:59 UTC 
(In reply to comment #4)
> Where can I find the patch used to fix this issue?

http://git.fedorahosted.org/git/?p=sssd.git;a=commit;h=24a5809cbc96d10001535999a292cb26242ad01d

However, as is being discussed right now at https://fedorahosted.org/pipermail/sssd-devel/2010-September/004540.html we've discovered that this patch has introduced some issues with group resolution under certain circumstances.
Comment 6 releng-rhel@redhat.com 2010-11-10 21:40:05 UTC 
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.
Note You need to log in before you can comment on or make changes to this bug.