Bug 1482555 - sysdb index improvements - missing ghost attribute indexing, unneeded objectclass index etc..
Summary: sysdb index improvements - missing ghost attribute indexing, unneeded objectc...
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: sssd
Version: 7.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: SSSD Maintainers
QA Contact: Amith
Aneta Šteflová Petrová
URL:
Whiteboard:
Keywords:
: 1486222 (view as bug list)
Depends On:
Blocks: 1472255
TreeView+ depends on / blocked
 
Reported: 2017-08-17 14:47 UTC by Jakub Hrozek
Modified: 2018-05-23 11:19 UTC (History)
13 users (show)

(edit)
Improved SSSD performance when adding new users

Previously, a missing index for an attribute caused the System Security Services Daemon (SSSD) to check every object in the SSSD cache for the attribute when adding a new user object to the cache. Consequently, adding a new user to a cache with many entries was sometimes unusually slow and caused a considerable increase in system load. This update adds an index for the attribute. As a result, the amount of time and CPU resources required to add a user to the cache no longer depends on the number of objects in the cache, which significantly improves the performance of SSSD when adding new users.
Clone Of:
(edit)
Last Closed: 2018-04-10 17:13:24 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2018:0929 None None None 2018-04-10 17:15 UTC

Description Jakub Hrozek 2017-08-17 14:47:53 UTC
This bug is created as a clone of upstream ticket:
https://pagure.io/SSSD/sssd/issue/3472

This ticket more or less tracks https://github.com/SSSD/sssd/pull/332 which indexes the ghost attribute, removes unneeded indexes on the objectclass attribute etc

This provides a very noticeable performance enhancement.

Comment 3 Thorsten Scherf 2017-08-29 09:21:49 UTC
*** Bug 1486222 has been marked as a duplicate of this bug. ***

Comment 4 Jakub Hrozek 2017-09-14 15:07:24 UTC
The indexing fixes were merged: 9acdf51bf32d7b4389f3faea0fc6b73c56b6da71

Comment 5 Jakub Hrozek 2017-09-14 15:07:57 UTC
Actually, let's mark this as POST when the objectcategory ticket is also merged.

Comment 10 Jakub Hrozek 2017-10-05 15:23:22 UTC
Upstream ticket:
https://pagure.io/SSSD/sssd/issue/3503

Comment 15 Jakub Hrozek 2017-11-27 20:43:47 UTC
* master:
 * 2927da49dd8a16fff6312d89ad43cc355655800c                                                                                                                                                                                                    
 * 98195e591c4d97caa6125e8214879660b740973f
 * 0e238c259c066cf997aaa940d33d6bda96c15925
 * 0cce3d3ad14caf406303cf2ce6bf80171b708a93

Comment 17 Lukas Slebodnik 2018-01-15 11:11:51 UTC
master:
* 5b78fff78bb44d1af5420db23b02210f755f5f17

Comment 20 Amith 2018-02-01 08:17:14 UTC
Verified this bug on SSSD Version : sssd-1.16.0-15.el7.x86_64

Steps followed during verification:
1. Create large number of users who are member of large groups.

2. Run id <user> command, group lookup cmd and collect the time statistics.

3. Compare the stats with older builds, to verify improved performance.

GLOSSARY OF VARIABLES USED:
----------------------------------------------------------------------------------
TEST_200_ID_STAT : ID command time for fetching record of a member of 200 groups        : 
TEST_500_ID_STAT : ID command time for fetching record of a member of 500 groups
TEST_1000_ID_STAT: ID command time for fetching record of a member of 1000 groups
TEST_1500_ID_STAT: ID command time for fetching record of a member of 1500 groups
TEST_2000_ID_STAT: ID command time for fetching record of a member of 2000 groups
TEST_3000_ID_STAT: ID command time for fetching record of a member of 3000 groups

TEST_200_GRP_LKP : Lookup time for fetching a group with 200 users
TEST_500_GRP_LKP : Lookup time for fetching a group with 500 users
TEST_1000_GRP_LKP: Lookup time for fetching a group with 1000 users
TEST_1500_GRP_LKP: Lookup time for fetching a group with 1500 users
TEST_2000_GRP_LKP: Lookup time for fetching a group with 2000 users
TEST_3000_GRP_LKP: Lookup time for fetching a group with 3000 users
----------------------------------------------------------------------------------

PERFORMANCE REPORT FROM LATEST TESTS COMPARING OLDER BUILDS & RHEL VERSIONS:
----------------------------------------------------------------------------------
VARIABLES         |   RHEL-7.2     |   RHEL-7.4     |     RHEL-7.5
----------------------------------------------------------------------------------
TEST_200_ID_STAT  | 01m : 57.9297s | 00m : 20.2516s | 00m : 15.7532s
TEST_500_ID_STAT  | 04m : 54.1192s | 00m : 52.9034s | 00m : 39.1856s
TEST_1000_ID_STAT | 09m : 47.6430s | 01m : 50.4866s | 01m : 22.0058s
TEST_1500_ID_STAT | 14m : 46.5822s | 03m : 27.0541s | 02m : 8.4198s
TEST_2000_ID_STAT | 19m : 54.5008s | 04m : 56.6328s | 02m : 50.7445s
TEST_3000_ID_STAT | 30m : 3.6612s  | 05m : 52.1647s | 04m : 29.1755s
                  |                |                |               
TEST_200_GRP_LKP  | 00m : 0.3210s  | 00m : 0.2295s  | 00m : 0.1384s 
TEST_500_GRP_LKP  | 00m : 0.5551s  | 00m : 0.3717s  | 00m : 0.2113s 
TEST_1000_GRP_LKP | 00m : 1.0775s  | 00m : 0.5761s  | 00m : 0.3944s 
TEST_1500_GRP_LKP | 00m : 1.6610s  | 00m : 0.8278s  | 00m : 0.4977s 
TEST_2000_GRP_LKP | 00m : 2.3798s  | 00m : 1.1068s  | 00m : 0.6232s 
TEST_3000_GRP_LKP | 00m : 4.4606s  | 00m : 1.5501s  | 00m : 0.9165s 
---------------------------------------------------------------------------------- 

Observation:
1. The statistics were collected from multiple rounds of automated tests. All the tests were conducted on Beaker VMs.
2. Overall the latest build on RHEL-7.5 shows better performance than its predecessors.

Comment 23 errata-xmlrpc 2018-04-10 17:13:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0929


Note You need to log in before you can comment on or make changes to this bug.