Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1400653

Summary: Improve memberof performance, caching for each group their ancestors DN
Product: Red Hat Enterprise Linux 7 Reporter: thierry bordaz <tbordaz>
Component: 389-ds-baseAssignee: Noriko Hosoi <nhosoi>
Status: CLOSED ERRATA QA Contact: Viktor Ashirov <vashirov>
Severity: high Docs Contact: Aneta Šteflová Petrová <apetrova>
Priority: high    
Version: 7.4CC: mreynolds, nkinder, pasik, rmeggins, sramling
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.3.6.1-3.el7 Doc Type: Enhancement
Doc Text:
See the Doc Text in https://bugzilla.redhat.com/show_bug.cgi?id=1395940
 Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 21:12:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1362239, 1399979    

Description thierry bordaz 2016-12-01 17:40:56 UTC 
Description of problem:

When updating a membership attribute of a group, impacted members will be updated (update of 'memberof'). It triggers internal searches of groups to compute their ancestors. But the same groups a searched multiple times.
The problem is described in http://www.port389.org/docs/389ds/design/memberof-scalability.html

Adding a cache that holds the ancestors of each group, would reduce the number of internal searches

Version-Release number of selected component (if applicable):

All versions

How reproducible:

 
Steps to Reproduce:
1. download https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py,

2. edit it with
users=1000,
groups=100,
groups_per_user=100,
nested_groups_max_level=2,
nested_groups_per_user=5,
hosts=1000,
hostgroups=100,
hostgroups_per_host=100,
nested_hostgroups_max_level=2,
nested_hostgroups_per_host=5,
direct_sudorules=5,  # users, hosts
indirect_sudorules=50,  # groups, hostgroups
sudorules_per_user=5,
sudorules_per_group=10,
sudorules_per_host=5,
sudorules_per_hostgroup=10,
direct_hbac=5,  # users, hosts
indirect_hbac=50,  # groups, hostgroups
hbac_per_user=5,
hbac_per_group=10,
hbac_per_host=5,
hbac_per_hostgroup=10

3. turn on internal op logging (to count the internal searches)

4. create a ldif file and import the entry (ldapadd)

Actual results:




Expected results:


Additional info:
Comment 1 thierry bordaz 2016-12-01 17:42:01 UTC 
upstream tickets

https://fedorahosted.org/389/ticket/49031
Comment 2 thierry bordaz 2017-01-11 15:26:41 UTC 
Patch pushed upstream https://fedorahosted.org/389/ticket/49031#comment:12
Comment 5 Sankar Ramalingam 2017-05-19 05:21:03 UTC 
Made few changes to ticket48861_test.py script and verified from Pytest.
https://pagure.io/389-ds-base/issue/49031#comment-127558
https://pagure.io/389-ds-base/issue/raw/files/7db1d4f6ce18f73b95f2299ee45247cb8cf46b44def08a5efd274205b931e9c2-ticket48861_test.py

[0 root@qeos-125 tickets]# py.test  -v ticket48861_test.py 
=================== test session starts ======================
platform linux2 -- Python 2.7.5, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /usr/bin/python
cachedir: .cache
metadata: {'Python': '2.7.5', 'Platform': 'Linux-3.10.0-663.el7.x86_64-x86_64-with-redhat-7.4-Maipo', 'Packages': {'py': '1.4.33', 'pytest': '3.0.7', 'pluggy': '0.4.0'}, 'Plugins': {'beakerlib': '0.7.1', 'html': '1.14.2', 'cov': '2.5.1', 'metadata': '1.5.0'}}
DS build: 1.3.6.1
389-ds-base: 1.3.6.1-13.el7
nss: 3.28.4-8.el7
nspr: 4.13.1-1.0.el7_3
openldap: 2.4.44-4.el7
svrcore: 4.1.3-2.el7

rootdir: /mnt/testarea/test/ds/dirsrvtests/tests/tickets, inifile:
plugins: metadata-1.5.0, html-1.14.2, cov-2.5.1, beakerlib-0.7.1
collected 2 items 

ticket48861_test.py::test_ticket48861 PASSED
ticket48861_test.py::test_ticket48861_final PASSED
=================== 2 passed in 11.59 seconds ===================
Comment 6 thierry bordaz 2017-05-19 09:05:36 UTC 
Hi Sankar,

thanks for the updating tests and results.
Just for confirmation, this BZ is mainly a performance oriented bug. When you switch it to 'verified' does that mean you completed performance testings ?

regards
Comment 7 Viktor Ashirov 2017-05-19 12:03:01 UTC 
Thierry, you're right, I'm switching bz back to ON_QA.

Sankar, please change status to VERIFIED, once you provide performance test results.

Thanks.
Comment 9 errata-xmlrpc 2017-08-01 21:12:24 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2086