RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1400653 - Improve memberof performance, caching for each group their ancestors DN
Summary: Improve memberof performance, caching for each group their ancestors DN
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: 389-ds-base
Version: 7.4
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Noriko Hosoi
QA Contact: Viktor Ashirov
Aneta Šteflová Petrová
URL:
Whiteboard:
Depends On:
Blocks: 1362239 1399979
TreeView+ depends on / blocked
 
Reported: 2016-12-01 17:40 UTC by thierry bordaz
Modified: 2020-09-13 21:53 UTC (History)
5 users (show)

Fixed In Version: 389-ds-base-1.3.6.1-3.el7
Doc Type: Enhancement
Doc Text:
See the Doc Text in https://bugzilla.redhat.com/show_bug.cgi?id=1395940
Clone Of:
Environment:
Last Closed: 2017-08-01 21:12:24 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github 389ds 389-ds-base issues 2090 0 None closed Improve memberof with a cache of group parents 2020-09-25 09:42:27 UTC
Red Hat Product Errata RHBA-2017:2086 0 normal SHIPPED_LIVE 389-ds-base bug fix and enhancement update 2017-08-01 18:37:38 UTC

Description thierry bordaz 2016-12-01 17:40:56 UTC 
Description of problem:

When updating a membership attribute of a group, impacted members will be updated (update of 'memberof'). It triggers internal searches of groups to compute their ancestors. But the same groups a searched multiple times.
The problem is described in http://www.port389.org/docs/389ds/design/memberof-scalability.html

Adding a cache that holds the ancestors of each group, would reduce the number of internal searches

Version-Release number of selected component (if applicable):

All versions

How reproducible:

 
Steps to Reproduce:
1. download https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py,

2. edit it with
users=1000,
groups=100,
groups_per_user=100,
nested_groups_max_level=2,
nested_groups_per_user=5,
hosts=1000,
hostgroups=100,
hostgroups_per_host=100,
nested_hostgroups_max_level=2,
nested_hostgroups_per_host=5,
direct_sudorules=5,  # users, hosts
indirect_sudorules=50,  # groups, hostgroups
sudorules_per_user=5,
sudorules_per_group=10,
sudorules_per_host=5,
sudorules_per_hostgroup=10,
direct_hbac=5,  # users, hosts
indirect_hbac=50,  # groups, hostgroups
hbac_per_user=5,
hbac_per_group=10,
hbac_per_host=5,
hbac_per_hostgroup=10

3. turn on internal op logging (to count the internal searches)

4. create a ldif file and import the entry (ldapadd)

Actual results:




Expected results:


Additional info:
Comment 1 thierry bordaz 2016-12-01 17:42:01 UTC 
upstream tickets

https://fedorahosted.org/389/ticket/49031
Comment 2 thierry bordaz 2017-01-11 15:26:41 UTC 
Patch pushed upstream https://fedorahosted.org/389/ticket/49031#comment:12
Comment 5 Sankar Ramalingam 2017-05-19 05:21:03 UTC 
Made few changes to ticket48861_test.py script and verified from Pytest.
https://pagure.io/389-ds-base/issue/49031#comment-127558
https://pagure.io/389-ds-base/issue/raw/files/7db1d4f6ce18f73b95f2299ee45247cb8cf46b44def08a5efd274205b931e9c2-ticket48861_test.py

[0 root@qeos-125 tickets]# py.test  -v ticket48861_test.py 
=================== test session starts ======================
platform linux2 -- Python 2.7.5, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /usr/bin/python
cachedir: .cache
metadata: {'Python': '2.7.5', 'Platform': 'Linux-3.10.0-663.el7.x86_64-x86_64-with-redhat-7.4-Maipo', 'Packages': {'py': '1.4.33', 'pytest': '3.0.7', 'pluggy': '0.4.0'}, 'Plugins': {'beakerlib': '0.7.1', 'html': '1.14.2', 'cov': '2.5.1', 'metadata': '1.5.0'}}
DS build: 1.3.6.1
389-ds-base: 1.3.6.1-13.el7
nss: 3.28.4-8.el7
nspr: 4.13.1-1.0.el7_3
openldap: 2.4.44-4.el7
svrcore: 4.1.3-2.el7

rootdir: /mnt/testarea/test/ds/dirsrvtests/tests/tickets, inifile:
plugins: metadata-1.5.0, html-1.14.2, cov-2.5.1, beakerlib-0.7.1
collected 2 items 

ticket48861_test.py::test_ticket48861 PASSED
ticket48861_test.py::test_ticket48861_final PASSED
=================== 2 passed in 11.59 seconds ===================
Comment 6 thierry bordaz 2017-05-19 09:05:36 UTC 
Hi Sankar,

thanks for the updating tests and results.
Just for confirmation, this BZ is mainly a performance oriented bug. When you switch it to 'verified' does that mean you completed performance testings ?

regards
Comment 7 Viktor Ashirov 2017-05-19 12:03:01 UTC 
Thierry, you're right, I'm switching bz back to ON_QA.

Sankar, please change status to VERIFIED, once you provide performance test results.

Thanks.
Comment 9 errata-xmlrpc 2017-08-01 21:12:24 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2086
Note You need to log in before you can comment on or make changes to this bug.