Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2184046

Summary: Slow search when using filter with a virtual attribute (eg: nsRole ).
Product: Red Hat Directory Server Reporter: Têko Mihinto <tmihinto>
Component: 389-ds-baseAssignee: thierry bordaz <tbordaz>
Status: CLOSED ERRATA QA Contact: LDAP QA Team <idm-ds-qe-bugs>
Severity: urgent Docs Contact: Evgenia Martynyuk <emartyny>
Priority: high    
Version: 11.6CC: bsmejkal, emartyny, idm-ds-dev-bugs, mgokhool, mreynolds, msauton, pasik, tbordaz, vashirov
Target Milestone: DS12.3Keywords: FutureFeature, Triaged
Target Release: dirsrv-12.3   
Hardware: x86_64   
OS: Linux   
Whiteboard: sync-to-jira
Fixed In Version: redhat-ds-12-9030020230711000312-1674d57 Doc Type: Enhancement
Doc Text:
Feature: A filter component may contain 'nsrole' attribute that can not be indexed. WIth this feature the component is rewritten into another component that can be indexed. This RFE only applies to managed and filtered roles but not to nested roles. Reason: the virtual attribute 'nsrole' is supported in filter component. As a virtual attribute it can not be indexed and the component is unindexed. If the others components of the filter are not indexed then the search will be unindexed and result in very long operation. Result: 'nsroles' attribute used in filter component are rewritten and then can be indexed. Resulting with better response time. Only for managed and filtered roles.
Story Points: ---
Clone Of:
: 2265530 2265536 (view as bug list) Environment:
Last Closed: 2023-11-21 15:13:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2265530, 2265536    

Description Têko Mihinto 2023-04-03 13:43:32 UTC
Description of problem:

This customer has an LDAP DB of 1.2 million of entries.
There are applications which are running queries using the nsRole virtual attribute.
For instance:
SRCH base="dc=example,dc=com" scope=2 filter="(&(nsRole=cn=NSROLE,dc=example,dc=com)(objectClass=USER))" attrs="distinguishedName "

The initial query is taking too long to complete and applications are hitting their timeouts.
Subsequent queries are faster.

1st run:
SRCH base="dc=example,dc=com" scope=2 filter="(&(nsRole=cn=NSROLE,dc=example,dc=com)(objectClass=USER))" attrs="distinguishedName "
RESULT err=0 tag=101 nentries=11 wtime=0.000070735 optime=179.484774853 etime=179.484840590 notes=A details="Fully Unindexed Filter"

Following queries
SRCH base="dc=example,dc=com" scope=2 filter="(&(nsRole=cn=NSROLE,dc=example,dc=com)(objectClass=USER))" attrs="distinguishedName"
RESULT err=0 tag=101 nentries=11 wtime=0.000060507 optime=0.558663030 etime=0.558718327 notes=A details="Fully Unindexed Filter


Version-Release number of selected component (if applicable):

$ cat <SOS_REPORT>/etc/redhat-release
Red Hat Enterprise Linux release 8.6 (Ootpa)
$
$ grep ^389-ds <SOS_REPORT>/installed-rpms
389-ds-base-1.4.3.29-3.module+el8dsrv+14615+a86efbbf.x86_64 Mon Sep  5 10:44:21 2022
389-ds-base-libs-1.4.3.29-3.module+el8dsrv+14615+a86efbbf.x86_64 Mon Sep  5 10:44:19 2022
$

How reproducible:
Always.

Steps to Reproduce:
1. Create a DB with a million of entries
2. Add the nsRoleDN attribute to 20K entries
3. Run searches using the nsRole attribute

Actual results:
Slow searches

Expected results:
Faster results so applications won't timeout.
After increasing the Normalized DN cache, subsequent queries are faster.
Now the concern is about how to improve the first search.

Additional info:
* Upstream ticket:
    https://github.com/389ds/389-ds-base/issues/5695
* Indexing and virtual attributes:
    https://bugzilla.redhat.com/show_bug.cgi?id=1773643

Comment 21 thierry bordaz 2023-05-09 13:25:54 UTC
Fix is pushed upstream -> POST

Comment 28 Viktor Ashirov 2023-08-17 09:43:02 UTC
Automated test passed:
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.9.17, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
389-ds-base: 2.3.5-1.module+el9dsrv+19320+04706864
nss: 3.90.0-3.el9_2
nspr: 4.35.0-3.el9_2
openldap: 2.6.3-1.el9
cyrus-sasl: not installed
FIPS: disabled
rootdir: /root/ds/dirsrvtests, configfile: pytest.ini
collected 1 item

dirsrvtests/tests/suites/roles/basic_test.py::test_managed_and_filtered_role_rewrite PASSED                                                                                                                  [100%]

==================================================================================== 1 passed, 12 warnings in 72.19s (0:01:12) =====================================================================================

Marking as VERIFIED.

Comment 30 errata-xmlrpc 2023-11-21 15:13:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (redhat-ds:12 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:7429