Bug 1952241

Summary: RFE - Increase the default IDL scan limit and evaluate the possibility of a dynamic limit.
Product: Red Hat Directory Server Reporter: Têko Mihinto <tmihinto>
Component: 389-ds-baseAssignee: Pierre Rogier <progier>
Status: CLOSED ERRATA QA Contact: LDAP QA Team <idm-ds-qe-bugs>
Severity: medium Docs Contact: Evgenia Martynyuk <emartyny>
Priority: low    
Version: 11.2CC: aadhikar, emartyny, idm-ds-dev-bugs, mreynolds, pasik, pcech, progier, vashirov
Target Milestone: ---Keywords: Triaged
Target Release: dirsrv-12.2   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: redhat-ds-12-9020020230314150545.1674d574 Doc Type: Deprecated Functionality
Doc Text:
.The `nsslapd-idlistscanlimit` parameter is deprecated and its default value has been changed With the new filter reordering optimization, the `nsslapd-idlistscanlimit` attribute impact on search performance is more harmful than helpful. As a result, the attribute is deprecated. Additionally, the default value has been changed to `2147483646` (unlimited).
 Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-30 09:40:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Têko Mihinto 2021-04-21 20:29:38 UTC 
Description of problem:

Customers are storing more and more data in LDAP.
In a recent customer ticket there we 21 million of entries and the IDL scan limit was set at the default value of 4000.
About 4300 entries ( 0.02 % ) were matching a filter, so the related search was unindexed.
In this peculiar case this was an internal search to build the ACI cache, thus the unindexed search
was causing a delay of 20 to 30 minutes before the LDAP server could be started. 


Version-Release number of selected component (if applicable):
$ grep 389-ds-base-1 installed-rpms 
389-ds-base-1.4.3.13-1.module+el8dsrv+8334+69a46a2e.x86_64  Sun Feb 28 18:01:43 2021
$

How reproducible:
Always.

Steps to Reproduce:
1. In a large DB, make sure to have more entries with the ldapsubentry objectClass than the IDL scan limit

2. The startup should take minutes due to an internal unindexed search that is used to build the ACI cache

Actual results:
Long startup time.

Expected results:
Fast startup.

Additional info:
This RFE is to evaluate if the default value could be increased ( maybe to 10K )
and if a dynamic behavior could be implemented ( X % of the total number of entries in a suffix would be indexed
with a hard limit of 50K? )
Comment 3 mreynolds 2023-02-08 16:14:59 UTC 
Upstream ticket:

https://github.com/389ds/389-ds-base/issues/2435
Comment 7 Viktor Ashirov 2023-05-09 10:24:13 UTC 
Build tested:
389-ds-base-2.2.7-2.module+el9dsrv+18726+78959e84.x86_64

On a default installation IDL scan limit is raised to INT_MAX:
# ldapsearch -x -D "cn=Directory Manager" -w password -b cn=config | grep nsslapd-idlistscanlimit:
nsslapd-idlistscanlimit: 2147483646

Marking as VERIFIED.
Comment 8 Evgenia Martynyuk 2023-05-10 08:49:11 UTC 
Hi Pierre! @progier

Could you please review the DoxText field with the RN draft. 

Thanks, 
Evgenia
Comment 15 errata-xmlrpc 2023-05-30 09:40:35 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (redhat-ds:12 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:3344