Description of problem:
* A customer has about 250K entries in a large static group.
* The MemberOf Plugin is enabled and is configured to process all suffixes.
* The number of DB locks is set to 100K
* The Referential Integrity Plugin is enabled and configured as below:
dn: cn=referential integrity postoperation,cn=plugins,cn=config
cn: referential integrity postoperation
nsslapd-pluginVendor: 389 Project
nsslapd-pluginDescription: referential integrity plugin
Version-Release number of selected component (if applicable):
Customer RHEL and 389-ds versions:
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.7 (Maipo)
$ grep 389-ds-base-1 installed-rpms
389-ds-base-18.104.22.168-12.el7_7.x86_64 Fri Dec 6 13:05:57 2019
Always when the Referential Integrity Plugin is enabled.
Not reproducible if RI plugin is disabled.
Steps to Reproduce:
1. Mimic the customer configuration
2. Run a MODRDN operation on a user in the large static group.
3. Monitor the number of DB locks and check the errors log
The LDAP instance runs out of DB locks:
[05/May/2020:14:03:10.389741470 +0200] - ERR - id2entry - db error 12 (Cannot allocate memory)
[05/May/2020:14:03:10.391192839 +0200] - ERR - ldbm_back_next_search_entry_ext - next_search_entry db err 12
[05/May/2020:14:03:10.392725298 +0200] - ERR - libdb - BDB2055 Lock table is out of available lock entries
[05/May/2020:14:03:10.394284112 +0200] - ERR - id2entry - db error 12 (Cannot allocate memory)
[05/May/2020:14:03:10.395875506 +0200] - ERR - ldbm_back_next_search_entry_ext - next_search_entry db err 12
[05/May/2020:14:03:10.397305021 +0200] - ERR - libdb - BDB2055 Lock table is out of available lock entries
The server might also become unresponsive and could not be stopped gracefully.
* Monitor the number of current DB locks and exits gracefully when getting close to the limit
* Automatically adjust the upper limit ( more complicated as a restart is required to take the new value into account ).
* RFE - Monitor the current DB locks ( nsslapd-db-current-locks ).
* This behavior is somehow expected when dealing with large groups:
* Some customers are experiencing DB corruption issue forcing them to reinitialize their large DB across the topology.
could you enable logging of internal operations.
I have the suspicion that one of the plugins does some extensive searches and they are done in the txn of the modrdn operation
Thanks Teko, sorry I had missed c2.
so this shows two things:
- memberof does a search of all backends, not sure if this is necessary or can be prvented by config (Thierry?)
- it does substring searches for member, uniquemember, owner, seealso, memberuid - and I don't think they all have substring indexes. Is it necessary to include owner and seealso in the member attrs ?
indeed config parameter memberOfAllBackends is likely turned 'on' (default is 'off') that explain all backend lookup. Is it necessary, it depends on how data are spread on backends. Also in such case substring index must be tuned on all backends. In case they are not indexed it leads to an unindexed search under a txn !