Bug 842438 - Report during startup if nsslapd-cachememsize is too small
Report during startup if nsslapd-cachememsize is too small
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: 389-ds-base (Show other bugs)
6.4
Unspecified Unspecified
medium Severity unspecified
: rc
: ---
Assigned To: Rich Megginson
Sankar Ramalingam
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-07-23 16:56 EDT by Nathan Kinder
Modified: 2013-02-21 03:20 EST (History)
3 users (show)

See Also:
Fixed In Version: 389-ds-base-1.2.11.12-1.el6
Doc Type: Bug Fix
Doc Text:
Cause: To improve the performance, the entry cache size is supposed to be larger than the primary database size if possible. But no checking or reporting was made. Consequence: Customers may not notice that the size of entry cache is small although the cache size could be configured larger. Fix: When starting the server, the configured entry size and the primary database size are examined. If the entry cache is too small, the warning is logged in the error log. Result: Customers have better chance to improve the performance.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-02-21 03:20:16 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Nathan Kinder 2012-07-23 16:56:43 EDT
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/409

The server should be able to compare the size of id2entry.db with the configured value of nsslapd-cachememsize and at least whine if the cache allocation is too small by default:

"You have XYZ bytes allocated to the cache. ABC is recommended"
Comment 1 Nathan Kinder 2012-07-23 17:01:47 EDT
Precise verification steps will need to be added after this issue is fixed, but the basics are as follows:

- import enough entries to make id2entry.db4 larger then the current nsslapd-cachememsize setting.
- restart the Directory Server instance and verify that the errors log has a message about the cache being too small (along with a recommended size.

It still needs to be determined is if a warning should be output to the terminal when start the DS instance.  There also may be some multiplier used against the id2entry.db4 size to determine how large the cache needs to be.  It might not be exactly 1.0x.
Comment 2 Noriko Hosoi 2012-07-25 15:38:36 EDT
Pushed to the external as well as the internal 389-ds-base-1.2.11 branch:

$ git push origin 389-ds-base-1.2.11-ext:389-ds-base-1.2.11 
Counting objects: 17, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.74 KiB, done.
Total 9 (delta 7), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   f16d3a0..43a71e3  389-ds-base-1.2.11-ext -> 389-ds-base-1.2.11

$ git push redhat 389-ds-base-1.2.11-int:389-ds-base-1.2.11
Counting objects: 17, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.77 KiB, done.
Total 9 (delta 7), reused 0 (delta 0)
To ssh://git.app.eng.bos.redhat.com/srv/git/389-ds-base.git
   9d17dbd..73ff8d1  389-ds-base-1.2.11-int -> 389-ds-base-1.2.11
Comment 3 Noriko Hosoi 2012-08-24 17:26:50 EDT
Jan Rusnacko wrote:
> Hi Noriko, hi Nathan,
>
> I am working on https://bugzilla.redhat.com/show_bug.cgi?id=842438
> automation - could either of you add more precise verification steps ?
> Thank you !

Hi Jan,

This is the description of the bug fix/enhancement (Please see also https://fedorahosted.org/389/ticket/409).  Basically, we just added a code to issue a warning if the cache size is smaller than the entire db size.

Replying to nhosoi:
 Let's assume we have 3 backends: userRoot, NetscapeRoot?, mycom.

Case 1) each entry cache size is larger than the size of id2entry.db.
By default, it reports nothing. If the log level 0x80000 == 524288 is set, these sizes are logged in the errors log:
[...] - userRoot: entry cache size: 1048576000B; db size: 271990784B
[...] - NetscapeRoot?: entry cache size: 104857600B; db size: 90112B
[...] - mycom: entry cache size: 104857600B; db size: 2744320B
[...] - Total entry cache size: 1258291200B; dbcache size: 10000000B; available memory size: 3961982976B

Case 2) one of the entry cache is less than the size of id2entry.db.
By default, it reports only the smaller cache size:
[...] - WARNING: userRoot: entry cache size 10485760B is less than db size 271990784B; We recommend to increase the entry cache size nsslapd-cachememsize.
If the log level 0x80000 == 524288 is set, it's logged in with other info:
[...] - WARNING: userRoot: entry cache size 10485760B is less than db size 271990784B; We recommend to increase the entry cache size nsslapd-cachememsize.
[...] - NetscapeRoot?: entry cache size: 104857600B; db size: 90112B
[...] - mycom: entry cache size: 104857600B; db size: 2744320B
[...] - Total entry cache size: 220200960B; dbcache size: 10000000B; available memory size: 3961982976B

Case 3) total cache is less than the physical memory size.
By default:
[...] - WARNING -- Possible CONFIGURATION ERROR -- cachesize (3688353792) may be configured to use more than the available physical memory.
Log level 0x80000:
[...] - WARNING -- Possible CONFIGURATION ERROR -- cachesize (3688353792) may be configured to use more than the available physical memory.
[...] - userRoot: entry cache size: 4048576000B; db size: 271990784B
[...] - NetscapeRoot?: entry cache size: 104857600B; db size: 90112B
[...] - mycom: entry cache size: 104857600B; db size: 2744320B
[...] - Total entry cache size: 4258291200B; dbcache size: 10000000B; available memory size: 3961982976B
Comment 4 Jenny Galipeau 2012-08-28 09:22:26 EDT
Test automation :: testcase 842438 in import/import.sh
Comment 6 Ján Rusnačko 2012-10-24 08:22:21 EDT
Steps to verify:

1) Change nsslapd-cachememsize to lowest possible value
ldapmodify -h localhost -p 22222 -D "cn=directory manager" -w "password" <<EOF
dn: cn=userRoot,cn=ldbm database,cn=plugins,cn=config
changetype: modify
replace: nsslapd-cachememsize
nsslapd-cachememsize: 512000
EOF

2) Import many entries
ldif2db -n userRoot -i 842438.ldif 

3) 
cat $log_dir/errors | grep "We recommend to increase the entry cache size nsslapd-cachememsize"
echo $?
0

[jrusnack@dstet 6.0]$ rpm -qa | grep 389
389-ds-base-1.2.11.15-2.el6.x86_64

Verified
Comment 7 errata-xmlrpc 2013-02-21 03:20:16 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0503.html

Note You need to log in before you can comment on or make changes to this bug.