Bug 842438

Summary: Report during startup if nsslapd-cachememsize is too small
Product: Red Hat Enterprise Linux 6 Reporter: Nathan Kinder <nkinder>
Component: 389-ds-baseAssignee: Rich Megginson <rmeggins>
Status: CLOSED ERRATA QA Contact: Sankar Ramalingam <sramling>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 6.4CC: jgalipea, jrusnack, nhosoi
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.2.11.12-1.el6 Doc Type: Bug Fix
Doc Text:
Cause: To improve the performance, the entry cache size is supposed to be larger than the primary database size if possible. But no checking or reporting was made. Consequence: Customers may not notice that the size of entry cache is small although the cache size could be configured larger. Fix: When starting the server, the configured entry size and the primary database size are examined. If the entry cache is too small, the warning is logged in the error log. Result: Customers have better chance to improve the performance.
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-21 08:20:16 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nathan Kinder 2012-07-23 20:56:43 UTC
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/409

The server should be able to compare the size of id2entry.db with the configured value of nsslapd-cachememsize and at least whine if the cache allocation is too small by default:

"You have XYZ bytes allocated to the cache. ABC is recommended"

Comment 1 Nathan Kinder 2012-07-23 21:01:47 UTC
Precise verification steps will need to be added after this issue is fixed, but the basics are as follows:

- import enough entries to make id2entry.db4 larger then the current nsslapd-cachememsize setting.
- restart the Directory Server instance and verify that the errors log has a message about the cache being too small (along with a recommended size.

It still needs to be determined is if a warning should be output to the terminal when start the DS instance.  There also may be some multiplier used against the id2entry.db4 size to determine how large the cache needs to be.  It might not be exactly 1.0x.

Comment 2 Noriko Hosoi 2012-07-25 19:38:36 UTC
Pushed to the external as well as the internal 389-ds-base-1.2.11 branch:

$ git push origin 389-ds-base-1.2.11-ext:389-ds-base-1.2.11 
Counting objects: 17, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.74 KiB, done.
Total 9 (delta 7), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   f16d3a0..43a71e3  389-ds-base-1.2.11-ext -> 389-ds-base-1.2.11

$ git push redhat 389-ds-base-1.2.11-int:389-ds-base-1.2.11
Counting objects: 17, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.77 KiB, done.
Total 9 (delta 7), reused 0 (delta 0)
To ssh://git.app.eng.bos.redhat.com/srv/git/389-ds-base.git
   9d17dbd..73ff8d1  389-ds-base-1.2.11-int -> 389-ds-base-1.2.11

Comment 3 Noriko Hosoi 2012-08-24 21:26:50 UTC
Jan Rusnacko wrote:
> Hi Noriko, hi Nathan,
>
> I am working on https://bugzilla.redhat.com/show_bug.cgi?id=842438
> automation - could either of you add more precise verification steps ?
> Thank you !

Hi Jan,

This is the description of the bug fix/enhancement (Please see also https://fedorahosted.org/389/ticket/409).  Basically, we just added a code to issue a warning if the cache size is smaller than the entire db size.

Replying to nhosoi:
 Let's assume we have 3 backends: userRoot, NetscapeRoot?, mycom.

Case 1) each entry cache size is larger than the size of id2entry.db.
By default, it reports nothing. If the log level 0x80000 == 524288 is set, these sizes are logged in the errors log:
[...] - userRoot: entry cache size: 1048576000B; db size: 271990784B
[...] - NetscapeRoot?: entry cache size: 104857600B; db size: 90112B
[...] - mycom: entry cache size: 104857600B; db size: 2744320B
[...] - Total entry cache size: 1258291200B; dbcache size: 10000000B; available memory size: 3961982976B

Case 2) one of the entry cache is less than the size of id2entry.db.
By default, it reports only the smaller cache size:
[...] - WARNING: userRoot: entry cache size 10485760B is less than db size 271990784B; We recommend to increase the entry cache size nsslapd-cachememsize.
If the log level 0x80000 == 524288 is set, it's logged in with other info:
[...] - WARNING: userRoot: entry cache size 10485760B is less than db size 271990784B; We recommend to increase the entry cache size nsslapd-cachememsize.
[...] - NetscapeRoot?: entry cache size: 104857600B; db size: 90112B
[...] - mycom: entry cache size: 104857600B; db size: 2744320B
[...] - Total entry cache size: 220200960B; dbcache size: 10000000B; available memory size: 3961982976B

Case 3) total cache is less than the physical memory size.
By default:
[...] - WARNING -- Possible CONFIGURATION ERROR -- cachesize (3688353792) may be configured to use more than the available physical memory.
Log level 0x80000:
[...] - WARNING -- Possible CONFIGURATION ERROR -- cachesize (3688353792) may be configured to use more than the available physical memory.
[...] - userRoot: entry cache size: 4048576000B; db size: 271990784B
[...] - NetscapeRoot?: entry cache size: 104857600B; db size: 90112B
[...] - mycom: entry cache size: 104857600B; db size: 2744320B
[...] - Total entry cache size: 4258291200B; dbcache size: 10000000B; available memory size: 3961982976B

Comment 4 Jenny Severance 2012-08-28 13:22:26 UTC
Test automation :: testcase 842438 in import/import.sh

Comment 6 Ján Rusnačko 2012-10-24 12:22:21 UTC
Steps to verify:

1) Change nsslapd-cachememsize to lowest possible value
ldapmodify -h localhost -p 22222 -D "cn=directory manager" -w "password" <<EOF
dn: cn=userRoot,cn=ldbm database,cn=plugins,cn=config
changetype: modify
replace: nsslapd-cachememsize
nsslapd-cachememsize: 512000
EOF

2) Import many entries
ldif2db -n userRoot -i 842438.ldif 

3) 
cat $log_dir/errors | grep "We recommend to increase the entry cache size nsslapd-cachememsize"
echo $?
0

[jrusnack@dstet 6.0]$ rpm -qa | grep 389
389-ds-base-1.2.11.15-2.el6.x86_64

Verified

Comment 7 errata-xmlrpc 2013-02-21 08:20:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0503.html