Bug 503615 - Need more detail about dbcachesize, cachememsize
Need more detail about dbcachesize, cachememsize
Status: CLOSED CURRENTRELEASE
Product: Red Hat Directory Server
Classification: Red Hat
Component: Doc-administration-guide (Show other bugs)
8.2
All Linux
high Severity medium
: ---
: ---
Assigned To: Deon Ballard
ecs-bugs
: Documentation
Depends On:
Blocks: 555930 639035
  Show dependency treegraph
 
Reported: 2009-06-01 18:42 EDT by Rich Megginson
Modified: 2011-06-14 22:37 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-06-14 22:37:09 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Rich Megginson 2009-06-01 18:42:30 EDT
Admin Guide section 14.4.1, 14.4.2, 16.2.1, 16.4

CCF Reference sections 2.3.2, 3.4.1.3, 3.4.1.4, 3.4.1.5, 3.4.1.23, 3.4.3, 3.4.3.1, 3.4.3.2, 3.4.6

We need to better explain what exactly the dbcachesize is, what it does, and how you set this in the UI.  The same applies to cachememsize.

We also need an image of the per-database configuration tab - Configuration Tab->Data node->suffix (e.g. dc=example,dc=com)->Database name (e.g. userRoot) -> Database Settings tab.  We have an image of the LDBM Plug-in Settings tab just not the Database Settings tab.

We should say something like this in section 16.4:

"If you have enough memory (RAM), you will get the most search performance benefit by making the entry cache large enough to contain all entries in the database.  The entry cache size is set via the 'Memory available for cache' setting in the 'Database Settings' tab for each database node under each suffix listed under 'Data' in the main directory server 'Configuration' tab.  This is also set via the attribute nsslapd-cachememsize in the entry cn=<database name>,cn=ldbm database,cn=plugins,cn=config.  The entry cache contains the entries in their internal representation.  If there is no caching, the server must read the entry from the entry index (currently id2entry.db4) which involves disk I/O, then it must convert the entry from the on-disk format (currently LDIF) into the in-memory representation (Slapi_Entry).  If the entry is in the entry cache, the server can avoid the expensive disk I/O and conversion steps.  When setting this value, be sure to monitor the entry cache usage.

After optimizing your entry cache, if you still have some RAM available, consider increasing your database cache size, which is the amount of RAM used to cache the Berkeley database index files (the .db4 files in the database directory).  This is set via the 'Maximum cache size' setting in the LDBM Plug-in Settings tab under the 'Database Settings' node in the 'Data' node in the 'Configuration' tab of the main directory server Console window.  This is also set via the attribute nsslapd-dbcachesize in the entry cn=config, cn=ldbm database, cn=plugins,cn=config.  When setting this value, be sure to monitor the database cache usage.

Note that the operating system also has a file system cache which may compete with the database cache for RAM usage.  Please refer to your operating system documentation about file system cache settings and how to monitor your file system cache."

Then we should have a link to the Admin Guide section 14.4.1 and 14.4.2 which shows how to monitor the effects of the entry cache setting, and section 14.3.1 "Global Database Cache" which shows how to monitor the effects of the database cache setting.  Note that 14.3.2 is incomplete - it does not show how to monitor the "Global Database Cache" from the command line.  The entry is cn=monitor,cn=ldbm database,cn=plugins,cn=config and the attributes are documented in the CCF Reference in section 3.4.2.  In Table 14.5, the description for 'tries' is incorrect - change 'requests performed on the directory' to 'database accesses'  Table 14.8 needs to correct the descriptions of Entry Cache Hit Ratio and Maximum Entry Cache Size (in Bytes) - the correct UI field is called "Memory available for cache" and we should mention that this is the same as the attribute nsslapd-cachememsize in cn=<database name>,cn=ldbm database,cn=plugins,cn=config.
Table 14.11 also needs similar correction for the attributes entrycachehitratio, currententrycachesize, and maxentrycachesize.

Note that, in tables 14.8 and 14.11, we do not allow setting the number of entries cached, only the amount of memory used for caching.  We used to allow setting the caches based on the number of entries cached, but it was very difficult to gauge the RAM usage.  It is much easier to simply say "the cache uses X amount of RAM".  In table 14.8 we should get rid of Maximum Entry Cache Size (in Entries) since you cannot set this in the console, and just say that Current Entry Cache Size (in Entries) is the number of entries in the entry cache.
Comment 1 Rich Megginson 2009-06-02 12:12:11 EDT
Places in the docs (Admin Guide, CCF Ref) that refer to cache settings in terms of numbers of entries should be removed, if possible, or marked as "Deprecated - do not use" if not possible to remove them.

Places in the docs that refer to cache sizes should clearly state the units - bytes, megabytes, gigabytes, whatever.

Can we have links to Berkeley DB documentation in our docs?  If so, the doc for the set_cachesize function (http://www.oracle.com/technology/documentation/berkeley-db/db/api_c/env_set_cachesize.html) is a good description of what the nsslapd-dbcachesize setting does.  Also, the page http://www.oracle.com/technology/documentation/berkeley-db/db/ref/am_conf/cachesize.html is a good explanation of what the cache does, and how to monitor the cache usage using db_stat -m.
Comment 2 Deon Ballard 2009-08-28 16:53:15 EDT
The biggest change was adding the cache tuning section, as section 16.3 (formerly section 16.4):
http://www.redhat.com/docs/manuals/dir-server/8.1/admin/memoryusage.html

That has most of the text from the description, the new screenshot, and links to section 14.4 and the Berkeley DB documentation.

This also led to some edits in section 16.2.1, for tuning search parameters, mainly adding links to the cache tuning section:
http://www.redhat.com/docs/manuals/dir-server/8.1/admin/Tuning_DS_Performance-Tuning_Database_Performance.html#Tuning_Database_Performance-Optimizing_Search_Performance

There were some problems in tables which were fixed:
* table 14.5: http://www.redhat.com/docs/manuals/dir-server/8.1/admin/Monitoring_Server_and_Database_Activity-Monitoring_Server_Activity.html#tab.Global_Database_Cache_Information
* table 14.8: http://www.redhat.com/docs/manuals/dir-server/8.1/admin/Monitoring_Server_and_Database_Activity-Monitoring_Database_Activity.html#tab.Summary_Information_Table
* table 14.11: http://www.redhat.com/docs/manuals/dir-server/8.1/admin/Monitoring_Server_and_Database_Activity-Monitoring_Database_Activity.html#tab.db-command-line

I added information to a couple of the parameters for the units (bytes) used in cache sizing in table 14.11, as mentioned in comment #1. I also added mention of the units to the CCFR in sections 3.4.1.5 and 3.4.3.2.

The only attribute I could see that explicitly stated that it managed the cache by the number of entries was nsslapd-cachesize. I changed that attribute write-up to say it was deprecated and to use nsslapd-cachememsize instead and removed the reference in the cn=changelog5 section to the cachesize attribute, revising the cachemesize references slightly: 
* http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Core_Server_Configuration_Reference-Core_Server_Configuration_Attributes_Reference.html#Configuration_Command_File_Reference-Core_Server_Configuration_Attributes_Reference-cnchangelog5
* http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html#Configuration_Command_File_Reference-Database_Attributes_under_cnNetscapeRoot_cnldbm_database_cnplugins_cnconfig_and_cnUserRoot_cnldbm_database_cnplugins_cnconfig-nsslapd_cachesize

I also added links to BerkeleyDB docs for nsslapd-dbcachesize and nsslapd-cachememsize:
* http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html#Configuration_Command_File_Reference-Database_Plug_in_Attributes-Database_Attributes_under_cnconfig_cnldbm_database_cnplugins_cnconfig
* http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html#Configuration_Command_File_Reference-Database_Attributes_under_cnNetscapeRoot_cnldbm_database_cnplugins_cnconfig_and_cnUserRoot_cnldbm_database_cnplugins_cnconfig-nsslapd_cachememsize

There were several sections listed in the description for the CCFR (3.4.1.3 - nsslapd-cache-autosize, 3.4.1.4 - nsslapd-cache-autosize-split, 3.4.1.23 - nsslapd-import-cache-autosize, 3.4.3 - the intro for attributes for cn=NetscapeRoot, cn=ldbm database,..., and 3.4.6 - the section for attributes in cn=monitor, cn=NetscapeRoot, cn=ldbm database,...). 
I cannot find any place in these sections that need updated with the other changes for nsslapd-cachememsize or nsslapd-dbcachesize, so I didn't make any changes there.

I think this covers everything in the bug. Changing status to modified.
Comment 4 Deon Ballard 2010-02-19 12:01:11 EST
Setting version to 8.2.
Comment 7 Deon Ballard 2011-03-04 13:29:45 EST
I'm assigning these to Andrew Ross to verify and close. All of these changes were done when 8.2 was released last year and are ready to be closed.
Comment 11 Andrew Ross 2011-06-14 22:37:09 EDT
Docs are live. URLs as per comment#6. Closing.

Note You need to log in before you can comment on or make changes to this bug.