Bug 1395940

Summary:	[RFE] Autotune threads at startup
Product:	Red Hat Enterprise Linux 7	Reporter:	Noriko Hosoi <nhosoi>
Component:	389-ds-base	Assignee:	wibrown <wibrown>
Status:	CLOSED ERRATA	QA Contact:	Viktor Ashirov <vashirov>
Severity:	unspecified	Docs Contact:	Aneta Šteflová Petrová <apetrova>
Priority:	unspecified
Version:	7.3	CC:	abokovoy, mreynolds, nkinder, pasik, rmeggins, spichugi, wibrown
Target Milestone:	rc	Keywords:	FutureFeature
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
URL:	http://www.port389.org/docs/389ds/design/autotuning.html
Whiteboard:
Fixed In Version:	389-ds-base-1.3.6.1-3.el7	Doc Type:	Enhancement
Doc Text:	Improved performance of the IdM server The Identity Management (IdM) server has a higher performance across many of the common workflows and setups. These improvements include: * Vault performance has been increased by reducing the round trips within the IdM server management framework. * The IdM server management framework has been tuned to reduce the time spent in internal communication and authentication. * The Directory Server connection management has been made more scalable with the use of the nunc-stans framework. * On new installations, the Directory Server now auto-tunes the database entry cache and the number of threads based on the hardware resources of the server. * The memberOf plug-in performance has been improved when working with large or nested groups.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-08-01 21:12:24 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Noriko Hosoi 2016-11-17 00:45:39 UTC

This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/49021

We hardcode a default threadnumber of 30. This is a pretty conservative default, but it's not great. 

Modern hardware is much larger than this, and people are not tuning their systems to match (see also FreeIPA). We should be doing this automatically.

We should make two changes.

First, is that we should change the default thread count from 30 to 0 (-1). 

When 0 or -1 is set, we should take:
- number of hardware threads * factor

As the thread count. That factor may change, and should be tested. We have previously advised factor == 2, but with modern CPU's it may be better to put this to 3 or 4. 

This would be a pretty simple change code wise, but it would take some time and testing on a few different hardware platforms to get it right.

Comment 6 Simon Pichugin 2017-06-07 17:08:28 UTC

Test suite was automated at dirsrvtests/tests/suites/config/autotuning_test.py

[root@qeos-99 ds]# py.test -v -k threads dirsrvtests/tests/suites/config/autotuning_test.py
====================== test session starts ======================
platform linux2 -- Python 2.7.5, pytest-3.1.1, py-1.4.34, pluggy-0.4.0 -- /usr/bin/python
cachedir: .cache
metadata: {'Python': '2.7.5', 'Platform': 'Linux-3.10.0-675.el7.x86_64-x86_64-with-redhat-7.4-Maipo', 'Packages': {'py': '1.4.34', 'pytest': '3.1.1', 'pluggy': '0.4.0'}, 'Plugins': {'beakerlib': '0.7.1', 'html': '1.14.2', 'cov': '2.5.1', 'metadata': '1.5.0'}}
DS build: 1.3.6.1
389-ds-base: 1.3.6.1-16.el7
nss: 3.28.4-8.el7
nspr: 4.13.1-1.0.el7_3
openldap: 2.4.44-4.el7
svrcore: 4.1.3-2.el7

rootdir: /mnt/tests/rhds/tests/upstream/ds, inifile:
plugins: metadata-1.5.0, html-1.14.2, cov-2.5.1, beakerlib-0.7.1
collected 18 items

dirsrvtests/tests/suites/config/autotuning_test.py::test_threads_basic PASSED
dirsrvtests/tests/suites/config/autotuning_test.py::test_threads_invalid_value[-2] PASSED
dirsrvtests/tests/suites/config/autotuning_test.py::test_threads_invalid_value[0] PASSED
dirsrvtests/tests/suites/config/autotuning_test.py::test_threads_invalid_value[invalid] PASSED
dirsrvtests/tests/suites/config/autotuning_test.py::test_threads_back_from_manual_value PASSED

====================== 13 tests deselected ======================
============ 5 passed, 13 deselected in 4.24 seconds ============

Tests are passed. Also, no regressions were present. Marking as verified.

Comment 7 errata-xmlrpc 2017-08-01 21:12:24 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2086