Description of problem:
There is a need to document necessary tuning changes to the connection pool and db connection settings for large RHV deployments with moderate amount of hosts and user concurrency.
Default connection pool and max_connection settings will become saturated in large RHV deployments.
Several times in the past it was seen in large environments with many hosts, that users may receive 'Unable to get managed connection for java:/ENGINEDataSource' in the /var/log/ovirt-engine/engine.log.
#The effects of this issue presents in several different scenarios at scale such as:
1) 'mass outages' where the engine needs to re-establish connection with hosts, users will notice the overall enviroment appears 'slow/sluggish' and some hosts may remain in non responsive state.
2) sluggish UI for the webadmin on the engine and then overtime eventual inability to log in to the webadmin .
3) noticeable delays in loading initial webadmin logon page after service ovirt-engine restart.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Setup env with 100 hosts or more # much easier to reproduce with 200.
2. Activate Hosts concurrently from UI
3. Connection pool usage will increase, and eventually become saturated once additional hosts, or concurrency is involved at default connection pool and db connection settings.
Receive 'Unable to get managed connection for java:/ENGINEDataSource'
Connection pool saturation on engine.
Normal operation without exception or sluggish behavior due to connection pool saturation.
I think this is already documented by Avital here
max_connections = 250
This document is being reviewed.