Bug 1003445

Summary:	PostgreSQL gear limits too low
Product:	OpenShift Online	Reporter:	Jason Shepherd <jshepherd>
Component:	Containers	Assignee:	Hiro Asari <hasari>
Status:	CLOSED UPSTREAM	QA Contact:	libra bugs <libra-bugs>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	2.x	CC:	ccoleman, hasari, jkeck
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-09-04 20:15:44 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jason Shepherd 2013-09-02 03:51:48 UTC

Description of problem:

The resource limits for Postgres are too low.  Max connections were set to only 10, shared buffer set to only 2MB, and effective_cache_size set to only 32. This can be found in postgresql.conf or a gear.


Version-Release number of selected component (if applicable):

Openshift Online 2.0.32
postgresql 9.2


How reproducible:

Create a DJango application that uses Postgres, and load test so that > 10 active DB connections are created.


Actual results:

In the New Relic logs you get:

psycopg2:OperationalError: FATAL: sorry, too many clients already


Expected results:

The application should be able to handle > 10 current connections. 

Additional Notes:

The settings should be configurable.

Comment 1 Clayton Coleman 2013-09-03 12:40:37 UTC

Based on discussion from the thread we need to make the following improvements to the default config:

The smallest config that is relevant is a small gear sharing space with the application server:

max_connections = 100
shared_buffers = 64 megs
effective_cache_size = 186 megs
work_mem = 3 megs

Larger sized gears and non-shared gears could use more of both.

Comment 2 Clayton Coleman 2013-09-03 12:56:14 UTC

For reference we were using out of the box parameters for postgresql - they were completely untuned for real deployments.

Comment 3 Hiro Asari 2013-09-04 14:18:24 UTC

What do limits for larger gears look like?

The jbossas cartridge, for example, sets:

max memory x0.5 for max heap size
max memory x0.2 for PermGen

Comment 4 Hiro Asari 2013-09-04 20:15:44 UTC

Adjusted currently insufficient parameters with https://github.com/openshift/origin-server/pull/3547

The rest will be handled with https://trello.com/c/kuZecQIv/269-provide-a-convenient-means-of-configuring-db-parameters-mysql-mongodb-postgresql

Comment 5 openshift-github-bot 2013-09-04 21:36:28 UTC

Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/c39f1491699d9e6abc928a69b3f845b6133d802e
Bump up PostgreSQL memory parameters

Bug 1003445
Values are discussed in
https://bugzilla.redhat.com/show_bug.cgi?id=1003445#c1

Larger gears should allow even larger values, but further discussions
are necessary.

Values are lower on embedded cartridges.