Bug 1404074 - Monitor the DB connection pool usage
Summary: Monitor the DB connection pool usage
Keywords:
Status: CLOSED DUPLICATE of bug 1411570
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-oslo-db
Version: 6.0 (Juno)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 6.0 (Juno)
Assignee: Michael Bayer
QA Contact: Amit Ugol
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-12-13 01:35 UTC by PURANDHAR SAIRAM MANNIDI
Modified: 2020-09-10 10:02 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-06-06 13:45:46 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description PURANDHAR SAIRAM MANNIDI 2016-12-13 01:35:59 UTC
Description of problem:
While deploying stack from heat, following error was  observed
016-11-14 14:00:09.917 30197 INFO heat.engine.service [req-2b65a0ad-fc9e-4bf8-bb0f-246849078233 None] Creating stack Etcd-1.0.24-eDQ1ipYM1Ag
2016-11-14 14:00:30.874 30197 ERROR heat.engine.resource [-] DB error QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30
2016-11-14 14:00:31.138 30197 ERROR heat.engine.resource [-] Error marking resource as failed

The Customer wants to monitor the DB connection pool usage during the deployment
of templates using heat.

Customer was suggested with following command:

lsof -i4:3306 -sTCP:ESTABLISHED -c ^haproxy -c ^mysqld +c0 | sed '1d' | awk '{ print $1 }' | uniq -c | sort -k 2

Customer would like to have better approach to monitor DB connection pool.

Version-Release number of selected component (if applicable):
RH OSP 6.0

How reproducible:
Always

Steps to Reproduce:
1. deploy a stack with large template file

Actual results:
Horizon was inaccessible because of heat error mentioned above

Expected results:
Successful deployment of instances using templates

Additional info:
The issue was fixed by increasing max_pool_size=20 and max_overflow=30.
Customer still wants to monitor the DB connection pool.

Comment 1 Zane Bitter 2016-12-13 19:54:28 UTC
Note that this was fixed in Mitaka (OSP 9) so that the max overflow size is always at least the size of the thread pool: http://git.openstack.org/cgit/openstack/heat/commit/?id=d7c2de60cb9ed0d3ddb077d603b7028f8d91d238

Heat uses Oslo libraries both for managing DB connections (oslo_db) and for providing debugging information at runtime (oslo_service). It wouldn't make sense to implement anything directly in Heat to report on connection usage, since that is not specific to Heat, so I'm changing the component to python-oslo-db.

Since the customer rejected your solution, perhaps they could elaborate on what they _would_ consider an acceptable solution?

Comment 25 Michael Bayer 2017-06-06 13:45:46 UTC
RFE is at bz#1411570

*** This bug has been marked as a duplicate of bug 1411570 ***


Note You need to log in before you can comment on or make changes to this bug.