Red Hat Bugzilla – Bug 959115
Database connection pool leaks under load
Last modified: 2013-11-26 22:35:27 EST
Description of problem:
Connections are being leaked occasionally in production, until the pool is exhausted.
Version-Release number of selected component (if applicable):
version 2.3.0 (20130426-0119)
Requires several days of production load.
Steps to Reproduce:
1. Boot Zanata
2. Allow users to log in to the editor and generate load
3. Keep watching https://zanata.example.com/admin/monitoring?part=graph&graph=usedConnections&period=semaine every few hours
The number of used connections goes up and down depending on load, but gradually creeps higher and higher (most connections are returned to the pool, only some of them are leaked), until the pool is exhausted and any database operation causes an error.
During quiet periods, the number of used connections should drop. If the server is completely idle, it should drop to 1 (the minimum).
Fixed in https://github.com/zanata/zanata/commit/6bf6c409f24cf4c688a666d6955f39f66b66d9ea in release branch.
Fixed another potential culprit in release branch: https://github.com/zanata/zanata/commit/c4fb479347a6391c40bf39d540d34a9eed2db6b3
When testing this, please verify the functionality of the translation state cache:
validation status, last translated date and last translator in the doc list
and also up to date TM results when TextFlows have just become translated or untranslated in the editor.
Other than that, please watch out for errors like this in the server log:
[org.jboss.resource.connectionmanager.CachedConnectionManager] (http-0.0.0.0-8080-23) Closing a connection for you. Please close them yourself: org.jboss.resource.adapter.jdbc.jdk6.WrappedConnectionJDK6@1a4f4082
Note that this error message will only be generated if the debug option is true in jboss-as-web/server/production/deploy/jca-jboss-beans.xml (which it is on our test machines)
VERIFIED with Zanata version 3.0.0-SNAPSHOT (20130514-1433)
and Zanata version 2.3.2-SNAPSHOT (20130514-0010).
Closing VERIFIED bugs for Zanata versions <= 3.1.