Ding-Yi Chen 2013-10-14 11:46:02 EST: Connection leak is still in production system. Version-Release number of selected component (if applicable): Zanata version 3.1.1-SNAPSHOT (20131011-1233) How reproducible: Requires several days of production load. Steps to Reproduce: 1. Boot Zanata 2. Allow users to log in to the editor and generate load Expected: No connection leak. Actual: Connection leak still happening, which stops new connections.
Correction, the affected version is: Zanata version 3.0.3 (20130919-2218)
Correction, the https://zanata.example.com/admin/monitoring?part=graph&graph=usedConnections&period=semaine still works. Steps to get to the monitoring graph: 1. Log in with admin role. 2. Go to Administration menu. 3. Click server monitoring 4. Click the diagram "Used JDBC connection"
Simultaneously launching two instances of the same editor (same language) from a project with many documents seems to be fairly reliable at reproducing the exception, as long as the relevant documents/statistics haven't been cached already. (You can use JavaMelody to clear all data caches.) 07:07:12,839 INFO [org.jboss.jca.core.connectionmanager.listener.TxConnectionListener] (http-/0.0.0.0:8080-7) IJ000311: Throwable from unregister connection: java.lang.IllegalStateException: Trying to return an unknown connection2! org.jboss.jca.adapters.jdbc.jdk6.WrappedConnectionJDK6@b0da531 Every time that exception is logged, another connection is leaked (or already was), although the cause is probably a little earlier, possibly in a different thread.
Pull request: https://github.com/zanata/zanata-server/pull/229
VERIFIED Zanata version 3.1.1-SNAPSHOT (20131017-1642)
*** Bug 1020585 has been marked as a duplicate of this bug. ***
Created attachment 815210 [details] When the connections were created (according to JavaMelody)
False alarm (I think). It appears that the connections are being retained deliberately by the DataSource, not leaked, because there are no warning messages in the log such as "Trying to return an unknown connection2!". Also, several of the connections were created at the same time, which may suggest that they were created to expand a connection pool (unless they *all* happened to leak at the same time). Looking at the stack traces, I can't find any suspicous code (eg ResultSet handling) that would explain such a leak. Unless we actually run out of connections (or get the above warning message again), I think we'll have to assume it's behaving normally. I think it's just normal DataSource connection pool behaviour. However, if we get the above log warning, or a server failure like "javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout" or "javax.resource.ResourceException: IJ000453: Unable to get managed connection for ..." we should create a new bug.
Closing VERIFIED bugs for Zanata versions <= 3.1.