Bug 1143048

Summary:	[postgres] Classloader leak when postgres server inventoried
Product:	[Other] RHQ Project	Reporter:	Lukas Krejci <lkrejci>
Component:	Plugins	Assignee:	Nobody <nobody>
Status:	NEW ---	QA Contact:
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.13	CC:	hrupp
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1143050 (view as bug list)		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1143050

Description Lukas Krejci 2014-09-17 19:51:24 UTC

Description of problem:

Either database or postgres plugin leave a dangling timer thread running past the plugin container lifecycle. These threads hold on to the classloader of the code that started them and thus keeping reference to the plugin classloader of the database plugin.

Each plugin container restart (not a full agent restart) leaves 1 such thread running so over time there is a large number of classloaders accumulated and the JVM will eventually run out of permgen due to the number of loaded classes.

Version-Release number of selected component (if applicable):
4.13.0-SNAPSHOT

How reproducible:
always

Steps to Reproduce:
1. Inventory a postgres server
2. restart plugin container repeatedly

Actual results:
OOM error reporting permgen depletion

Expected results:
no errors

Additional info:

Comment 1 Lukas Krejci 2014-09-19 21:00:44 UTC

This is a bug in the Postgres JDBC driver.

The fix was fairly simple, so I went ahead and proposed a PR to the pgjdbc project: https://github.com/pgjdbc/pgjdbc/pull/188

Still, we probably should do something about the leak.

One possibility is to add a plugin lifecycle listener to the RHQ postgres plugin and on shutdown do a little bit of reflection magic to access the "cancelTimer" field on the org.postgresql.Driver class and call the cancel() method on it.

This should fix the issue even with the current version of the JDBC driver.

I've not gone ahead with that as of yet due to other work.