Bug 1143050

Summary:	[postgres] Classloader leak when postgres server inventoried
Product:	[JBoss] JBoss Operations Network	Reporter:	Lukas Krejci <lkrejci>
Component:	Plugin -- Other	Assignee:	RHQ Project Maintainer <rhq-maint>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Filip Brychta <fbrychta>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	JON 3.3.0	CC:	fbrychta, hrupp, mfoley, myarboro, rhq-maint
Target Milestone:	ER05
Target Release:	JON 3.3.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1143048	Environment:
Last Closed:	2014-12-11 14:05:00 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1143048
Bug Blocks:	1145561

Description Lukas Krejci 2014-09-17 19:53:36 UTC

+++ This bug was initially created as a clone of Bug #1143048 +++

Description of problem:

Either database or postgres plugin leave a dangling timer thread running past the plugin container lifecycle. These threads hold on to the classloader of the code that started them and thus keeping reference to the plugin classloader of the database plugin.

Each plugin container restart (not a full agent restart) leaves 1 such thread running so over time there is a large number of classloaders accumulated and the JVM will eventually run out of permgen due to the number of loaded classes.

Version-Release number of selected component (if applicable):
4.13.0-SNAPSHOT

How reproducible:
always

Steps to Reproduce:
1. Inventory a postgres server
2. restart plugin container repeatedly

Actual results:
OOM error reporting permgen depletion

Expected results:
no errors

Additional info:

Comment 1 Lukas Krejci 2014-10-08 15:27:36 UTC

master: 1cd368ebd06bde6a496b6afbc442c3a937f013fb
release/jon3.3.x: 91ecc35da63f74d2b4571c55f28b9be4849a94c4
Author: Lukas Krejci <lkrejci>
Date:   Wed Oct 8 17:25:17 2014 +0200

    [BZ 1143050] Work around the CL leak in Posgres JDBC driver.
    
    (cherry picked from commit 1cd368ebd06bde6a496b6afbc442c3a937f013fb)

Comment 2 Simeon Pinder 2014-10-21 20:24:33 UTC

Moving to ON_QA as available to test with the latest brew build:
https://brewweb.devel.redhat.com//buildinfo?buildID=394734

Comment 3 Filip Brychta 2014-10-23 11:01:59 UTC

Still visible in Version :	
3.3.0.ER05
Build Number :	
92b6d6a:2cdb528

Not sure if the leak is in postgres plugin. Heapdump provided. Please create a separate bz if necessary

Comment 4 Lukas Krejci 2014-10-23 11:16:08 UTC

Judged by the inspection of the provided heapdumps, the leak seems to be coming from the rhqserver plugin.

Could you please re-test this with the rhqserver agent plugin deleted from the JON server?

Comment 5 Lukas Krejci 2014-10-23 12:46:37 UTC

FYI, created BZ 1156035 to track the rhqserver plugin leak.

Comment 6 Filip Brychta 2014-10-23 14:35:24 UTC

You are right the issues is not visible any more without the rhqserver agent plugin.

Version :	
3.3.0.ER05
Build Number :	
92b6d6a:2cdb528