Bug 1156035 - rhqserver plugin never closes connection to RHQ server
Summary: rhqserver plugin never closes connection to RHQ server
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Operations Network
Classification: JBoss
Component: Plugin -- Other
Version: JON 3.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: CR01
: JON 3.3.1
Assignee: Michael Burman
QA Contact: Mike Foley
URL:
Whiteboard:
Depends On: 1156034
Blocks: 1145561
TreeView+ depends on / blocked
 
Reported: 2014-10-23 12:45 UTC by Lukas Krejci
Modified: 2015-02-27 19:58 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 1156034
Environment:
Last Closed: 2015-02-27 19:58:34 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
RHQ Agent permgen (12.87 KB, image/png)
2015-01-14 13:13 UTC, Stian Lund
no flags Details

Description Lukas Krejci 2014-10-23 12:45:36 UTC
+++ This bug was initially created as a clone of Bug #1156034 +++

Description of problem:
The rhqserver plugin reads stats from the RHQ server using JMX. It fails to close the JMX connection to the RHQ server when the RHQ server resource component stops.

Version-Release number of selected component (if applicable):
4.13.0-SNAPSHOT

How reproducible:
always

Steps to Reproduce:
1. Inventory RHQ server
2. restart the plugin container of your agent repeatedly

Actual results:
OOMEs after a couple of restarts (cca 20) (this is due to the connection handling threads holding on to the plugin classloaders).

Expected results:
stable heap

Additional info:

Comment 1 Stian Lund 2015-01-14 13:13:26 UTC
I am observing the same on RHQ 4.13 on the RHQ server local agent. After several restarts permgen keeps growing and number of threads as well.

Attaching screenshot from JVisualvm.

Comment 2 Stian Lund 2015-01-14 13:13:48 UTC
Created attachment 979994 [details]
RHQ Agent permgen

Comment 3 Simeon Pinder 2015-01-19 20:52:59 UTC
Moving into CR01 target milestone as missed ER01 cutoff.

Comment 5 Michael Burman 2015-01-30 10:48:34 UTC
Pushed to master, closes the connection, but does not fix the memory leak:

commit 1f15123dfda98f86f647f271e33f3173511a4530
Author: Michael Burman <miburman>
Date:   Fri Jan 30 12:45:33 2015 +0200

    [BZ 1156035] Close EmsConnection on shutdown

Comment 6 Simeon Pinder 2015-02-06 16:44:26 UTC
This is cherry-picked to release/jon3.3.x with commit: 884011bad1ada to jon.git.
Moving to MODIFIED for processing with next brew build.

Still need to spawn a new BZ and JIRA to track work from comment 4.

Comment 7 Libor Zoubek 2015-02-10 08:31:39 UTC
I've opened Bug 1191002 to track mem leak in remoting-jmx

Comment 8 Simeon Pinder 2015-02-16 04:49:38 UTC
Moving to ON_QA as available to test with latest CP build:
http://download.devel.redhat.com/brewroot/packages/org.jboss.on-jboss-on-parent/3.3.0.GA/16/maven/org/jboss/on/jon-server-patch/3.3.0.GA/jon-server-patch-3.3.0.GA.zip
*Note: jon-server-patch-3.3.0.GA.zip maps to CR01 build of jon-server-3.3.0.GA-update-01.zip.

Comment 9 Filip Brychta 2015-02-16 11:38:17 UTC
Marking as verified. Code review should be enough.


Note You need to log in before you can comment on or make changes to this bug.