Bug 1156035

Summary:

rhqserver plugin never closes connection to RHQ server

Product:

[JBoss] JBoss Operations Network

Reporter:

Lukas Krejci <lkrejci>

Component:

Plugin -- Other

Assignee:

Michael Burman <miburman>

Status:

CLOSED CURRENTRELEASE

QA Contact:

Mike Foley <mfoley>

Severity:

high

Docs Contact:

Priority:

unspecified

Version:

JON 3.3.0

CC:

fbrychta, hrupp, lzoubek, mfoley, miburman, myarboro, rhq-maint, spinder, stianlund+bugzilla

Target Milestone:

CR01

Keywords:

Triaged

Target Release:

JON 3.3.1

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

1156034

Environment:

Last Closed:

2015-02-27 19:58:34 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

1156034

Bug Blocks:

1145561

Attachments:

Description	Flags
RHQ Agent permgen	none

Description Lukas Krejci 2014-10-23 12:45:36 UTC

+++ This bug was initially created as a clone of Bug #1156034 +++

Description of problem:
The rhqserver plugin reads stats from the RHQ server using JMX. It fails to close the JMX connection to the RHQ server when the RHQ server resource component stops.

Version-Release number of selected component (if applicable):
4.13.0-SNAPSHOT

How reproducible:
always

Steps to Reproduce:
1. Inventory RHQ server
2. restart the plugin container of your agent repeatedly

Actual results:
OOMEs after a couple of restarts (cca 20) (this is due to the connection handling threads holding on to the plugin classloaders).

Expected results:
stable heap

Additional info:

Comment 1 Stian Lund 2015-01-14 13:13:26 UTC

I am observing the same on RHQ 4.13 on the RHQ server local agent. After several restarts permgen keeps growing and number of threads as well.

Attaching screenshot from JVisualvm.

Comment 2 Stian Lund 2015-01-14 13:13:48 UTC

Created attachment 979994 [details]
RHQ Agent permgen

Comment 3 Simeon Pinder 2015-01-19 20:52:59 UTC

Moving into CR01 target milestone as missed ER01 cutoff.

Comment 5 Michael Burman 2015-01-30 10:48:34 UTC

Pushed to master, closes the connection, but does not fix the memory leak:

commit 1f15123dfda98f86f647f271e33f3173511a4530
Author: Michael Burman <miburman>
Date:   Fri Jan 30 12:45:33 2015 +0200

    [BZ 1156035] Close EmsConnection on shutdown

Comment 6 Simeon Pinder 2015-02-06 16:44:26 UTC

This is cherry-picked to release/jon3.3.x with commit: 884011bad1ada to jon.git.
Moving to MODIFIED for processing with next brew build.

Still need to spawn a new BZ and JIRA to track work from comment 4.

Comment 7 Libor Zoubek 2015-02-10 08:31:39 UTC

I've opened Bug 1191002 to track mem leak in remoting-jmx

Comment 8 Simeon Pinder 2015-02-16 04:49:38 UTC

Moving to ON_QA as available to test with latest CP build:
http://download.devel.redhat.com/brewroot/packages/org.jboss.on-jboss-on-parent/3.3.0.GA/16/maven/org/jboss/on/jon-server-patch/3.3.0.GA/jon-server-patch-3.3.0.GA.zip
*Note: jon-server-patch-3.3.0.GA.zip maps to CR01 build of jon-server-3.3.0.GA-update-01.zip.

Comment 9 Filip Brychta 2015-02-16 11:38:17 UTC

Marking as verified. Code review should be enough.