Created attachment 516174 [details]
Image showing the java classes that are leaking
Description of problem: RHQ Agent leaks java classes when issued command "plugins update". Unbounded increase in class instance counts when this command is issued on the agent, even when there are no updates to the plugin needed. Unbounded growth of java programs leads to exhaustion of system resources and robustness/duration/longevity issues.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Attach Jprofiler to the agent
2. on the agent, do "plugins update"
3. in jprofiler, do GC
4. in jprofiler, set a baseline
5. on agent, do "plugins update"
6. in jprofiler, do GC
7. in jprofiler, observe instance counts on java classes increased
8. do another "plugins update" ... followed by a GC
9. observe unbounded growth of java class instance counts
also ... tested:
ping .... minor leak (but may be important if this command is issued automatically frequently)
discovery -f ... minor leak
metrics ... OK
avail ... OK
Assigning to mazz for comment
IIRC this was to do with the plugin container getting reloaded, and is a known issue, with no reasonable fix.
Work around would be restarting the agent after a certain number of plugin updates (which in production systems are typically only done during patching or upgrades)
related to bug 615377
see its tenth comment https://bugzilla.redhat.com/show_bug.cgi?id=615377#c10
seems to be some minor perm gen leakage that occurs when restarting the
internals (using either "shutdown/start", "plugins update" or "pc stop/pc
start" - which all restart the PC"
Setting to medium priority as per mazz's comment