Red Hat Bugzilla – Bug 727270
RHQ Agent leaks java classes on scheduled operations
Last modified: 2011-10-31 16:48:03 EDT
Created attachment 516185 [details]
image showing the java classes that leak as operations execute
Description of problem: RHQ Agent leaks java classes on scheduled operations. Unbounded increase in class instance counts as scheduled operations execute on the RHQ Agent resource. Can lead to robustness, duration, longevity issues.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. jprofiler attached to agent
2. create a scheduled operation on the RHQ Agent, to execute an availability scan every 60 seconds for an unlimited amount of time
3. observe the scheduled operation execute a few times
4. in jprofiler ... perform GC ...mark baseline
5. observe instance counts increase as operation executes ... even after performing GC ... in an unbounded manner.
unbounded increase in class instance counts
no unbounded increases
Assigning to mazz, to investigate further. This seems worse than 727245 since scheduled operations should be a fairly common thing.
Severity is going to depend on rate of leak, and whether its related to the operation being called for example, e.g. are some operations really bad.
I'm not convinced this is anything. Would need someone to perform true, long term testing to see if memory actually does bloat over time. (BTW: I assume this BZ is reporting bloating PERMGEN memory).
Read bug 615377 - I spent a good week on tracking down and fixing as many memory leaks as I could - some could not be fixed that I could see. Note that JDK7 has introduced a ClassLoader.close() API that could help us. As it stands, there is a design flaw in the JRE6's ClassLoader impl that means we may never correct all memory leaks until we go to JDK7
Also, see if these JDK settings help:
Created attachment 531037 [details]
JProfiler usage after 351 operations
I'm not seeing any leak. I ran a simple operation ("Get Current Date/Time") on the RHQ Agent resource. I scheduled it to run every 4 seconds. I ended up running it 351 times is rapid succession. After waiting for GC to run, and taking snapshots before and after the test, I do not see any memory leakage. In fact, I see the memory with the typical "sawtooth" usage behavior - heap usage goes up, then GC kicks in and frees the used memory, back to roughly the starting point of the 20MB mark. At the end, I was still using 20MB as per the attached image from jprofiler.
There is no leak detected. After examining the heap memory usage after rapidly executing an operation every 4 seconds for 25 minutes straight, memory consumption looks normal. Nothing appears to leak unbounded. Garbage collection reclaims heap periodically back down to its original level.