Bug 727270
| Summary: | RHQ Agent leaks java classes on scheduled operations | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Other] RHQ Project | Reporter: | Mike Foley <mfoley> | ||||||
| Component: | Agent | Assignee: | John Mazzitelli <mazz> | ||||||
| Status: | CLOSED NOTABUG | QA Contact: | Mike Foley <mfoley> | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | 4.1 | CC: | hrupp | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | All | ||||||||
| OS: | All | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2011-10-31 20:48:03 UTC | Type: | --- | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 625146 | ||||||||
| Attachments: |
|
||||||||
Assigning to mazz, to investigate further. This seems worse than 727245 since scheduled operations should be a fairly common thing. Severity is going to depend on rate of leak, and whether its related to the operation being called for example, e.g. are some operations really bad. I'm not convinced this is anything. Would need someone to perform true, long term testing to see if memory actually does bloat over time. (BTW: I assume this BZ is reporting bloating PERMGEN memory). Read bug 615377 - I spent a good week on tracking down and fixing as many memory leaks as I could - some could not be fixed that I could see. Note that JDK7 has introduced a ClassLoader.close() API that could help us. As it stands, there is a design flaw in the JRE6's ClassLoader impl that means we may never correct all memory leaks until we go to JDK7 Also, see if these JDK settings help: https://bugzilla.redhat.com/show_bug.cgi?id=615377#c9 -XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled Created attachment 531037 [details]
JProfiler usage after 351 operations
I'm not seeing any leak. I ran a simple operation ("Get Current Date/Time") on the RHQ Agent resource. I scheduled it to run every 4 seconds. I ended up running it 351 times is rapid succession. After waiting for GC to run, and taking snapshots before and after the test, I do not see any memory leakage. In fact, I see the memory with the typical "sawtooth" usage behavior - heap usage goes up, then GC kicks in and frees the used memory, back to roughly the starting point of the 20MB mark. At the end, I was still using 20MB as per the attached image from jprofiler.
There is no leak detected. After examining the heap memory usage after rapidly executing an operation every 4 seconds for 25 minutes straight, memory consumption looks normal. Nothing appears to leak unbounded. Garbage collection reclaims heap periodically back down to its original level. |
Created attachment 516185 [details] image showing the java classes that leak as operations execute Description of problem: RHQ Agent leaks java classes on scheduled operations. Unbounded increase in class instance counts as scheduled operations execute on the RHQ Agent resource. Can lead to robustness, duration, longevity issues. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. jprofiler attached to agent 2. create a scheduled operation on the RHQ Agent, to execute an availability scan every 60 seconds for an unlimited amount of time 3. observe the scheduled operation execute a few times 4. in jprofiler ... perform GC ...mark baseline 5. observe instance counts increase as operation executes ... even after performing GC ... in an unbounded manner. Actual results: unbounded increase in class instance counts Expected results: no unbounded increases Additional info: