Description of problem: Either database or postgres plugin leave a dangling timer thread running past the plugin container lifecycle. These threads hold on to the classloader of the code that started them and thus keeping reference to the plugin classloader of the database plugin. Each plugin container restart (not a full agent restart) leaves 1 such thread running so over time there is a large number of classloaders accumulated and the JVM will eventually run out of permgen due to the number of loaded classes. Version-Release number of selected component (if applicable): 4.13.0-SNAPSHOT How reproducible: always Steps to Reproduce: 1. Inventory a postgres server 2. restart plugin container repeatedly Actual results: OOM error reporting permgen depletion Expected results: no errors Additional info:
This is a bug in the Postgres JDBC driver. The fix was fairly simple, so I went ahead and proposed a PR to the pgjdbc project: https://github.com/pgjdbc/pgjdbc/pull/188 Still, we probably should do something about the leak. One possibility is to add a plugin lifecycle listener to the RHQ postgres plugin and on shutdown do a little bit of reflection magic to access the "cancelTimer" field on the org.postgresql.Driver class and call the cancel() method on it. This should fix the issue even with the current version of the JDBC driver. I've not gone ahead with that as of yet due to other work.