Bug 778837 (SOA-1283) - Memory leak in jbr-provider with http protocol
Summary: Memory leak in jbr-provider with http protocol
Keywords:
Status: CLOSED NEXTRELEASE
Alias: SOA-1283
Product: JBoss Enterprise SOA Platform 4
Classification: JBoss
Component: JBossESB
Version: 4.3 GA,4.3 CP01
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.3 CP02
Assignee: Kevin Conner
QA Contact:
URL: http://jira.jboss.org/jira/browse/SOA...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-04-09 08:21 UTC by Martin Vecera
Modified: 2009-08-18 11:26 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
RHEL5, Oracle10g (IMHO DB is irrelevant)
Last Closed: 2009-08-18 11:26:29 UTC
Type: Bug


Attachments (Terms of Use)
server.log.bz2 (140.34 KB, application/x-bzip)
2009-04-09 08:22 UTC, Martin Vecera
no flags Details
jprofiler_reports.tar.bz2 (153.59 KB, application/x-bzip)
2009-04-09 08:24 UTC, Martin Vecera
no flags Details
milyn-smooks-core-1.0.1.jar (506.01 KB, application/x-java-archive)
2009-04-15 06:03 UTC, Kevin Conner
no flags Details
jboss-smooks-1.0.aop (1.97 KB, application/octet-stream)
2009-04-15 06:03 UTC, Kevin Conner
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 778806 0 high CLOSED JBossRemotingGatewayListener creating and instance of WSSecurityInfoExtractor on every request/invocation 2021-02-22 00:41:40 UTC
Red Hat Issue Tracker SOA-1283 0 None None None Never

Internal Links: 778806

Description Martin Vecera 2009-04-09 08:21:43 UTC
Date of First Response: 2009-04-14 06:26:25
Workaround Description: Remove the current smooks.esb/milyn-smooks-core-1.0.1.jar, replacing it by deploying both the aspectised version from this issue and the jboss-smooks-1.0.aop into smooks.esb.
project_key: SOA

For each incoming message some classes are instantiated (e.g. [org.milyn.javabean.BeanPopulator] Bean Populator created for [userNameToken:org.jboss.soa.esb.services.security.auth.ws.UsernameToken]), which are never released nor collected by GC.

To reproduce the issue, you can use webservice_producer QS:
ant deploy
ant saygoodbye_over_http

Attached is server.log and several profiling reports created by jProfiler.

This issue is crashing one of our performance tests because the server runs out of memory.

Comment 1 Martin Vecera 2009-04-09 08:22:16 UTC
Attachment: Added: server.log.bz2


Comment 2 Martin Vecera 2009-04-09 08:24:56 UTC
Attachment: Added: jprofiler_reports.tar.bz2


Comment 3 Kevin Conner 2009-04-14 10:26:25 UTC
A quick look at the profiler output would suggest that the following may be the cause of the leak.

Each request results in a new WSSecurityInfoExtractor being instantiated which, in turn, creates a new Smooks instance.

Each Smooks instance results in a new StandaloneApplicationContext which in turn instantiates SmooksResourceConfigurationStore.

One leak, from a cursory inspection, would appear to be due to SmooksResourceConfigurationStore.  During the construction of this class it will register a shutdown hook but this is never cleaned up, resulting in a hard reference to the SmooksResourceConfigurationStore.

Other instances of this issue would appear to be present in AbstractStreamSplitter, SmooksHandler, SmooksMapper and SoapUIClientService.

A quick fix would be to disable the shutdownHook for every instance of Smooks being created but the whole creation of the Smooks instance would appear to be heavyweight and it would be a better solution to handle this in a less resource intensive manner.

The modification would be something like the following

smooks.getApplicationContext().getStore().removeShutdownHook();

Kev


Comment 4 Kevin Conner 2009-04-14 10:53:55 UTC
Link: Added: This issue depends JBESB-2520


Comment 5 Tom Fennelly 2009-04-14 14:45:18 UTC
Ah OK... this is actually a duplicate of SOA-1256, with the addition that Smooks.close() should be called as part of the cleanup of the ESB resources. close() will also remove that shutdown hook, as well as executing the cleanup of whatever resources are created/used.

Comment 6 Kevin Conner 2009-04-14 15:01:19 UTC
It is not a duplicate but it is related.  This is more general and affects other areas than the one mentioned in that issue.

Comment 7 Mark Little 2009-04-14 15:40:58 UTC
Link: Added: This issue is related to SOA-1256


Comment 8 Kevin Conner 2009-04-15 06:03:28 UTC
Attachment: Added: milyn-smooks-core-1.0.1.jar


Comment 9 Kevin Conner 2009-04-15 06:03:45 UTC
Attachment: Added: jboss-smooks-1.0.aop


Comment 10 Kevin Conner 2009-04-15 06:06:37 UTC
Hiya Martin.

I have attached two jars which I believe will disable the smooks' shutdown hook.  Could you please give these a try and see if this is the only source of a leak or whether there are also other issues?

These should be placed within smooks.esb.

Kev


Comment 11 Martin Vecera 2009-04-15 12:32:14 UTC
Hello, the workaround has been successfully verified, although there seems to be slight (1-2%) performance hit.

Comment 12 Kevin Conner 2009-04-15 13:48:57 UTC
Workaround Description: Added: Remove the current smooks.esb/milyn-smooks-core-1.0.1.jar, replacing it by deploying both the aspectised version from this issue and the jboss-smooks-1.0.aop into smooks.esb.


Comment 13 Dana Mison 2009-08-11 07:28:23 UTC
added to 4.3.CP02 release notes as resolved:

JBESB-2520 - A number of services were leaking memory due the shutdown hook registered by org.milyn.cdr.SmooksResourceConfigurationStore() never being removed.  The shutdown hook registration has been removed.

Comment 14 Dana Mison 2009-08-11 07:30:51 UTC
doh! That should read:

JBESB-2520 - A memory leak affecting Smooks has been fixed.  This was caused by the shutdown hook registered by org.milyn.cdr.SmooksResourceConfigurationStore() never being removed. The shutdown hook has been removed.

Comment 15 Martin Vecera 2009-08-18 11:26:28 UTC
Verified with 4.3 C02 CR2


Note You need to log in before you can comment on or make changes to this bug.