Bug 1003987 - When doing remote EJB calls, the marshalling code creates a very large number of sun.reflect.GeneratedSerializedConstructor objects, taking the vast majority of the heap space, causing memory consumption issues.
When doing remote EJB calls, the marshalling code creates a very large number...
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Remoting (Show other bugs)
All All
unspecified Severity urgent
: ER1
: EAP 6.2.0
Assigned To: David M. Lloyd
Jitka Kozana
Russell Dickenson
Depends On:
Blocks: 1003990 1010377 1012664 1120777
  Show dependency treegraph
Reported: 2013-09-03 12:26 EDT by Andrig T Miller
Modified: 2014-07-17 12:03 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-12-15 11:20:43 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker JBMAR-153 Major Resolved Memory leak in reflection data 2017-03-20 21:51 EDT

  None (edit)
Description Andrig T Miller 2013-09-03 12:26:31 EDT
Description of problem:

During our profiling of SPECjEnterprise2010, we discovered a very large number of sun.reflect.GeneratedSerializedConstructor* objects being creating, taking the vast majority of the heap space, and they are end up in the old generation, causing large garbage collection spikes.  This causes scalability problems with EAP.

Version-Release number of selected component (if applicable):

JBoss Marshalling 1.3.x

How reproducible:

Always, with any EJB remote calls.

Steps to Reproduce:
1.Run SPECjEnterprise2010 at tx1400 rate.

Actual results:

Over 2.5 GB's of memory in the old generation will be seen (JProfiler) just for two of these constructor objects, and more than 3GB's of memory consumption when you count all constructor object references.

Expected results:

Should be very few instances, with a very small memory footprint.

Additional info:

The 1.4.0 version that David Lloyd has tagged as Final today, contains the fix for this problem.

The EJB client library will have to be rebuilt along with JBoss Marshalling in order for this to work properly.
Comment 1 Andrig T Miller 2013-09-03 12:30:30 EDT
Also, I forgot to add that this issue is in JIRA as:

Comment 2 Andrig T Miller 2013-09-03 12:35:36 EDT
Changed the component to remoting, since JBoss Marshalling is a part of Remoting.
Comment 3 David M. Lloyd 2013-09-03 13:00:38 EDT
Initialized flags.
Comment 4 David M. Lloyd 2013-09-03 15:21:32 EDT
Please be sure to also ack https://bugzilla.redhat.com/show_bug.cgi?id=1003990 if you ack this issue.
Comment 7 Ladislav Thon 2013-09-24 06:00:43 EDT
Verified with EAP 6.2.0.ER1.
Comment 8 JBoss JIRA Server 2013-09-26 14:41:24 EDT
Brad Maxwell <bmaxwell@redhat.com> made a comment on jira JBMAR-153

Background info: Originally JBoss Marshalling used a weak map so that it would not leak memory when a deployment was undeployed.  This had a side effect, if a weak reference was cleared, JBoss Marshalling would generate a new constructor, except the JDK still kept the reference to the old one. So they would pile up over time, especially if you GC a lot in between invocations for whatever reason. The fix is to never regenerate them, by replacing the weak map with a strong map.  Since a strong map is used the API had to be changed so that on undeploy the map could be released and avoid a memory leak, which the JBoss EAP code calls when an application is undeployed.

Note You need to log in before you can comment on or make changes to this bug.