Bug 1003600 - Memory leak in CDI / JPA integration code
Memory leak in CDI / JPA integration code
Status: CLOSED CURRENTRELEASE
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: CDI/Weld, JPA (Show other bugs)
6.1.0,6.2.0,6.1.1
Unspecified Unspecified
unspecified Severity unspecified
: ER6
: EAP 6.2.0
Assigned To: Tomaz Cerar
Marek Schmidt
Russell Dickenson
https://community.jboss.org/thread/23...
:
Depends On: 1010467
Blocks: 1033754
  Show dependency treegraph
 
Reported: 2013-09-02 09:10 EDT by Tomaz Cerar
Modified: 2013-12-15 11:21 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-12-15 11:21:57 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker WFLY-1697 Major Closed Fix JPA SerializableValidatorFactory memory leak 2017-08-17 10:04 EDT

  None (edit)
Description Tomaz Cerar 2013-09-02 09:10:40 EDT
WeldJpaInjectionServices holds hard reference to DeploymentUnit which causes DeploymentUnit and all everything resulting from it to not get cleaned up.

Same problems was fixed in upstream for WildFly 8 alpha3 as part of 
https://github.com/wildfly/wildfly/pull/4795

see for some more details.
http://lists.jboss.org/pipermail/wildfly-dev/2013-July/000466.html
Comment 2 Stuart Douglas 2013-09-02 11:21:42 EDT
I don't see how this is possible. WeldJpaInjectionServices is only held by the weld bootstrap service, which should be removed once the deployment is removed.
Comment 3 Tomaz Cerar 2013-09-02 12:21:23 EDT
I misspoke, problem is ServiceRegistry reference, which is never cleaned up during undeploy as it is server-wide resource.
Comment 4 Stuart Douglas 2013-09-02 13:18:20 EDT
I'm still not sure how the ServiceRegistry reference can be a problem. The WeldJpaInjectionServices instance will not be referenced, so it should still be GC'ed.
Comment 5 Tomaz Cerar 2013-09-03 17:35:56 EDT
Stuart, this is example of leak taken from yourkit
Paths from "GC Roots" to "Object DeploymentUnitImpl #597743"

+-------------------------------------------------------------------------------------------------+-----------------+----------------+
|                                              Name                                               |  Retained Size  |  Shallow Size  |
+-------------------------------------------------------------------------------------------------+-----------------+----------------+
|  +---org.jboss.as.server.deployment.DeploymentUnitImpl                                          |        916.016  |            24  |
|    |                                                                                            |                 |                |
|    +---deploymentUnit of org.jboss.as.weld.services.bootstrap.WeldJpaInjectionServices          |        916.032  |            16  |
|      |                                                                                          |                 |                |
|      +---value of java.util.HashMap$Entry                                                       |             48  |            24  |
|        |                                                                                        |                 |                |
|        +---[27] of java.util.HashMap$Entry[32]                                                  |            944  |           144  |
|          |                                                                                      |                 |                |
|          +---table of java.util.HashMap                                                         |          1.008  |            48  |
|            |                                                                                    |                 |                |
|            +---services of org.jboss.weld.bootstrap.api.helpers.SimpleServiceRegistry           |          1.024  |            16  |
|              |                                                                                  |                 |                |
|              +---services of org.jboss.weld.manager.BeanManagerImpl                             |      1.235.320  |           120  |
|                |                                                                                |                 |                |
|                +---beanManager of org.jboss.arquillian.testenricher.cdi.container.CDIExtension  |      1.235.656  |           336  |
|                  |                                                                              |                 |                |
|                  +---[0] of java.lang.Object[320]                                               |      1.323.720  |         1.296  |
|                    |                                                                            |                 |                |
|                    +---elementData of java.util.Vector                                          |      1.323.744  |            24  |
|                      |                                                                          |                 |                |
|                      +---classes of org.jboss.modules.ModuleClassLoader                         |      1.588.192  |            80  |
|                        |                                                                        |                 |                |
|                        +---referent of java.util.WeakHashMap$Entry                              |            888  |            40  |
|                          |                                                                      |                 |                |
|                          +---[108] of java.util.WeakHashMap$Entry[128]                          |         25.968  |           528  |
|                            |                                                                    |                 |                |
|                            +---table of java.util.WeakHashMap                                   |         26.056  |            56  |
|                              |                                                                  |                 |                |
|                              +---loaderToCache of java.lang.reflect.Proxy [Class]               |         38.048  |           344  |
+-------------------------------------------------------------------------------------------------+-----------------+----------------+

Generated by YourKit Java Profiler 12.0.3 3.9.2013 23:34:48
Comment 6 Tomaz Cerar 2013-09-03 17:39:21 EDT
better formatting: http://pastebin.test.redhat.com/161784

this is from current 6.1.x branch aka same as 6.1.1.GA
problem is that DeploymentUnit instances are saying around after undeploy
with proposed fix, there are no deployment units left after undeploy.
Comment 7 Stuart Douglas 2013-09-04 02:36:34 EDT
No, that is an Arquillian leak. The BM is being held from org.jboss.arquillian.testenricher.cdi.container.CDIExtension#beanManager.

When reading these graphs you have to read them from the bottom up, not the top down, the closest thing to the bottom that should have been GC'ed is generally the culprit. 

I'm thinking this should be closed, as it does not affect EAP itself?
Comment 8 Tomaz Cerar 2013-09-04 06:27:40 EDT
I had same impression that the problem is with static reference to BeanManager in CDIExtension
see:
https://github.com/arquillian/arquillian-core/blob/master/testenrichers/cdi/src/main/java/org/jboss/arquillian/testenricher/cdi/container/CDIExtension.java

But given that CDI extensions are deployment scoped it should be GC-ed on undeploy.
Comment 9 Stuart Douglas 2013-09-04 06:37:29 EDT
No, instances of the extension will be GC'ed, however because the bean manager reference is held in a static field it will only be cleared when the class itself is GCed. 

Assuming that Arquillian has actually been undeployed we should be looking at what is still holding onto the class loader references that prevents the Arquillian deployments class loader from being GCed.
Comment 10 Tomaz Cerar 2013-09-04 09:39:01 EDT
Issue is related also with https://hibernate.atlassian.net/browse/HHH-8363
Comment 11 Tomaz Cerar 2013-09-04 09:41:22 EDT
After discussion with Stuart we both agree that is memory leak, but my proposed fix just mask the real memory leaks.

Currently we identified possible causes in not clearing TCCL in executor service and linked hibernate issue.
Comment 14 Tomaz Cerar 2013-09-17 18:08:44 EDT
This is not complete yet, linked PR was just first fix.
for complete fix hibernate upgrade is required.
Hibernate fix was already applied to upstream and is included in 4.3.0.Beta4
for EAP it needs 4.2.x release.
We can track that separately or as part of this issue.
Comment 20 Tomaz Cerar 2013-10-07 11:06:50 EDT
I can reproduce this, it is related to buggy h2 we use for ExampleDB datasource.
I will send PR to address this same way we did in upstream
Comment 21 Tomaz Cerar 2013-10-07 11:54:16 EDT
https://github.com/jbossas/jboss-eap/pull/515
This PR changes default ExampleDS configuration to disable H2 DatabaseCloser thread creation which can hold reference to hibernate CL.

This same problem exists also in this quickstart
https://github.com/wildfly/quickstart/blob/master/kitchensink-html5-mobile/src/test/resources/arquillian-ds.xml also needs this change 
;DB_CLOSE_ON_EXIT=FALSE should be added to url.

with that change memory leak does not appear anymore.

This problem is related only to h2 in memory database and does not affect any other jdbc drivers I was testing.

If I run quickstart with modified datasource this memory leak does not appear anymore
Comment 25 Marek Schmidt 2013-10-18 11:23:37 EDT
Verified on EAP 6.2.0.ER6. 

I have noticed the instances are still laying around in memory for quite a while after the deployment ends awaiting finalization, but I expect this to be just Java being lazy about finalization, as there are no GC roots to be found for them now...

Note You need to log in before you can comment on or make changes to this bug.