This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours
Bug 702390 - Large number of BundleDeployments for a single Bundle causes fetch serialization issues
Large number of BundleDeployments for a single Bundle causes fetch serializat...
Status: CLOSED CURRENTRELEASE
Product: RHQ Project
Classification: Other
Component: Core Server (Show other bugs)
3.0.0
All All
medium Severity high (vote)
: ---
: ---
Assigned To: Jay Shaughnessy
Mike Foley
:
Depends On:
Blocks: jon3 702748
  Show dependency treegraph
 
Reported: 2011-05-05 10:32 EDT by Jay Shaughnessy
Modified: 2013-09-02 03:20 EDT (History)
4 users (show)

See Also:
Fixed In Version: 4.0.1
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-02 03:20:36 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Jay Shaughnessy 2011-05-05 10:32:45 EDT
Bundles with a lot of deployments, roughly >=50, will cause the HibernateDetachUtility to throw exceptions like:

2011-05-02 12:54:45,534 WARN  [org.rhq.enterprise.server.util.HibernateDetachUtility] Getting different object hierarchies back from calls: org.rhq.core.domain.bundle.BundleDeployment

Although these are warnings, and the detach utility "scrubbing" completes, subsequent serialization of the data may fail with errors like:

Caused by: org.hibernate.LazyInitializationException: could not initialize proxy - no Session

This can then cause issues in the remote client, like CoreGUI, which can have unpredictable results.  CoreGUI has been reported to go into a "loop" when tying to render the deployments for the bundle.
Comment 1 Jay Shaughnessy 2011-05-05 10:40:52 EDT
For a Bundle there is a "Live" BundleDeployment and it replaces the previous live deployment.  The exception being the first deployment which does not replace anything.

The BundleDeployment entity defines a OneToOne relationship to itself to define the 'replacedBundleDeployment' field.  This is eager fetched, meaning that when fetching the live deployment hibernate has no choice but to also fetch the replacedBundleDeployment.  This then happens recursively, grabbing the entire history of BundleDeployments.

The HibernateDetachUtility has a max depth to prevent against unexpected recursion issues, and this is what is getting violated (the WARNING in the description text).

This can not be lazy fetched (we already mistakenly tried to do this, resulting in the problem) because the association is optional.  Hibernate must eager fetch optional XxxToOne associations because it must check for null. 

[Note - we should go through our domain model and look for this mistake in other places]

So, the proposed solution is to remove the entity association and treat the field as an id Integer field, thus preventing any [recursive] entity fetch.
Comment 2 Jay Shaughnessy 2011-05-05 14:01:23 EDT
commit 976c5fc6ed0e7e5354ea6a890caafa58b3f1c7bc
Author: Jay Shaughnessy <jshaughn@redhat.com>
Date:   Thu May 5 13:45:09 2011 -0400

Remove self-referring entity association for BundleDeployment.replacedBundleDeployment. This prevents eager loading of the entire deployment chain when fetching a BundleDeployment, and in turn avoids issues with Hibernate class scrubbing when remote fetching deployments.

Instead of using the entity as the field member, just use the ID.


Test Notes:
Please test this by exercising bundle create/deploy/revert. As well as delete of version and destination with deployments.

Finally, with at least 5 deployments for a single destination tune down the following setting (see bug 602109) :

rhq.server.hibernate-detach-utility.depth-allowed=5

Then ensure that you can still successfully view the depoyment list for the destination, in the gui.
Comment 3 Charles Crouch 2011-05-09 16:05:34 EDT
Adding for triage for RHQ 4.0.1 release
Comment 4 Jay Shaughnessy 2011-05-10 18:04:57 EDT
Cherry Picked the following three commits from master to release-4.0.0 to support this:

4e75aadafeb841c6e4c8030265197ec029efb28a
2dcc812cb2e84ccff01dc62824958baded28eec6
79fe9be4b6ef8adf0c376221438afacab1c2014c
Comment 5 Sunil Kondkar 2011-05-20 08:24:43 EDT
Verified on build#38 (Version: 4.0.1-SNAPSHOT Build Number: a104cdf)

Verified bundle creation, deploy and revert. Deleted versions and destinations with deployments.

Deployed a bundle 5 times at a single destination and had an entry in rhq-server.properties.

I was able to view the deployment list for the destination, in the gui.

Marking as verified.
Comment 6 Heiko W. Rupp 2013-09-02 03:20:36 EDT
Bulk closing of issues that were VERIFIED, had no target release and where the status changed more than a year ago.

Note You need to log in before you can comment on or make changes to this bug.