Bug 1079303
Summary: | Excessive database size when using bundles causing UI to fail | ||
---|---|---|---|
Product: | [JBoss] JBoss Operations Network | Reporter: | Shaun Appleton <sappleto> |
Component: | Database | Assignee: | Thomas Segismont <tsegismo> |
Status: | CLOSED CURRENTRELEASE | QA Contact: | Armine Hovsepyan <ahovsepy> |
Severity: | unspecified | Docs Contact: | |
Priority: | unspecified | ||
Version: | JON 3.1.2 | CC: | ahovsepy, bkramer, loleary, mfoley, tsegismo |
Target Milestone: | ER03 | ||
Target Release: | JON 3.3.0 | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: |
The bundle view was retrieving all audit messages for all versions of the bundle when the bundle details were requested. Additionally the relationship between the data tables that contained the bundle deployment information and the bundle's audit messages were missing an index, which resulted in the bundle view page taking a very long time to display. In some cases, it would not even load. If the user attempted to delete the old bundle information, the deletion would very often fail due to a transaction timeout. A relationship index is now added to the bundle deployment and audit message data tables. Audit messages are only loaded for the bundle version that is currently being accessed. Additionally, when a bundle version delete operation is performed, its corresponding audit messages are deleted asynchronously. The bundle view now loads much faster as it is not retrieving unnecessary data, and the delete operation no longer requires the bundle's audit messages to be deleted at the same time as the bundle version.
|
Story Points: | --- |
Clone Of: | Environment: | ||
Last Closed: | 2014-12-11 13:59:58 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1139186 | ||
Bug Blocks: |
Description
Shaun Appleton
2014-03-21 11:49:19 UTC
To reproduce this: 1. Create a bundle that contains lots of audit messages in the recipe file (30+); 2. Deploy above bundle 1000+ times; This will create a huge number of rows in the RHQ_BUNDLE_RES_DEP_HIST table and any attempt to delete bundle and it's content from JBoss ON UI will fail because of the transactions timeout. Additionally, in environments where RHQ_BUNDLE_RES_DEP_HIST table contains huge number of rows, the following performance issues can be observed: ** when deploying a bundle using JBoss ON CLI the slowness occurs (the waiting time is up to 2-3 min due to huge database tables); ** JBoss ON server does not start properly and will take up 100% CPU usage. Fixed in master (approved by Jay) commit 2d48aa446416af96ee5285ab16b3888057177ded Author: Thomas Segismont <tsegismo> Date: Fri Sep 5 11:59:17 2014 +0200 Do not load histories when showing the deployments view. Fetch histories aync when unfolding a resource deployment. Added index on resource deployment id in audit table ->Fast fetch histories aync when unfolding a resource deployment in the UI Dropped foreign key on resource deployment id in audit table ->This is to prepare the async deletion of orphan audit messages when deleting a bundle deployment (or a bundle version, or even the bundle itself) ->Documented in db-upgrade.xml and content-schema.xml why we are breaking the rule (dropping an FK) Added GUI messages when deleting version/destination/deployment/bundle so that the user gets correctly informed of what is going on No longer cascade remove audit message when deleting resource deployments Purge orphaned audit messages in DataPurgeJob No direct purge of audit messages during async resource deletion job Also, increased the default batch size property, as testing showed 1.5x higher performance (tested on 1 million and on 5 million rows). A community user with a large RHQ deployment reported better results with higher batch size as well. getBundleResourceDeploymentHistories returns a copy of resourceDeployment.getBundleResourceDeploymentHistories(): 1. It's safer (no direct entity collection manipulation) 2. GWT Serializer does not accept Hibernate's persistent bags Cherry-picked over to release/jon3.3.x commit b94cdff11fe8bb92687c5ea4bab4655d29df0ee8 Author: Thomas Segismont <tsegismo> Date: Fri Sep 12 16:30:02 2014 +0200 Do not load histories when showing the deployments view. Fetch histories aync when unfolding a resource deployment. Added index on resource deployment id in audit table ->Fast fetch histories aync when unfolding a resource deployment in the UI Dropped foreign key on resource deployment id in audit table ->This is to prepare the async deletion of orphan audit messages when deleting a bundle deployment (or a bundle version, or even the bundle itself) ->Documented in db-upgrade.xml and content-schema.xml why we are breaking the rule (dropping an FK) Added GUI messages when deleting version/destination/deployment/bundle so that the user gets correctly informed of what is going on No longer cascade remove audit message when deleting resource deployments Purge orphaned audit messages in DataPurgeJob No direct purge of audit messages during async resource deletion job Also, increased the default batch size property, as testing showed 1.5x higher performance (tested on 1 million and on 5 million rows). A community user with a large RHQ deployment reported better results with higher batch size as well. getBundleResourceDeploymentHistories returns a copy of resourceDeployment.getBundleResourceDeploymentHistories(): 1. It's safer (no direct entity collection manipulation) 2. GWT Serializer does not accept Hibernate's persistent bags (cherry picked from commit 2d48aa446416af96ee5285ab16b3888057177ded) Signed-off-by: Thomas Segismont <tsegismo> Conflicts: modules/core/dbutils/pom.xml Moving to ON_QA as available for test with the following brew build: https://brewweb.devel.redhat.com//buildinfo?buildID=385149 1. Viewing bundle with >500 deployments having >800 audit messages, which is 900000 rows in rhq_bundle_res_dep_hist -- takes ~1sec 2. Deleting bundle described above works without timeout, takes 1-3secs 3. re-creating bundle, deploying >20 times to the same destination takes 30secs-1min for each deployment 4. deleting bundle does not remove audit messages 5. cpu used during the restart is ~190% then it goes down to 1.0%-10.0% Purge (delete) data using smaller batches will be tested under #1139186 closing this issue as verified |