Bug 534365 (RHQ-1169)

Summary:	factor purge code, provide better metrics
Product:	[Other] RHQ Project	Reporter:	John Mazzitelli <mazz>
Component:	Core Server	Assignee:	John Mazzitelli <mazz>
Status:	CLOSED NEXTRELEASE	QA Contact:	Jeff Weiss <jweiss>
Severity:	medium	Docs Contact:
Priority:	low
Version:	unspecified	CC:	dajohnso
Target Milestone:	---	Keywords:	Improvement
Target Release:	---
Hardware:	All
OS:	All
URL:	http://jira.rhq-project.org/browse/RHQ-1169
Whiteboard:
Fixed In Version:	1.2	Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	534745
Bug Blocks:

Description John Mazzitelli 2008-11-22 23:33:00 UTC

We have purge code happening in two separate places - DataPurgeJob and MeasurementCompressionManagerBean.

The purge code should not happen in the compression bean, put that work in the DataPurgeJob so all purging is done in the same place (events, alerts, traits, etc.)

Add more metrics to our measurement monitor mbean - things like "number of events purged", "number of alerts purged", etc.

Comment 1 John Mazzitelli 2008-11-23 01:51:32 UTC

svn rev2103 is the refactoring of the data purge job.  now it is responsible for purging everything (events, alerts, availabilities, calltime data, traits).  compression manager is only responsible for compressing (it no longer purges events/alerts/calltime).

Comment 2 John Mazzitelli 2008-11-23 03:28:42 UTC

some new measurements that track how many things were purged (individual events, alerts, traits, etc).

the rhq-server plugin hosted on jopr will be able to track the new purged measurements
https://jira.jboss.org/jira/browse/JOPR-26

Comment 3 John Mazzitelli 2008-12-16 21:52:53 UTC

the new metrics can be seen if you enable the jmx-console and look at the MBean: 

rhq:service=MeasurementMonitor

Comment 4 John Mazzitelli 2008-12-16 21:54:03 UTC

the new metrics have names that start with "Purged"  like PurgedEvents, Purged Alerts, etc.

Comment 5 Jeff Weiss 2009-02-09 18:42:31 UTC

Just waiting on a purge job to run @2pm today

Comment 6 Jeff Weiss 2009-02-09 19:40:06 UTC

I set the purge interval to 2 days for a couple of items on the Admin server config page.  Waited until the top of the hour, and saw the below output in the logs.  see linked JIRA for defect found.


2009-02-09 14:00:00,022 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Data Purge Job STARTING
2009-02-09 14:00:00,024 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Measurement data compression starting at Mon Feb 09 14:00:00 EST 2009
2009-02-09 14:00:01,397 INFO  [org.rhq.enterprise.server.measurement.MeasurementCompressionManagerBean] Compressed from RHQ_MEAS_DATA_NUM_R09 into 1237 rows in RHQ_MEASUREMENT_DATA_NUM_1H in (0 seconds)
2009-02-09 14:00:01,398 INFO  [org.rhq.enterprise.server.measurement.MeasurementCompressionManagerBean] Truncating table: RHQ_MEAS_DATA_NUM_R10
2009-02-09 14:00:01,729 INFO  [org.rhq.enterprise.server.measurement.MeasurementCompressionManagerBean] Compressed from RHQ_MEASUREMENT_DATA_NUM_1H into 1237 rows in RHQ_MEASUREMENT_DATA_NUM_6H in (0 seconds)
2009-02-09 14:00:01,760 INFO  [org.rhq.enterprise.server.measurement.MeasurementCompressionManagerBean] Done purging [0] rows older than 1/26/09 2:00:00 PM from RHQ_MEASUREMENT_DATA_NUM_1H in (0 seconds)
2009-02-09 14:00:01,781 INFO  [org.rhq.enterprise.server.measurement.MeasurementCompressionManagerBean] Done purging [0] rows older than 1/9/09 2:00:00 PM from RHQ_MEASUREMENT_DATA_NUM_6H in (0 seconds)
2009-02-09 14:00:01,784 INFO  [org.rhq.enterprise.server.measurement.MeasurementCompressionManagerBean] Done purging [0] rows older than 2/10/08 2:00:00 PM from RHQ_MEASUREMENT_DATA_NUM_1D in (0 seconds)
2009-02-09 14:00:01,784 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Measurement data compression completed in [1761]ms
2009-02-09 14:00:01,784 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Measurement calltime data purge starting at Mon Feb 09 14:00:01 EST 2009
2009-02-09 14:00:01,784 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Purging calltime data that is older than Mon Feb 09 13:29:01 EST 2009
2009-02-09 14:00:01,789 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Calltime purged [0] - completed in [5]ms
2009-02-09 14:00:01,789 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Event data purge starting at Mon Feb 09 14:00:01 EST 2009
2009-02-09 14:00:01,790 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Purging event data older than Mon Feb 09 13:46:01 EST 2009
2009-02-09 14:00:01,826 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Event data purged [10165] - completed in [37]ms
2009-02-09 14:00:01,826 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Alert data purge starting at Mon Feb 09 14:00:01 EST 2009
2009-02-09 14:00:01,826 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Purging alert data older than Mon Feb 09 13:58:01 EST 2009
2009-02-09 14:00:01,832 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Alert data purged [12] - completed in [6]ms
2009-02-09 14:00:01,832 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Alert definition purge starting at Mon Feb 09 14:00:01 EST 2009
2009-02-09 14:00:01,834 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Alert definitions purged [0] - completed in [2]ms
2009-02-09 14:00:01,835 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Trait data purge starting at Mon Feb 09 14:00:01 EST 2009
2009-02-09 14:00:01,835 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Purging traits that are older than Mon Feb 09 13:58:01 EST 2009
2009-02-09 14:00:01,845 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Traits data purged [4] - completed in [10]ms
2009-02-09 14:00:01,845 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Availability data purge starting at Mon Feb 09 14:00:01 EST 2009
2009-02-09 14:00:01,845 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Purging availablities that are older than Mon Feb 09 13:57:01 EST 2009
2009-02-09 14:00:01,854 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Availability data purged [4484] - completed in [8]ms
2009-02-09 14:00:01,854 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Purging OOBs older than 72h ...
2009-02-09 14:00:01,857 INFO  [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] Removed [0] old OOB entries
2009-02-09 14:00:01,858 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Purging of old OOBs completed in [4]ms
2009-02-09 14:00:01,858 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Database maintenance starting at Mon Feb 09 14:00:01 EST 2009
2009-02-09 14:00:01,858 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Performing hourly database maintenance
2009-02-09 14:00:02,642 INFO  [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] Database maintenance completed in [784]ms

Comment 7 Red Hat Bugzilla 2009-11-10 20:27:13 UTC

This bug was previously known as http://jira.rhq-project.org/browse/RHQ-1169