Bug 472983 - Mint database auto-purge feature
Mint database auto-purge feature
Status: CLOSED ERRATA
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: cumin (Show other bugs)
1.1
All Linux
urgent Severity medium
: 1.1
: ---
Assigned To: Nuno Santos
Frantisek Reznicek
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-11-25 17:12 EST by Ted Ross
Modified: 2015-11-15 19:06 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-02-04 10:35:45 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ted Ross 2008-11-25 17:12:04 EST
Mint should automatically purge the following database records once their age has exceeded 24 hours:

- Statistic update records
- Job records (statistic and property)

The time (defaulting to 24 hours) should be configurable via command-line or config file.
Comment 1 Nuno Santos 2008-12-01 16:00:56 EST
Added the database cleanup/expiration feature to mint at svn rev 2900-2901. 

The default behavior is to perform the cleanup every 10 minutes, deleting any stats records (plus Job records) that are older than 24 hours.
Comment 3 Justin Ross 2008-12-10 09:34:30 EST
To test this, I recommend using psql to modify the update timestamp of, say, a queue object.  It should be something like this untested snippet:

  update queue set qmf_update_time = now() - interval '25 hours' where id = $somequeueid;

Then, start up cumin and check that (a) the log indicates something got deleted and (b) you can no longer find $somequeueid in the database.
Comment 4 Justin Ross 2008-12-10 09:35:24 EST
BTW, the psql command for cumin is this: psql -d cumin -U cumin -h localhost
Comment 5 Jan Sarenik 2008-12-11 08:34:25 EST
First I would like to thank Justin for the command examples. They helped
me a lot to get kickstarted.

But so far my results are that no records get deleted - I am testing it on
cumin-0.1.2968-1.el5.i386

Please look at the transcript and let me know in case I am doing it the
wrong way.

--------------------------------------------------------------
# rpm -q cumin
cumin-0.1.2968-1.el5

### Cumin and Mint are running:
# ps aux | grep cumin
root     27096  0.1  0.8 141408 17136 pts/5    Sl   11:47   0:15 /usr/bin/python /usr/bin/cumin
root     27101  0.1  0.5  50532 10712 pts/5    Sl   11:47   0:13 /usr/bin/python /usr/bin/mint-server --data postgresql://cumin@localhost/cumin --log-file /var/lib/cumin/log/cumin.log --log-level warn --expire-frequency 600 --expire-threshold 86400
postgres 27102  0.0  0.2  22300  4892 ?        S    11:47   0:00 postgres: cumin cumin 127.0.0.1(33060) idle       
postgres 27105  0.0  0.1  21344  2232 ?        S    11:47   0:00 postgres: cumin cumin 127.0.0.1(33061) idle       
postgres 27108  0.0  0.1  21604  3344 ?        S    11:47   0:00 postgres: cumin cumin 127.0.0.1(33062) idle       
postgres 27119  0.0  0.1  21344  2228 ?        S    11:47   0:00 postgres: cumin cumin 127.0.0.1(33064) idle       
postgres 27120  0.0  0.2  22300  4892 ?        S    11:47   0:00 postgres: cumin cumin 127.0.0.1(33065) idle       
postgres 27121  0.0  0.1  21604  3324 ?        S    11:47   0:00 postgres: cumin cumin 127.0.0.1(33066) idle       
root     27159  0.0  0.0   4024   716 pts/5    S+   14:17   0:00 grep cumin


# psql -d cumin -U cumin -h localhost
cumin=# select id from queue;
 id 
----
  2
  1
(2 rows)

cumin=# update queue set qmf_update_time = now() - interval '25 hours' where id =1;
UPDATE 1
cumin=# select id,auto_delete,qmf_update_time from queue;
 id | auto_delete |      qmf_update_time       
----+-------------+----------------------------
  2 | t           | 2008-12-11 14:32:09.4123
  1 | t           | 2008-12-10 13:30:07.340433
(2 rows)
Comment 6 Nuno Santos 2008-12-11 17:31:23 EST
The expiration is only for statistics (the only exception is jobs), so queue will not work. You will need to use one of the *_stats tables, for instance queue_stats.

So do 

update queue_stats set qmf_update_time = now() - interval '25 hours' where id=XXX;

(where XXX is an existing id)

and then run cumin; that record should be deleted when the expiration thread runs.
Comment 7 Justin Ross 2008-12-11 17:57:35 EST
Sorry, Jan.  My instructions misled you.
Comment 8 Jan Sarenik 2008-12-12 04:52:42 EST
To Justin: No problem.

To Nuno: thanks for hint.

I have verified it really works as expected.
For the sake of this test I have run mint from command-line:
mint-server --data postgresql://cumin@localhost/cumin --log-file log --log-level debug --expire-frequency 20 --expire-threshold 1

After that, I have even checked with 'psql' and the records were deleted.

Here is the mint-server log (the important lines contain 'records expired'):
--------------------------------------------------------------
2008-12-12 10:44:34,294 DEBUG Expiring database records older than 1 seconds, every 20 seconds
2008-12-12 10:44:34,298 INFO Polling for registration changes every 5 seconds
2008-12-12 10:44:34,309 INFO Adding qmf broker at amqp://127.0.0.1:5672
2008-12-12 10:44:34,347 INFO Adding qmf broker at amqp://north-15.lab.bos.redhat.com:5672
2008-12-12 10:45:21,279 DEBUG 4098 records expired
2008-12-12 10:45:21,296 DEBUG SysimageStats(1) created
2008-12-12 10:45:21,311 DEBUG ExchangeStats(1) created
2008-12-12 10:45:21,315 DEBUG ExchangeStats(2) created
2008-12-12 10:45:21,321 DEBUG StoreStats(1) created
2008-12-12 10:45:21,325 DEBUG ExchangeStats(3) created
2008-12-12 10:45:21,330 DEBUG ExchangeStats(4) created
... snip ...
Comment 9 Jan Sarenik 2008-12-12 04:55:25 EST
One more proof (while I have it still on the console):

-------------------------------------------------------
cumin=# select id,qmf_update_time from queue_stats;
  id  |   qmf_update_time
------+---------------------
 6634 | 2008-12-12 10:46:57
 6635 | 2008-12-12 10:46:57
 6636 | 2008-12-12 10:46:57
 6637 | 2008-12-12 10:46:57
 6638 | 2008-12-12 10:46:57
 6639 | 2008-12-12 10:46:57
(6 rows)
-------------------------------------------------------
Comment 11 errata-xmlrpc 2009-02-04 10:35:45 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-0035.html

Note You need to log in before you can comment on or make changes to this bug.