Bug 1570147

Summary: panko events are kept in DB forever, panko-expirer utility inclusion
Product: Red Hat OpenStack Reporter: Andrew Ludwar <aludwar>
Component: puppet-pankoAssignee: Pradeep Kilambi <pkilambi>
Status: CLOSED ERRATA QA Contact: Sasha Smolyak <ssmolyak>
Severity: high Docs Contact:
Priority: high    
Version: 12.0 (Pike)CC: akaris, jamsmith, jdanjou, pkilambi, slinaber, ssmolyak
Target Milestone: z3Keywords: Triaged, ZStream
Target Release: 12.0 (Pike)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: puppet-panko-11.5.0-2.el7ost puppet-tripleo-7.4.12-7.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-08-20 12:59:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Andrew Ludwar 2018-04-20 17:58:20 UTC
Description of problem:

Customer has a large panko database size and is affecting performance. The events don't seem to have a cleanup or expiry process.


Version-Release number of selected component (if applicable):

OSP12

How reproducible:

Every time.

Steps to Reproduce:
1. Deploy OSP12
2. Observe panko events count always increase:

MariaDB [panko]> use panko;
Database changed
MariaDB [panko]> select count(*) from event;
+----------+
| count(*) |
+----------+
|  1736043 |
+----------+
1 row in set (0.37 sec)

3. Observe MySQL database growing in size.


Actual results:

Events and DB size grows uncontrolled.

Expected results:

Events and DB size can be managed by a utility.

Additional info:

A retention utility is needed for panko.

There's been upstream activity to address this:

https://review.openstack.org/#/c/539579/
https://bugs.launchpad.net/tripleo/+bug/1746514
https://review.openstack.org/#/q/topic:bug/1746514+(status:open+OR+status:merged)

May we please include this in a future maintenance release of OSP12? Creating a BZ to track this activity. I didn't see this code in the latest shipped puppet-panko.

Comment 7 Sasha Smolyak 2018-08-01 12:46:23 UTC
puppet-tripleo code didn't make it to the puddle, so the code is there, but the crone isn't configured.
Running
> sudo docker exec -ti panko_api crontab -u panko -l 
on controller gives " no crontab for panko"

Comment 11 Sasha Smolyak 2018-08-08 10:17:52 UTC
The fix is not in the latest puddle

Comment 15 Sasha Smolyak 2018-08-13 10:50:48 UTC
The latest puddle contains puppet-tripleo-7.4.12-8,
the cron job exists:
[heat-admin@controller-0 ~]$ sudo docker exec -ti panko_api crontab -u panko -l 
# HEADER: This file was autogenerated at 2018-08-13 08:53:59 +0000 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: panko-expirer
PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh
1 0 * * * panko-expirer

Comment 17 errata-xmlrpc 2018-08-20 12:59:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2331