Bug 1570147 - panko events are kept in DB forever, panko-expirer utility inclusion
Summary: panko events are kept in DB forever, panko-expirer utility inclusion
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: puppet-panko
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z3
: 12.0 (Pike)
Assignee: Pradeep Kilambi
QA Contact: Sasha Smolyak
URL:
Whiteboard:
Keywords: Triaged, ZStream
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-20 17:58 UTC by Andrew Ludwar
Modified: 2018-08-20 13:00 UTC (History)
6 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2018-08-20 12:59:48 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2331 None None None 2018-08-20 13:00 UTC
OpenStack gerrit 539591 None None None 2018-07-12 12:12 UTC
OpenStack gerrit 544598 None None None 2018-07-12 12:09 UTC
OpenStack gerrit 563212 None None None 2018-07-12 12:09 UTC
OpenStack gerrit 587810 None None None 2018-08-01 18:38 UTC

Description Andrew Ludwar 2018-04-20 17:58:20 UTC
Description of problem:

Customer has a large panko database size and is affecting performance. The events don't seem to have a cleanup or expiry process.


Version-Release number of selected component (if applicable):

OSP12

How reproducible:

Every time.

Steps to Reproduce:
1. Deploy OSP12
2. Observe panko events count always increase:

MariaDB [panko]> use panko;
Database changed
MariaDB [panko]> select count(*) from event;
+----------+
| count(*) |
+----------+
|  1736043 |
+----------+
1 row in set (0.37 sec)

3. Observe MySQL database growing in size.


Actual results:

Events and DB size grows uncontrolled.

Expected results:

Events and DB size can be managed by a utility.

Additional info:

A retention utility is needed for panko.

There's been upstream activity to address this:

https://review.openstack.org/#/c/539579/
https://bugs.launchpad.net/tripleo/+bug/1746514
https://review.openstack.org/#/q/topic:bug/1746514+(status:open+OR+status:merged)

May we please include this in a future maintenance release of OSP12? Creating a BZ to track this activity. I didn't see this code in the latest shipped puppet-panko.

Comment 7 Sasha Smolyak 2018-08-01 12:46:23 UTC
puppet-tripleo code didn't make it to the puddle, so the code is there, but the crone isn't configured.
Running
> sudo docker exec -ti panko_api crontab -u panko -l 
on controller gives " no crontab for panko"

Comment 11 Sasha Smolyak 2018-08-08 10:17:52 UTC
The fix is not in the latest puddle

Comment 15 Sasha Smolyak 2018-08-13 10:50:48 UTC
The latest puddle contains puppet-tripleo-7.4.12-8,
the cron job exists:
[heat-admin@controller-0 ~]$ sudo docker exec -ti panko_api crontab -u panko -l 
# HEADER: This file was autogenerated at 2018-08-13 08:53:59 +0000 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: panko-expirer
PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh
1 0 * * * panko-expirer

Comment 17 errata-xmlrpc 2018-08-20 12:59:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2331


Note You need to log in before you can comment on or make changes to this bug.