Bug 1706774 - [OSP14] cinder cron job definition should have randomized sleep command like puppet-heat for decreasing DB spike or lock contention.
Summary: [OSP14] cinder cron job definition should have randomized sleep command like ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 14.0 (Rocky)
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ---
: 14.0 (Rocky)
Assignee: Pablo Caruana
QA Contact: Sasha Smolyak
URL:
Whiteboard:
Depends On:
Blocks: 1603014
TreeView+ depends on / blocked
 
Reported: 2019-05-06 09:50 UTC by Pablo Caruana
Modified: 2019-07-02 20:08 UTC (History)
17 users (show)

Fixed In Version: openstack-tripleo-heat-templates-9.3.1-0.20190314162766.d0a6cb1.el7ost puppet-cinder-13.3.3-0.20190118031243.0d80a9f.el7ost
Doc Type: Bug Fix
Doc Text:
This patch introduces two new parameters: * CinderCronDbPurgeMaxDelay - Configure maxdelay parameter in the Cinder database purge cron job. * NovaCronDBArchiveMaxDelay - Configure maxdelay for the Nova database archive cron job.
Clone Of: 1603014
Environment:
Last Closed: 2019-07-02 20:08:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1823274 0 None None None 2019-05-06 09:50:26 UTC
OpenStack gerrit 644716 0 None None None 2019-05-06 09:58:27 UTC
OpenStack gerrit 656660 0 None None None 2019-05-06 09:58:27 UTC
Red Hat Product Errata RHBA-2019:1672 0 None None None 2019-07-02 20:08:59 UTC

Description Pablo Caruana 2019-05-06 09:50:27 UTC
Description of problem:
cinder cron job definition should have randomized sleep command like puppet-heat for decreasing DB spike or lock contention.

Version-Release number of selected component (if applicable):
Current puppet-cinder.

Additional info:
Current puppet-cinder configures the cron job like
~~~
# HEADER: This file was autogenerated at 2018-07-13 06:00:08 +0000 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: cinder-manage db purge
PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh
1 0 * * * cinder-manage db purge 30 >>/var/log/cinder/cinder-rowsflush.log 2>&1
~~~

If many DB cron jobs are executed at the same time, it will trigger lock contention or DB spike.

We should have randomized sleep in puppet-heat[0].

[0] https://github.com/openstack/puppet-heat/blob/master/manifests/cron/purge_deleted.pp#L70-L75

Comment 14 errata-xmlrpc 2019-07-02 20:08:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1672


Note You need to log in before you can comment on or make changes to this bug.