Bug 1567251 - Metrics Casandra PV running out of space due to snapshots
Summary: Metrics Casandra PV running out of space due to snapshots
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Hawkular
Version: 3.9.0
Hardware: x86_64
OS: Linux
Target Milestone: ---
: 3.10.0
Assignee: John Sanda
QA Contact: Junqi Zhao
Depends On:
Blocks: 1567222 1567250
TreeView+ depends on / blocked
Reported: 2018-04-13 16:22 UTC by John Sanda
Modified: 2021-12-10 15:57 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: auto_snapshot was true in cassandra.yaml, and due to changes in Hawkular Metrics introduced in OCP 3.7, lots of snapshots are generated. Consequence: The snapshots can eventually fill up the disk. Fix: Disable auto_snapshot and make snapshotting configurable. Snapshots will only be generated now when the installer runs if the openshift_metrics_cassandra_take_snapshot property is set to true (case insensitive) in the ansible inventory file. Result:
Clone Of: 1567250
Last Closed: 2018-07-30 19:12:49 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:1816 0 None None None 2018-07-30 19:13:33 UTC

Comment 1 John Sanda 2018-04-17 14:55:58 UTC
PRs with changes:


The Cassandra image now sets auto_snapshot to false in cassandra.yaml. The image also has a new environment variable, TAKE_SNAPSHOT, which defaults to false. The environment variable can be controlled by setting the openshift_metrics_cassandra_take_snapshot property in your ansible inventory file. When set to true, a snapshot will be generated during the post startup life cycle hook. The name of the snapshot directory will be a timestamp.

Comment 7 Junqi Zhao 2018-05-17 01:49:23 UTC
Default value for openshift_metrics_cassandra_take_snapshot is false, environment variable TAKE_SNAPSHOT is also false, when set openshift_metrics_cassandra_take_snapshot to true, environment variable TAKE_SNAPSHOT will be true, and can take snapshot.

openshift-ansible vesion


Comment 9 errata-xmlrpc 2018-07-30 19:12:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.