Bug 1570140
Summary: | Hawkular is not clearing out old data even though hawkular.metrics.default-ttl is specified | ||||||
---|---|---|---|---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Luke Stanton <lstanton> | ||||
Component: | Hawkular | Assignee: | John Sanda <jsanda> | ||||
Status: | CLOSED DUPLICATE | QA Contact: | Junqi Zhao <juzhao> | ||||
Severity: | medium | Docs Contact: | |||||
Priority: | unspecified | ||||||
Version: | 3.7.0 | CC: | aos-bugs, juzhao, lstanton | ||||
Target Milestone: | --- | ||||||
Target Release: | 3.7.z | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2018-05-31 01:40:48 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | 1567222 | ||||||
Bug Blocks: | |||||||
Attachments: |
|
Description
Luke Stanton
2018-04-20 17:28:02 UTC
The customer might be hitting bug 1567222. Can I get the output of `du -h /cassandra_data/data/hawkular_metrics`. Cassandra is getting bogged down with garbage collection which is very likely the cause for most of the exceptions you are seeing in the logs. I recommend doubling the memory to 4 GB for the Cassandra pod. Created attachment 1424759 [details]
Older output from du /cassandra_data/data/hawkular_metrics
I don't know if this is still useful but customer had attached this data in an earlier comment.
(In reply to Luke Stanton from comment #4) > Created attachment 1424759 [details] > Older output from du /cassandra_data/data/hawkular_metrics > > I don't know if this is still useful but customer had attached this data in > an earlier comment. Definitely useful. It does look like the customer is hitting bug 1567222. As a temporary work around until the fix is pushed out run: $ oc -n openshift-infra <cassandra pod> nodetool clearsnapshot *** This bug has been marked as a duplicate of bug 1567222 *** |