Bug 1522793 - Hawkular Cassandra SSTables corruptions
Summary: Hawkular Cassandra SSTables corruptions
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Hawkular
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.5.z
Assignee: John Sanda
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On: 1531790 1531795 1595317
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-12-06 13:18 UTC by Vladislav Walek
Modified: 2021-03-11 16:32 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-04 13:59:38 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Vladislav Walek 2017-12-06 13:18:05 UTC
Description of problem:

Issue with hawkular cassandra - the cassandra is failing due the error below:

 "Failed to perform operation due to an error: Cassandra failure during read query at consistency LOCAL_ONE (1 responses were required but only 0 replica responded, 1 failed)", and we think that it could be related to SSTables corruptions.


When running the scrub tool the error message is showing the error:

/opt/apache-cassandra/bin/nodetool scrub

WARN  [CompactionExecutor:4] 2017-12-05 04:15:32,264 OutputHandler.java:52 - Row starting at position 1538453 is unreadable; skipping to next
WARN  [CompactionExecutor:4] 2017-12-05 04:15:32,265 OutputHandler.java:57 - Error reading row (stacktrace follows):
java.io.IOError: java.io.IOException: Unable to read row key from data file


The possible issue was already reported here:
https://bugzilla.redhat.com/show_bug.cgi?id=1404643


Version-Release number of selected component (if applicable):
OpenShift Container Platform 3.5

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 10 John Sanda 2018-02-16 22:36:58 UTC
I think the root cause for this is bug 1531790. Updating status to POST since the fix is addressed by bug 1531790.


Note You need to log in before you can comment on or make changes to this bug.