Bug 1462461 - Cassandra in CrashLoopBackOff with sstables error message
Summary: Cassandra in CrashLoopBackOff with sstables error message
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Hawkular
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Matt Wringe
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-17 20:52 UTC by Steven Walter
Modified: 2020-07-16 09:50 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-06-17 22:32:04 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Steven Walter 2017-06-17 20:52:39 UTC
Description of problem:
Cassandra pod in CrashLoopBackOff. Cassandra logs has message like:

ERROR 20:28:24 Detected unreadable sstables /cassandra_data/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/mc-1469-big-Statistics.db,

Version-Release number of selected component (if applicable):
3.4.1

UNCONFIRMED

Actual results:

..8<
INFO  20:28:24 JVM Arguments: [-Duser.home=/home/jboss, -Duser.name=jboss, -Dcassandra.commitlog.ignorereplayerrors=true, -ea, -javaagent:/opt/apache-cassandra/lib/jamm-0.3.0.jar, -XX:+CMSClassUnloadingEnabled, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -Xms1955M, -Xmx1955M, -Xmn400M, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+UseTLAB, -XX:+PerfDisableSharedMem, -XX:CompileCommandFile=/opt/apache-cassandra/conf/hotspot_compiler, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:CMSWaitDuration=10000, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -Xloggc:/opt/apache-cassandra/logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Djava.net.preferIPv4Stack=true, -Dcassandra.jmx.local.port=7199, -XX:+DisableExplicitGC, -Djava.library.path=/opt/apache-cassandra/lib/sigar-bin, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/opt/apache-cassandra/logs, -Dcassandra.storagedir=/opt/apache-cassandra/data, -Dcassandra-foreground=yes]
WARN  20:28:24 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
WARN  20:28:24 jemalloc shared library could not be preloaded to speed up memory allocations
WARN  20:28:24 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
WARN  20:28:24 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
INFO  20:28:24 Initializing SIGAR library
WARN  20:28:24 Cassandra server running in degraded mode. Is swap disabled? : false,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : true 
WARN  20:28:24 Directory /opt/apache-cassandra/data/saved_caches doesn't exist
ERROR 20:28:24 Detected unreadable sstables /cassandra_data/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/mc-1469-big-Statistics.db,

Providing full logs in private comments

Comment 5 Steven Walter 2017-06-17 22:32:04 UTC
Determined images from a different OCP version were in use. Closing bug pending fixing the image tags.


Note You need to log in before you can comment on or make changes to this bug.