Description of problem:
Cassandra enables compression of SSTables by default. snappy is the default compression used. Compression is enabled for system tables as well. unlike user tables however, the system schema cannot be altered. This leads to severe consequences in the following, albeit unlikely, scenario.

I install a storage node running on OpenJDK or Oracle Java on a platform that snappy-java supports. Cassandra will create the system tables with compression enabled using snappy. I then later restart Cassandra using IBM Java. Due to an existing bug[1] in snappy-java that prevents the native library from being loaded when running on IBM Java, Cassandra will fail to initialize because it won't be able to decompress and read the system tables. I have logged an issue[2] with the Cassandra project to add support for making system table compression configurable.

If a user is running multiple nodes, a suitable work around might be to simple re-install that node. When the node is re-installed (running on IBM Java), data will be streamed from other nodes and data files will be written to disk without compression. 

I am not sure if we have a work around for single node deployments. Hopefully, support for making system table compression configurable will be added in time for JON 3.2. The option may be to use an upgraded version of snappy-java, which includes the fix, or use a patched version that includes the fix. 

[1] https://github.com/xerial/snappy-java/issues/34
[2] https://issues.apache.org/jira/browse/CASSANDRA-5548

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info: