Bug 1598445
| Summary: | [starter-us-west-1] metrics in crash loop backoff: java.lang.UnsupportedOperationException: Other bootstrapping/leaving/moving nodes detected, cannot bootstrap while cassandra.consistent.rangemovement is true | ||||||
|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Justin Pierce <jupierce> | ||||
| Component: | Installer | Assignee: | John Sanda <jsanda> | ||||
| Status: | CLOSED NOTABUG | QA Contact: | Johnny Liu <jialiu> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 3.10.0 | CC: | aos-bugs, jokerman, jsanda, jupierce, mmccomas, rvargasp, sdodson | ||||
| Target Milestone: | --- | Keywords: | Reopened | ||||
| Target Release: | 3.10.z | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2018-08-06 19:16:16 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Here is the exception message from Cassandra: Exception (java.lang.UnsupportedOperationException) encountered during startup: Other bootstrapping/leaving/moving nodes detected, cannot bootstrap while cassandra.consistent.rangemovement is true java.lang.UnsupportedOperationException: Other bootstrapping/leaving/moving nodes detected, cannot bootstrap while cassandra.consistent.rangemovement is true You cannot bring new instances into the Cassandra cluster concurrently. It has to be done serially. Once the pod reaches the ready state, then start the next one. We of course would get this behavior for free with StatefulSets, but I do not think it makes sense to invest the effort in moving Cassandra to StatefulSets at this point. I am going to close this out as this is known, expected behavior. This occurred during a normal upgrade. It may be an openshift-ansible issue. I also need to know how to rectify the existing condition safely. Cleared using manual recover. Suspicion is that a Cassandra was manually & ungracefully decommissioned. The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days |
Created attachment 1456777 [details] pod listings & exception from log Description of problem: hawkular-cassandra pod in crash loop after upgrade to v3.10.9 Version-Release number of selected component (if applicable): v3.10.9 How reproducible: Unknown Actual results: See listings in attachment