Description of problem: when trying to scale down from a two nodes cluster to a single one Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. create a single node cluster 2. expand it with an additional 3. try to get rid of the initial node by deleting him as member in etcd or deleting in Openshift with oc delete node and making him physically unavailable Actual results: access to api is lost, etcd hangs and crashes if rebooted Expected results: cluster continues to operate Additional info:
Created attachment 1724877 [details] etcd logs failure
Did you get this figured out can we close this bug? > ETCD_INITIAL_CLUSTER=qctsingle-master-1.karmalabs.com this is in the wrong format needs to be <key>=<value> > 3. try to get rid of the initial node by deleting him as member in etcd or deleting in Openshift with oc delete node and making him physically unavailable Can we validate full steps to understand the problem better?
no i havent figured it out
This bug hasn't had any activity in the last 30 days. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're marking this bug as "LifecycleStale" and decreasing the severity/priority. If you have further information on the current state of the bug, please update it, otherwise this bug can be closed in about 7 days. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. Additionally, you can add LifecycleFrozen into Keywords if you think this bug should never be marked as stale. Please consult with bug assignee before you do that.
as etcd is a static pod if you delete the node `oc delete node` the member will continue to run. if the node is still part of the cluster and you scale down etcd via Cluster API `etcdctl member remove $id` and you only have 2 nodes net result will be quorum loss. Reason being because node is still existent we will try to scale etcd back up on that node. Butr because it was a previous member its data-dir still exists. This will cause etcd to fail to start which has a net result of quorum loss in a 2 member cluster. This is why scaling is only supported with a 3 member cluster. I am going to mark this a dupe of 1880759. *** This bug has been marked as a duplicate of bug 1880759 ***