Bug 1470046

Description of problem:
When join a cluster to a federation, secret is generated before cluster's adding. When unjoin a cluster from a federation, cluster is removed and then secret is deleted. The problem is, with invalid command, cluster has been removed but secret left, or secret has been generated but cluster can't be added into the federation. This inconsistency will block next correct operation.    


Version-Release number of selected component (if applicable):
openshift v3.6.135
kubernetes v1.6.1+5115d708d7
etcd 3.2.1
registry.ops.openshift.com/openshift3/ose-federation   v3.6.140            


How reproducible:
Always

Steps to Reproduce:
1. Given there are two clusters in the federation
# oc get cluster --context=qwangfed
NAME       STATUS    AGE
cluster1   Ready     2h
cluster2   Ready     8m

2. Remove cluster2 from the federation with invalid parameters. Then check cluster and secret
# kubefed unjoin cluster2 --host-cluster-context=adc --context=qwangfed 
# oc get cluster --context=qwangfed
# oc get secret -n federation-system | grep cluster2

3. Join the deleted cluster to the federation again 
# kubefed join cluster2  --host-cluster-context=${HOST_CONTEXT} --cluster-context=${CLUSTER2_CONTEXT} --context=qwangfed

4. Delete secret 
# oc get secret -n federation-system | grep cluster2
# oc delete secret cluster2 -n federation-system

5. Do step 3 again and check cluster

6. Remove cluster2 from the federation with valid parameters and check cluster and secret
# kubefed unjoin cluster2 --context=qwangfed

7. Join cluster2 to the federation without mandatory parameters. Then Check cluster and secret
# kubefed join cluster2  --host-cluster-context=${HOST_CONTEXT}  --cluster-context=${CLUSTER2_CONTEXT}

8. Join cluster2 to the federation with correct parameters (Step 3).



Actual results:
2. # kubefed unjoin cluster2 --host-cluster-context=adc --context=qwangfed --federation-system-namespace='123'
error: context "adc" does not exist

# oc get cluster --context=qwangfed
NAME       STATUS    AGE
cluster1   Ready     2h

--->Expected result: Cluster2 shouldn't be removed 

# oc get secret -n federation-system | grep cluster2
cluster2                                        Opaque                                1         4m

---> Cluster has been removed but secret left, this will the cluster2's rejoin.


3. # kubefed join cluster2  --host-cluster-context=${HOST_CONTEXT} --cluster-context=${CLUSTER2_CONTEXT} --context=qwangfed
Error from server (AlreadyExists): secrets "cluster2" already exists


4. # oc delete secret cluster2 -n federation-system
secret "cluster2" deleted


5. # kubefed join cluster2  --host-cluster-context=${HOST_CONTEXT} --cluster-context=${CLUSTER2_CONTEXT} --context=qwangfed
cluster "cluster2" created

# oc get cluster --context=qwangfed
NAME       STATUS    AGE
cluster1   Ready     2h
cluster2   Ready     1m


6. # kubefed unjoin cluster2 --context=qwangfed
Successfully removed cluster "cluster2" from federation

# oc get cluster --context=qwangfed
NAME       STATUS    AGE
cluster1   Ready     2h

# oc get secret -n federation-system | grep cluster2


7. # kubefed join cluster2  --host-cluster-context=${HOST_CONTEXT}  --cluster-context=${CLUSTER2_CONTEXT}
error: server does not support API version "federation/v1beta1

# oc get cluster --context=qwangfed
NAME       STATUS    AGE
cluster1   Ready     2h

# oc get secret -n federation-system | grep cluster2
cluster2                                        Opaque

--->Expected result: secret shouldn't be generated. The secret will block cluster2's join 


8. # kubefed join cluster2  --host-cluster-context=${HOST_CONTEXT} --context=qwangfed --cluster-context=${CLUSTER2_CONTEXT}
Error from server (AlreadyExists): secrets "cluster2" already exists


Expected results:
2. With incorrect "unjoin" parameters, cluster shouldn't be removed.
7. Lack mandatory "join" parameters, secret shouldn't be created.

  
Additional info: