Description of problem:
After creating a pod on a worker and then trying to delete the worker without draining it, the pod is successfully moved to another worker and gets stuck in 'Terminating' state on the deleted worker. The deleted worker can't be deleted and is still listed when checking the nodes but it's unreachable.
If trying to drain the node with the running pod after the mentioned above,
The pod is moved to another worker, And get's stuck in a 'Terminating' state on the current node. Hence, the current node is not deleted properly.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Create a pod on worker.
2. Delete the worker without draining. cmd: $ oc adm drain
$ oc annotate machine <worker-CONSUMER-name> machine.openshift.io/cluster-api-delete-machine=yes -n openshift-machine-api
$ oc delete bmh <worker-name> -n openshift-machine-api
3. The pod status is stuck on terminating, So we're unable to delete the worker node. When trying delete the worker node, the status became NotReady, SchedulingDisabled.
$ oc get nodes:
Status of deleted worker node: NotReady, SchedulingDisabled.
$ oc get machine -o wide -n openshift-machine-api:
The node is still listed.
Node is successfully deleted and is not listed anywhere(
$ oc get nodes
$ oc get bmh -n openshift-machine-api
$ oc get machine -o wide -n openshift-machine-api
Can you provide information on the crio and kubelet versions in use?
Also, we need the kubelet and crio logs as well if you have them.
Created attachment 1682043 [details]
I can not reproduce the above steps, in step 2:
$ oc delete bmh ip-10-0-168-12.us-east-2.compute.internal -n openshift-machine-api
Error from server (NotFound): baremetalhosts.metal3.io "ip-10-0-168-12.us-east-2.compute.internal" not found
It seems that baremetalhosts don't exist!
Setting target release to current development version (4.5) for investigation. Where fixes (if any) are required/requested for prior versions, cloned BZs will be created when appropriate.
The issue here is that the cluster-api-provider-baremetal revendored github.com/openshift/cluster-api. In 4.4 and newer, the relevant machine-api code has moved to github.com/openshift/machine-api-operator and should be vendored instead. github.com/openshift/cluster-api is no longer in development.
*** Bug 1837505 has been marked as a duplicate of this bug. ***
*** Bug 1840577 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.