Description of problem:

  KubeletConfig CR cannot be applied to a custom Machine Config Pool. After issuing 'oc create -f <KubeletConifg>', the MCP does not trigger a rolling update for nodes.


How reproducible:

Always

ENV:

# oc version
Client Version: 4.6.27
Server Version: 4.6.27
Kubernetes Version: v1.19.0+d856161


Steps to Reproduce:


 1) Create a custom MCP, then add a worker node to the custom MCP.
 
~~~
$ oc get node
NAME                                       STATUS   ROLES           AGE     VERSION
...
worker-1.ocp4628.lab.pnq2.cee.redhat.com   Ready    custom,worker   2d20h   v1.19.0+a5a0987

$ oc get mcp --show-labels 
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE     LABELS
custom   rendered-custom-f8e9b01721f86dd0b440dfe5eddf65ee   True      False      False      1              1                   1                     0                      13h     custom-kubelet=large-pods
master   rendered-master-e7b7f9790445eed70c72e85248bdaa8e   True      False      False      3              3                   3                     0                      2d21h   machineconfiguration.openshift.io/mco-built-in=,operator.machineconfiguration.openshift.io/required-for-upgrade=,pools.operator.machineconfiguration.openshift.io/master=
worker   rendered-worker-f8e9b01721f86dd0b440dfe5eddf65ee   True      False      False      1              1                   1                     0                      2d21h   machineconfiguration.openshift.io/mco-built-in=,pools.operator.machineconfiguration.openshift.io/worker=,worker-kubelet=small-pods
~~~


 2) Apply a KubeletConfig CR to the worker node, then it will apply the change to nodes in worker MCP with rolling update.

~~~
$ cat small-pod.yaml 
apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
  name: small-pods
spec:
  machineConfigPoolSelector:
    matchLabels:
      worker-kubelet: small-pods
  kubeletConfig:
    systemReserved:
      cpu: 100m

$ oc create -f small-pod.yaml  <--- After this, the worker nodes in worker MCP did rolling update to apply the change.
~~~

 3) However, if create a KubeletConfig CR to apply a change to the custom MCP, the custom MCP dosen't trigger a rolling update.

~~~
$ cat custom-memory.yaml 
apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
  name: custom-memory
spec:
  machineConfigPoolSelector:
    matchLabels:
      custom-kubelet: large-pods
  kubeletConfig:
    systemReserved:
      memory: 2Gi

$ oc create -f custom-momory.yaml  <--- did not triiger a rolling update.
~~~

Actual results: 

 The KubeletConfig CR for worker MCP can trigger a rolling update, however another KubeletConifg CR cannot override the configuration of a node that belongs to both worker and custom MCP, because it cannot trigger a rolling update after the worker MCP update.


Expected results:

custom MCP should trigger a rolling update after applying the kubeletconfig CR.

Additional info:

Question: 

 1. Is this behavior by designed?
 2. Do we have a workaround to override/replace the old kubeletconfig corresponding to worker MCP with a new kubeletconfig CR corresponding to custom MCP?