Bug 2086728

Summary: [hypershift] Move drain to controller
Product: OpenShift Container Platform Reporter: Colin Walters <walters>
Component: Machine Config OperatorAssignee: MCO Bug Bot <mco-triage>
Machine Config Operator sub component: Machine Config Operator QA Contact: Rio Liu <rioliu>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: mkrejci
Version: 4.11   
Target Milestone: ---   
Target Release: 4.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-08-10 11:12:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Colin Walters 2022-05-16 14:06:55 UTC 
In order to deliver hypershift in-place updates, we need to centralize drain in the controller.
Comment 5 Rio Liu 2022-06-16 10:58:30 UTC 
oc get clusterversion                                                                                                                                                                                                             ⎈ admin
NAME      VERSION       AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.11.0-fc.0   True        False         6m19s   Cluster version is 4.11.0-fc.0

# create machine config
oc create -f file-ig3.yaml                                                                                                                                                                                            ⎈ admin
machineconfig.machineconfiguration.openshift.io/test-file created

cat file-ig3.yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: test-file
spec:
  config:
    ignition:
      version: 3.2.0
    storage:
      files:
      - contents:
          source: data:text/plain;charset=utf;base64,c2VydmVyIGZvby5leGFtcGxlLm5ldCBtYXhkZWxheSAwLjQgb2ZmbGluZQpzZXJ2ZXIgYmFyLmV4YW1wbGUubmV0IG1heGRlbGF5IDAuNCBvZmZsaW5lCnNlcnZlciBiYXouZXhhbXBsZS5uZXQgbWF4ZGVsYXkgMC40IG9mZmxpbmUK
        mode: 420
        path: /etc/test

 oc get mc --sort-by metadata.creationTimestamp |tail -2                                                                                                                                                               ⎈ admin
test-file                                                                                     3.2.0             41s
rendered-worker-6f74ff89abbdc14629efc815d6c2d329   4fbbead6d1a6e83d9736e75cdc359e10d5adfb84   3.2.0             36s

oc get mcp/worker                                                                                                                                                                                                     ⎈ admin
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
worker   rendered-worker-6f74ff89abbdc14629efc815d6c2d329   True      False      False      3              3                   3                     0                      28m

# drain related annotation are updated
oc get node/ip-10-0-143-243.us-east-2.compute.internal -o yaml | yq -y '.metadata.annotations' |grep -i drain                                                                                                         ⎈ admin
machineconfiguration.openshift.io/desiredDrain: uncordon-rendered-worker-6f74ff89abbdc14629efc815d6c2d329
machineconfiguration.openshift.io/lastAppliedDrain: uncordon-rendered-worker-6f74ff89abbdc14629efc815d6c2d329

# check mcc log to verify drain behavior
oc get pod -n openshift-machine-config-operator|grep controller                                                                                                                                                       ⎈ admin
machine-config-controller-7fffff86d7-nqhj9   2/2     Running   0          31m

oc logs -n openshift-machine-config-operator machine-config-controller-7fffff86d7-nqhj9 |grep drain_controller                                                                                                        ⎈ admin
Defaulted container "machine-config-controller" out of: machine-config-controller, oauth-proxy
I0616 10:25:30.117452       1 drain_controller.go:128] Starting MachineConfigController-DrainController
I0616 10:25:41.450032       1 drain_controller.go:141] node ip-10-0-140-80.us-east-2.compute.internal: uncordoning
I0616 10:25:41.450180       1 drain_controller.go:141] node ip-10-0-140-80.us-east-2.compute.internal: initiating uncordon (currently schedulable: true)
I0616 10:25:41.453133       1 drain_controller.go:141] node ip-10-0-140-80.us-east-2.compute.internal: uncordon succeeded (currently schedulable: true)
I0616 10:25:41.453192       1 drain_controller.go:141] node ip-10-0-140-80.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:25:49.353779       1 drain_controller.go:141] node ip-10-0-209-65.us-east-2.compute.internal: uncordoning
I0616 10:25:49.353801       1 drain_controller.go:141] node ip-10-0-209-65.us-east-2.compute.internal: initiating uncordon (currently schedulable: true)
I0616 10:25:49.356615       1 drain_controller.go:141] node ip-10-0-209-65.us-east-2.compute.internal: uncordon succeeded (currently schedulable: true)
I0616 10:25:49.356626       1 drain_controller.go:141] node ip-10-0-209-65.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:26:34.044134       1 drain_controller.go:141] node ip-10-0-185-204.us-east-2.compute.internal: uncordoning
I0616 10:26:34.044164       1 drain_controller.go:141] node ip-10-0-185-204.us-east-2.compute.internal: initiating uncordon (currently schedulable: true)
I0616 10:26:34.047081       1 drain_controller.go:141] node ip-10-0-185-204.us-east-2.compute.internal: uncordon succeeded (currently schedulable: true)
I0616 10:26:34.047095       1 drain_controller.go:141] node ip-10-0-185-204.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:29:42.128876       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: uncordoning
I0616 10:29:42.128917       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: initiating uncordon (currently schedulable: true)
I0616 10:29:42.132665       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: uncordon succeeded (currently schedulable: true)
I0616 10:29:42.132680       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:30:14.028834       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: uncordoning
I0616 10:30:14.028853       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: initiating uncordon (currently schedulable: true)
I0616 10:30:14.032632       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: uncordon succeeded (currently schedulable: true)
I0616 10:30:14.032688       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:30:16.725505       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: uncordoning
I0616 10:30:16.725530       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: initiating uncordon (currently schedulable: true)
I0616 10:30:16.728720       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: uncordon succeeded (currently schedulable: true)
I0616 10:30:16.728734       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:45:46.780130       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: initiating drain
I0616 10:45:46.780146       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: cordoning
I0616 10:45:46.780165       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: initiating cordon (currently schedulable: true)
I0616 10:45:46.808745       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: cordon succeeded (currently schedulable: false)
E0616 10:45:47.482793       1 drain_controller.go:106] WARNING: ignoring DaemonSet-managed Pods: openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx5wh, openshift-cluster-node-tuning-operator/tuned-ppnwj, openshift-dns/dns-default-tm885, openshift-dns/node-resolver-mbwf6, openshift-image-registry/node-ca-6f8pn, openshift-ingress-canary/ingress-canary-rt9f8, openshift-machine-config-operator/machine-config-daemon-4smqs, openshift-monitoring/node-exporter-tvfxs, openshift-multus/multus-additional-cni-plugins-zbmzp, openshift-multus/multus-ffs2p, openshift-multus/network-metrics-daemon-w4qh9, openshift-network-diagnostics/network-check-target-xd6rp, openshift-sdn/sdn-kd24l
I0616 10:45:47.484877       1 drain_controller.go:106] evicting pod openshift-monitoring/thanos-querier-bfcf48dd5-nv4cl
I0616 10:45:47.484880       1 drain_controller.go:106] evicting pod openshift-monitoring/prometheus-k8s-0
I0616 10:45:47.484889       1 drain_controller.go:106] evicting pod openshift-image-registry/image-registry-75f4df699-2tqkd
I0616 10:45:47.484918       1 drain_controller.go:106] evicting pod openshift-monitoring/prometheus-operator-admission-webhook-cf8dd9f97-8bw6w
I0616 10:45:49.541866       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: Evicted pod openshift-monitoring/prometheus-operator-admission-webhook-cf8dd9f97-8bw6w
I0616 10:45:50.561177       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: Evicted pod openshift-monitoring/thanos-querier-bfcf48dd5-nv4cl
I0616 10:45:50.561675       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: Evicted pod openshift-monitoring/prometheus-k8s-0
I0616 10:46:15.558068       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: Evicted pod openshift-image-registry/image-registry-75f4df699-2tqkd
I0616 10:46:15.558102       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:47:21.736978       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: uncordoning
I0616 10:47:21.736998       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: initiating uncordon (currently schedulable: false)
I0616 10:47:21.756859       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: uncordon succeeded (currently schedulable: true)
I0616 10:47:21.756877       1 drain_controller.go:141] node ip-10-0-183-219.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:47:31.776381       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: initiating drain
I0616 10:47:31.776400       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: cordoning
I0616 10:47:31.776406       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: initiating cordon (currently schedulable: true)
I0616 10:47:31.798694       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: cordon succeeded (currently schedulable: false)
E0616 10:47:32.466210       1 drain_controller.go:106] WARNING: ignoring DaemonSet-managed Pods: openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-252rs, openshift-cluster-node-tuning-operator/tuned-h4rmt, openshift-dns/dns-default-s5wrs, openshift-dns/node-resolver-8486h, openshift-image-registry/node-ca-tbbvf, openshift-ingress-canary/ingress-canary-fvkj5, openshift-machine-config-operator/machine-config-daemon-qh22j, openshift-monitoring/node-exporter-85jvv, openshift-multus/multus-additional-cni-plugins-hknmt, openshift-multus/multus-gsz66, openshift-multus/network-metrics-daemon-2dlwv, openshift-network-diagnostics/network-check-target-mbsln, openshift-sdn/sdn-76lk6
I0616 10:47:32.468297       1 drain_controller.go:106] evicting pod openshift-monitoring/thanos-querier-bfcf48dd5-sd4m7
I0616 10:47:32.468308       1 drain_controller.go:106] evicting pod openshift-console/downloads-6b69dbff89-h4xst
I0616 10:47:32.468368       1 drain_controller.go:106] evicting pod openshift-monitoring/prometheus-k8s-0
I0616 10:47:32.468395       1 drain_controller.go:106] evicting pod openshift-monitoring/openshift-state-metrics-8679b4d578-z2pnv
I0616 10:47:32.468492       1 drain_controller.go:106] evicting pod openshift-monitoring/prometheus-operator-admission-webhook-cf8dd9f97-jtf9r
I0616 10:47:32.468520       1 drain_controller.go:106] evicting pod openshift-monitoring/prometheus-adapter-6bd77d4dbb-66sfp
I0616 10:47:32.468371       1 drain_controller.go:106] evicting pod openshift-image-registry/image-registry-75f4df699-qpqds
I0616 10:47:32.468377       1 drain_controller.go:106] evicting pod openshift-ingress/router-default-7d5777c977-bhc4n
I0616 10:47:32.468386       1 drain_controller.go:106] evicting pod openshift-monitoring/alertmanager-main-0
I0616 10:47:32.468299       1 drain_controller.go:106] evicting pod openshift-monitoring/kube-state-metrics-7788cd8d7f-l6kl4
I0616 10:47:32.468722       1 drain_controller.go:106] evicting pod openshift-monitoring/telemeter-client-66bc78d587-zpb7t
I0616 10:47:32.522627       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-console/downloads-6b69dbff89-h4xst
I0616 10:47:33.884522       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-monitoring/prometheus-operator-admission-webhook-cf8dd9f97-jtf9r
I0616 10:47:34.884154       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-monitoring/openshift-state-metrics-8679b4d578-z2pnv
I0616 10:47:35.084393       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-monitoring/thanos-querier-bfcf48dd5-sd4m7
I0616 10:47:35.484530       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-monitoring/prometheus-adapter-6bd77d4dbb-66sfp
I0616 10:47:35.908779       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-monitoring/kube-state-metrics-7788cd8d7f-l6kl4
I0616 10:47:36.287157       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-monitoring/alertmanager-main-0
I0616 10:47:36.484584       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-monitoring/telemeter-client-66bc78d587-zpb7t
I0616 10:47:36.888129       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-monitoring/prometheus-k8s-0
I0616 10:48:00.583084       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-image-registry/image-registry-75f4df699-qpqds
I0616 10:48:41.544256       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: Evicted pod openshift-ingress/router-default-7d5777c977-bhc4n
I0616 10:48:41.544291       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:48:41.556157       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: initiating drain
I0616 10:48:41.556175       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: cordoning
I0616 10:48:41.556181       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: initiating cordon (currently schedulable: false)
I0616 10:48:41.569856       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: cordon succeeded (currently schedulable: false)
E0616 10:48:42.221733       1 drain_controller.go:106] WARNING: ignoring DaemonSet-managed Pods: openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-252rs, openshift-cluster-node-tuning-operator/tuned-h4rmt, openshift-dns/dns-default-s5wrs, openshift-dns/node-resolver-8486h, openshift-image-registry/node-ca-tbbvf, openshift-ingress-canary/ingress-canary-fvkj5, openshift-machine-config-operator/machine-config-daemon-qh22j, openshift-monitoring/node-exporter-85jvv, openshift-multus/multus-additional-cni-plugins-hknmt, openshift-multus/multus-gsz66, openshift-multus/network-metrics-daemon-2dlwv, openshift-network-diagnostics/network-check-target-mbsln, openshift-sdn/sdn-76lk6
I0616 10:48:42.221755       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:49:47.328601       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: uncordoning
I0616 10:49:47.328634       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: initiating uncordon (currently schedulable: false)
I0616 10:49:47.348784       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: uncordon succeeded (currently schedulable: true)
I0616 10:49:47.348799       1 drain_controller.go:141] node ip-10-0-143-243.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:49:57.356978       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: initiating drain
I0616 10:49:57.357010       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: cordoning
I0616 10:49:57.357016       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: initiating cordon (currently schedulable: true)
I0616 10:49:57.385418       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: cordon succeeded (currently schedulable: false)
E0616 10:49:58.046191       1 drain_controller.go:106] WARNING: ignoring DaemonSet-managed Pods: openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j69mx, openshift-cluster-node-tuning-operator/tuned-p9pfd, openshift-dns/dns-default-9zw5s, openshift-dns/node-resolver-xztr9, openshift-image-registry/node-ca-btfn9, openshift-ingress-canary/ingress-canary-mkmh2, openshift-machine-config-operator/machine-config-daemon-cgv4p, openshift-monitoring/node-exporter-jjtgj, openshift-multus/multus-additional-cni-plugins-7kpsb, openshift-multus/multus-wv866, openshift-multus/network-metrics-daemon-7crbd, openshift-network-diagnostics/network-check-target-96tt5, openshift-sdn/sdn-65bg5
I0616 10:49:58.048284       1 drain_controller.go:106] evicting pod openshift-image-registry/image-registry-75f4df699-xl85k
I0616 10:49:58.048298       1 drain_controller.go:106] evicting pod openshift-monitoring/thanos-querier-bfcf48dd5-bg7qp
I0616 10:49:58.048308       1 drain_controller.go:106] evicting pod openshift-operator-lifecycle-manager/collect-profiles-27589590-j557q
I0616 10:49:58.048301       1 drain_controller.go:106] evicting pod openshift-network-diagnostics/network-check-source-5786845dcc-f6bbc
I0616 10:49:58.048400       1 drain_controller.go:106] evicting pod openshift-monitoring/alertmanager-main-1
I0616 10:49:58.048449       1 drain_controller.go:106] evicting pod openshift-ingress/router-default-7d5777c977-vqg5f
I0616 10:49:58.048460       1 drain_controller.go:106] evicting pod openshift-monitoring/prometheus-adapter-6bd77d4dbb-gnjsg
I0616 10:49:58.048451       1 drain_controller.go:106] evicting pod openshift-monitoring/prometheus-operator-admission-webhook-cf8dd9f97-vzjvq
I0616 10:49:58.048507       1 drain_controller.go:106] evicting pod openshift-console/downloads-6b69dbff89-spqwz
I0616 10:49:58.048290       1 drain_controller.go:106] evicting pod openshift-operator-lifecycle-manager/collect-profiles-27589605-dw8t2
I0616 10:49:58.048569       1 drain_controller.go:106] evicting pod openshift-monitoring/prometheus-k8s-1
I0616 10:49:58.100220       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-operator-lifecycle-manager/collect-profiles-27589590-j557q
I0616 10:49:58.101521       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-operator-lifecycle-manager/collect-profiles-27589605-dw8t2
I0616 10:49:58.132248       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-console/downloads-6b69dbff89-spqwz
I0616 10:50:00.106273       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-network-diagnostics/network-check-source-5786845dcc-f6bbc
I0616 10:50:01.075175       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-monitoring/alertmanager-main-1
I0616 10:50:01.275842       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-monitoring/prometheus-operator-admission-webhook-cf8dd9f97-vzjvq
I0616 10:50:01.486715       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-monitoring/prometheus-k8s-1
I0616 10:50:01.875990       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-monitoring/prometheus-adapter-6bd77d4dbb-gnjsg
I0616 10:50:02.078301       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-monitoring/thanos-querier-bfcf48dd5-bg7qp
I0616 10:50:26.120935       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-image-registry/image-registry-75f4df699-xl85k
I0616 10:50:51.128254       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: Evicted pod openshift-ingress/router-default-7d5777c977-vqg5f
I0616 10:50:51.128298       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:50:51.142878       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: initiating drain
I0616 10:50:51.142894       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: cordoning
I0616 10:50:51.142915       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: initiating cordon (currently schedulable: false)
I0616 10:50:51.152170       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: cordon succeeded (currently schedulable: false)
E0616 10:50:51.795568       1 drain_controller.go:106] WARNING: ignoring DaemonSet-managed Pods: openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j69mx, openshift-cluster-node-tuning-operator/tuned-p9pfd, openshift-dns/dns-default-9zw5s, openshift-dns/node-resolver-xztr9, openshift-image-registry/node-ca-btfn9, openshift-ingress-canary/ingress-canary-mkmh2, openshift-machine-config-operator/machine-config-daemon-cgv4p, openshift-monitoring/node-exporter-jjtgj, openshift-multus/multus-additional-cni-plugins-7kpsb, openshift-multus/multus-wv866, openshift-multus/network-metrics-daemon-7crbd, openshift-network-diagnostics/network-check-target-96tt5, openshift-sdn/sdn-65bg5
I0616 10:50:51.795591       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: operation successful; applying completion annotation
I0616 10:51:53.166051       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: uncordoning
I0616 10:51:53.166077       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: initiating uncordon (currently schedulable: false)
I0616 10:51:53.205147       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: uncordon succeeded (currently schedulable: true)
I0616 10:51:53.205167       1 drain_controller.go:141] node ip-10-0-255-65.us-east-2.compute.internal: operation successful; applying completion annotation

All regression tests are executed and tracked in https://issues.redhat.com/browse/MCO-272
Comment 6 errata-xmlrpc 2022-08-10 11:12:07 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069