Bug 1945247 - WMCO patch pub-key-hash annotation to Linux node
Summary: WMCO patch pub-key-hash annotation to Linux node
Keywords:
Status: CLOSED DUPLICATE of bug 1930791
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Windows Containers
Version: 4.7
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: 4.7.z
Assignee: Team Windows Containers
QA Contact: gaoshang
URL:
Whiteboard:
Depends On: 1930791 1945248
Blocks: 1934281
TreeView+ depends on / blocked
 
Reported: 2021-03-31 14:22 UTC by Sebastian Soto
Modified: 2022-04-05 19:44 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1930791
Environment:
Last Closed: 2021-03-31 14:24:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Sebastian Soto 2021-03-31 14:22:23 UTC
+++ This bug was initially created as a clone of Bug #1930791 +++

Description of problem:
After replaced cloud-private-key secret with a new private key, found WMCO is patching pub-key-hash annotation to Linux node, including both master/work node.

# oc describe node sgao-a-bnxdr-master-0 | grep -A 20 Annotation
Annotations:        k8s.ovn.org/hybrid-overlay-distributed-router-gateway-mac: 0a:58:0a:80:00:03
                    k8s.ovn.org/l3-gateway-config:
                      {"default":{"mode":"shared","interface-id":"br-ex_sgao-a-bnxdr-master-0","mac-address":"00:0d:3a:42:6b:39","ip-addresses":["10.0.0.8/19"],...
                    k8s.ovn.org/node-chassis-id: 32358f7c-bbf9-422a-893c-1d74482d5a3a
                    k8s.ovn.org/node-local-nat-ip: {"default":["169.254.8.64"]}
                    k8s.ovn.org/node-mgmt-port-mac-address: 4e:b8:d0:ce:cc:27
                    k8s.ovn.org/node-primary-ifaddr: {"ipv4":"10.0.0.8/19"}
                    k8s.ovn.org/node-subnets: {"default":"10.128.0.0/23"}
                    machine.openshift.io/machine: openshift-machine-api/sgao-a-bnxdr-master-0
                    machineconfiguration.openshift.io/currentConfig: rendered-master-4effb05b75d538407efd2b99f6c31c06
                    machineconfiguration.openshift.io/desiredConfig: rendered-master-4effb05b75d538407efd2b99f6c31c06
                    machineconfiguration.openshift.io/reason: 
                    machineconfiguration.openshift.io/state: Done
                    volumes.kubernetes.io/controller-managed-attach-detach: true
---------------------------------------------------------------------------------------------------
                    windowsmachineconfig.openshift.io/pub-key-hash: 
---------------------------------------------------------------------------------------------------
CreationTimestamp:  Thu, 18 Feb 2021 22:07:37 -0500
Taints:             node-role.kubernetes.io/master:NoSchedule
Unschedulable:      false
Lease:
  HolderIdentity:  sgao-a-bnxdr-master-0
  AcquireTime:     <unset>

Version-Release number of selected component (if applicable):
OCP version: 4.7.0-0.nightly-2021-02-18-110409
WMCO commit: f1f40153af071e9778d3ada5ec6dc93e9adfaa9d

How reproducible:
Always

Steps to Reproduce:
1, WMCO installed and Windows nodes has been bootstrapped
# oc get nodes -l kubernetes.io/os=windows
NAME            STATUS   ROLES    AGE   VERSION
windows-k9t5g   Ready    worker   98m   v1.20.0-1030+cac2421340a449

2, Delete secret cloud-private-key and create with another private key

# oc delete secret cloud-private-key
secret "cloud-private-key" deleted

# oc create secret generic cloud-private-key --from-file=private-key.pem=/root/.ssh/openshift-dev.pem 
secret/cloud-private-key created

3, Check WMCO log, found WMCO is patching pub-key-hash annotation to Linux node, and seems like there's duplicate message in log
"DEBUG	secret_controller	patched node object"
"INFO	secret_controller	updating secret"

# oc logs -f deployment.apps/windows-machine-config-operator
...
2021-02-19T13:52:08.366Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "secret_controller", "request": "openshift-windows-machine-config-operator/cloud-private-key"}
2021-02-19T13:52:08.370Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "secret_controller", "request": "openshift-windows-machine-config-operator/cloud-private-key"}
2021-02-19T13:52:19.235Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "secret_controller", "request": "openshift-windows-machine-config-operator/cloud-private-key"}
2021-02-19T13:52:19.242Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "secret_controller", "request": "openshift-windows-machine-config-operator/cloud-private-key"}
2021-02-19T13:53:56.589Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "secret_controller", "request": "openshift-windows-machine-config-operator/cloud-private-key"}
2021-02-19T13:54:01.223Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "secret_controller", "request": "openshift-windows-machine-config-operator/cloud-private-key"}
2021-02-19T13:54:01.228Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "secret_controller", "request": "openshift-windows-machine-config-operator/cloud-private-key"}
2021-02-19T13:54:16.637Z	DEBUG	secret_controller	patched node object	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "node": "sgao-a-bnxdr-master-0", "patch": "[{\"op\":\"add\",\"path\":\"/metadata/annotations/windowsmachineconfig.openshift.io~1pub-key-hash\",\"value\":\"\"}]"}
2021-02-19T13:54:16.654Z	DEBUG	secret_controller	patched node object	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "node": "sgao-a-bnxdr-master-1", "patch": "[{\"op\":\"add\",\"path\":\"/metadata/annotations/windowsmachineconfig.openshift.io~1pub-key-hash\",\"value\":\"\"}]"}
2021-02-19T13:54:16.667Z	DEBUG	secret_controller	patched node object	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "node": "sgao-a-bnxdr-master-2", "patch": "[{\"op\":\"add\",\"path\":\"/metadata/annotations/windowsmachineconfig.openshift.io~1pub-key-hash\",\"value\":\"\"}]"}
2021-02-19T13:54:16.699Z	DEBUG	secret_controller	patched node object	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "node": "sgao-a-bnxdr-worker-centralus1-cplbh", "patch": "[{\"op\":\"add\",\"path\":\"/metadata/annotations/windowsmachineconfig.openshift.io~1pub-key-hash\",\"value\":\"\"}]"}
2021-02-19T13:54:16.711Z	DEBUG	secret_controller	patched node object	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "node": "sgao-a-bnxdr-worker-centralus2-rtqm8", "patch": "[{\"op\":\"add\",\"path\":\"/metadata/annotations/windowsmachineconfig.openshift.io~1pub-key-hash\",\"value\":\"\"}]"}
2021-02-19T13:54:16.739Z	DEBUG	secret_controller	patched node object	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "node": "sgao-a-bnxdr-worker-centralus3-rkbdd", "patch": "[{\"op\":\"add\",\"path\":\"/metadata/annotations/windowsmachineconfig.openshift.io~1pub-key-hash\",\"value\":\"\"}]"}
2021-02-19T13:54:16.750Z	DEBUG	secret_controller	patched node object	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "node": "windows-k9t5g", "patch": "[{\"op\":\"add\",\"path\":\"/metadata/annotations/windowsmachineconfig.openshift.io~1pub-key-hash\",\"value\":\"\"}]"}
2021-02-19T13:54:16.751Z	INFO	secret_controller	updating secret	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "name": "windows-user-data"}
2021-02-19T13:54:16.752Z	DEBUG	windowsmachine-controller	reconciling	{"namespace": "openshift-machine-api", "name": "windows-k9t5g"}
2021-02-19T13:54:16.765Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "secret_controller", "request": "openshift-windows-machine-config-operator/cloud-private-key"}
2021-02-19T13:54:16.772Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "secret_controller", "request": "openshift-windows-machine-config-operator/cloud-private-key"}
2021-02-19T13:54:16.773Z	INFO	windowsmachine-controller	deleting machine	{"name": "windows-k9t5g"}
2021-02-19T13:54:16.803Z	INFO	windowsmachine-controller	machine has been remediated by deletion	{"name": "windows-k9t5g"}
2021-02-19T13:54:16.803Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "windowsmachine-controller", "request": "openshift-machine-api/windows-k9t5g"}
2021-02-19T13:54:16.803Z	DEBUG	windowsmachine-controller	reconciling	{"namespace": "openshift-machine-api", "name": "windows-k9t5g"}
2021-02-19T13:54:16.803Z	DEBUG	controller-runtime.manager.events	Normal	{"object": {"kind":"Machine","namespace":"openshift-machine-api","name":"windows-k9t5g","uid":"63a6d9aa-715e-4fd2-9baf-2c9a0cb2beb9","apiVersion":"machine.openshift.io/v1beta1","resourceVersion":"185934"}, "reason": "MachineDeleted", "message": "Machine windows-k9t5g has been remediated by deleting the Machine object"}
2021-02-19T13:54:16.816Z	INFO	windowsmachine-controller	deleting machine	{"name": "windows-k9t5g"}
2021-02-19T13:54:16.828Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "windowsmachine-controller", "request": "openshift-machine-api/windows-k9t5g"}
2021-02-19T13:54:16.828Z	DEBUG	windowsmachine-controller	reconciling	{"namespace": "openshift-machine-api", "name": "windows-k9t5g"}
2021-02-19T13:54:16.837Z	DEBUG	windowsmachine-controller	machine not provisioned	{"phase": "Deleting"}
2021-02-19T13:54:16.855Z	INFO	metrics	Prometheus configured	{"endpoints": "windows-machine-config-operator-metrics", "port": 9182, "name": "metrics"}
2021-02-19T13:54:16.856Z	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "windowsmachine-controller", "request": "openshift-machine-api/windows-k9t5g"}
2021-02-19T13:54:16.856Z	DEBUG	windowsmachine-controller	reconciling	{"namespace": "openshift-machine-api", "name": "windows-tzgf5"}


Actual results:
WMCO is trying to patch annotation to Linux node
 
Expected results:
WMCO should not patch annotation to Linux node

Additional info:

--- Additional comment from gaoshang on 2021-03-09 03:05:40 UTC ---

This bug has been verified on OCP 4.8.0-0.nightly-2021-03-06-055252 and passed, thanks.

Version:
WMCO commit: b0dd4992bf0fb4cdff2b433f6a9ece3960447842
OCP version: 4.8.0-0.nightly-2021-03-06-055252

Steps:

After repeat steps in bug, found WMCO is not patching pub-key-hash annotation to Linux node anymore.

# oc logs -f deployment.apps/windows-machine-config-operator
...

2021-03-09T02:41:26.532Z	DEBUG	secret_controller	patched node object	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "node": "ip-10-0-145-235.us-east-2.compute.internal", "patch": "[{\"op\":\"add\",\"path\":\"/metadata/annotations/windowsmachineconfig.openshift.io~1pub-key-hash\",\"value\":\"\"}]"}
2021-03-09T02:41:26.532Z	DEBUG	windowsmachine-controller	reconciling	{"namespace": "openshift-machine-api", "name": "sgao-w886x-windows-worker-us-east-2a-9srsq"}
2021-03-09T02:41:26.540Z	DEBUG	secret_controller	patched node object	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "node": "ip-10-0-147-105.us-east-2.compute.internal", "patch": "[{\"op\":\"add\",\"path\":\"/metadata/annotations/windowsmachineconfig.openshift.io~1pub-key-hash\",\"value\":\"\"}]"}
2021-03-09T02:41:26.540Z	INFO	secret_controller	updating secret	{"namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "namespace": "openshift-windows-machine-config-operator", "name": "cloud-private-key", "name": "windows-user-data"}

--- Additional comment from gaoshang on 2021-03-09 06:55:34 UTC ---

Also verified annotations applied by previous versions of WMCO will be removed.

Steps:

Version:
WMCO commit: b0dd4992bf0fb4cdff2b433f6a9ece3960447842
OCP version: 4.8.0-0.nightly-2021-03-06-055252

1, Patch pub-key-hash annotation to Linux node

# oc patch node ip-10-0-130-66.us-east-2.compute.internal -p '{"metadata":{"annotations":{"windowsmachineconfig.openshift.io/pub-key-hash":""}}}'
node/ip-10-0-130-66.us-east-2.compute.internal patched

# oc describe node ip-10-0-130-66.us-east-2.compute.internal | grep -A 20 Annotation
Annotations:        csi.volume.kubernetes.io/nodeid: {"ebs.csi.aws.com":"i-007263ab6e26bb9eb"}
                    k8s.ovn.org/hybrid-overlay-distributed-router-gateway-mac: 0a:58:0a:80:02:03
                    k8s.ovn.org/l3-gateway-config:
                      {"default":{"mode":"shared","interface-id":"br-ex_ip-10-0-130-66.us-east-2.compute.internal","mac-address":"02:a2:e6:76:68:36","ip-address...
                    k8s.ovn.org/node-chassis-id: b8cd6f57-7606-41bf-97f7-a31f55e1aa5b
                    k8s.ovn.org/node-local-nat-ip: {"default":["169.254.0.168"]}
                    k8s.ovn.org/node-mgmt-port-mac-address: 2a:b1:8c:ff:42:2c
                    k8s.ovn.org/node-primary-ifaddr: {"ipv4":"10.0.130.66/19"}
                    k8s.ovn.org/node-subnets: {"default":"10.128.2.0/23"}
                    machine.openshift.io/machine: openshift-machine-api/sgao-w886x-worker-us-east-2a-d4zm9
                    machineconfiguration.openshift.io/currentConfig: rendered-worker-e19e58fb3bc1d744149c30405a03bd64
                    machineconfiguration.openshift.io/desiredConfig: rendered-worker-e19e58fb3bc1d744149c30405a03bd64
                    machineconfiguration.openshift.io/reason: 
                    machineconfiguration.openshift.io/state: Done
                    volumes.kubernetes.io/controller-managed-attach-detach: true
---------------------------------------------------------------------------------------------------
                    windowsmachineconfig.openshift.io/pub-key-hash: 
---------------------------------------------------------------------------------------------------
CreationTimestamp:  Sun, 07 Mar 2021 23:37:19 -0500
Taints:             <none>
Unschedulable:      false
Lease:

2, Restart WMCO by deleting pod, check after WMCO restarted, pub-key-hash annotation on Linux node will be removed.

# oc delete pod/windows-machine-config-operator-865cc5d78c-9ghqw
pod "windows-machine-config-operator-865cc5d78c-9ghqw" deleted

# oc describe node ip-10-0-130-66.us-east-2.compute.internal | grep -A 20 Annotation
Annotations:        csi.volume.kubernetes.io/nodeid: {"ebs.csi.aws.com":"i-007263ab6e26bb9eb"}
                    k8s.ovn.org/hybrid-overlay-distributed-router-gateway-mac: 0a:58:0a:80:02:03
                    k8s.ovn.org/l3-gateway-config:
                      {"default":{"mode":"shared","interface-id":"br-ex_ip-10-0-130-66.us-east-2.compute.internal","mac-address":"02:a2:e6:76:68:36","ip-address...
                    k8s.ovn.org/node-chassis-id: b8cd6f57-7606-41bf-97f7-a31f55e1aa5b
                    k8s.ovn.org/node-local-nat-ip: {"default":["169.254.0.168"]}
                    k8s.ovn.org/node-mgmt-port-mac-address: 2a:b1:8c:ff:42:2c
                    k8s.ovn.org/node-primary-ifaddr: {"ipv4":"10.0.130.66/19"}
                    k8s.ovn.org/node-subnets: {"default":"10.128.2.0/23"}
                    machine.openshift.io/machine: openshift-machine-api/sgao-w886x-worker-us-east-2a-d4zm9
                    machineconfiguration.openshift.io/currentConfig: rendered-worker-e19e58fb3bc1d744149c30405a03bd64
                    machineconfiguration.openshift.io/desiredConfig: rendered-worker-e19e58fb3bc1d744149c30405a03bd64
                    machineconfiguration.openshift.io/reason: 
                    machineconfiguration.openshift.io/state: Done
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Sun, 07 Mar 2021 23:37:19 -0500
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  ip-10-0-130-66.us-east-2.compute.internal
  AcquireTime:     <unset>

Comment 1 Sebastian Soto 2021-03-31 14:24:54 UTC

*** This bug has been marked as a duplicate of bug 1930791 ***


Note You need to log in before you can comment on or make changes to this bug.