Description of problem: Starting on 4.4, workers nodes are not being registered as nodes. As attach, a complete journalctl log of a hypercube process in a worker, since it starts until it is restarted Jan 31 11:49:10 worker-01.mrnd-packet-44-e629.qe.devcluster.openshift.com hyperkube[3330]: E0131 11:49:10.155752 3330 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:"" }, ObjectMeta:v1.ObjectMeta{Name:"worker-01.mrnd-packet-44-e629.qe.devcluster.openshift.com.15eef651bb9dfbec", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, Creatio nTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string] string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"wo rker-01.mrnd-packet-44-e629.qe.devcluster.openshift.com", UID:"worker-01.mrnd-packet-44-e629.qe.devcluster.openshift.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting k ubelet.", Source:v1.EventSource{Component:"kubelet", Host:"worker-01.mrnd-packet-44-e629.qe.devcluster.openshift.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf8522ed892f5fec, ext:20357715707, loc:(*time. Location)(0x72aa6c0)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf8522ed892f5fec, ext:20357715707, loc:(*time.Location)(0x72aa6c0)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, e xt:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cann ot create resource "events" in API group "" in the namespace "default"' (will not retry!) Jan 31 11:49:10 worker-01.mrnd-packet-44-e629.qe.devcluster.openshift.com hyperkube[3330]: E0131 11:49:10.156188 3330 controller.go:135] failed to ensure node lease exists, will retry in 200ms, error: leases. coordination.k8s.io "worker-01.mrnd-packet-44-e629.qe.devcluster.openshift.com" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-l ease" Version-Release number of selected component (if applicable): 4.4.0-0.nightly-2020-01-31-013819 How reproducible: Steps to Reproduce: 1.Launch baremetal installation 2.Wait for worker to be accesible by SSH 3.Check hyperkube process log Actual results: Workers are not being registered as nodes Expected results: Workers are registered Additional info:
Is this possibly a side effect of https://github.com/openshift/machine-config-operator/pull/1421 Whereby in the past workers could boot using ignition from the bootstrap MCS and request their certificates before bootstrap certificate approval was terminated? Now that worker Ignition isn't available until after bootstrap is complete that process breaks.
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days