Bug 1821690

Summary:	When changed CSR signer hot loops on update
Product:	OpenShift Container Platform	Reporter:	Tomáš Nožička <tnozicka>
Component:	kube-controller-manager	Assignee:	Tomáš Nožička <tnozicka>
Status:	CLOSED ERRATA	QA Contact:	zhou ying <yinzhou>
Severity:	medium	Docs Contact:
Priority:	high
Version:	4.4	CC:	aos-bugs, maszulik, mfojtik, yinzhou
Target Milestone:	---
Target Release:	4.4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:	1821689	Environment:
Last Closed:	2020-05-04 11:48:34 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1821689
Bug Blocks:

Description Tomáš Nožička 2020-04-07 12:24:59 UTC

+++ This bug was initially created as a clone of Bug #1821689 +++

CKCMO keeps updating the csr-signer although no change is needed.

I0407 10:54:35.081638  298746 csrcontroller.go:167] CSRController sync done
I0407 10:54:35.081699  298746 csrcontroller.go:128] Starting CSRController sync
I0407 10:54:35.081791  298746 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"840796e5-22d8-479c-b539-05c11be3f958", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretUpdated' Updated Secret/csr-signer -n openshift-kube-controller-manager because it changed
I0407 10:54:46.046043  298746 core.go:281] Secret openshift-kube-controller-manager/csr-signer changes: {"type":null}
I0407 10:54:46.230221  298746 csrcontroller.go:164] Refreshed CSRSigner.
I0407 10:54:46.230237  298746 csrcontroller.go:167] CSRController sync done
I0407 10:54:46.230250  298746 csrcontroller.go:128] Starting CSRController sync
I0407 10:54:46.230249  298746 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"840796e5-22d8-479c-b539-05c11be3f958", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretUpdated' Updated Secret/csr-signer -n openshift-kube-controller-manager because it changed
I0407 10:54:57.144311  298746 core.go:281] Secret openshift-kube-controller-manager/csr-signer changes: {"type":null}
I0407 10:54:57.323193  298746 csrcontroller.go:164] Refreshed CSRSigner.
I0407 10:54:57.323220  298746 csrcontroller.go:167] CSRController sync done

Comment 3 zhou ying 2020-04-13 06:29:19 UTC

Confirmed with payload :4.4.0-0.nightly-2020-04-09-220855, the issue has fixed:
1) one terminal delete secrets :

`oc delete secrets csr-signer -n  openshift-kube-controller-manager`


2) on second terminal check logs from CKCMO:
oc logs -f po/kube-controller-manager-operator-5f47c4d756-l8p8p  -n openshift-kube-controller-manager-operator

I0413 01:57:25.894786       1 event.go:281] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"22273bd2-2b95-426d-93bc-06f3191ab756", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "NodeControllerDegraded: All master nodes are ready\nStaticPodsDegraded: nodes/ip-10-0-163-185.us-east-2.compute.internal pods/kube-controller-manager-ip-10-0-163-185.us-east-2.compute.internal container=\"cluster-policy-controller\" is not ready" to "NodeControllerDegraded: All master nodes are ready"





I0413 06:26:54.336426       1 event.go:281] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"22273bd2-2b95-426d-93bc-06f3191ab756", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretCreated' Created Secret/csr-signer -n openshift-kube-controller-manager because it was missing

Comment 5 errata-xmlrpc 2020-05-04 11:48:34 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581