Bug 1821690 - When changed CSR signer hot loops on update
Summary: When changed CSR signer hot loops on update
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-controller-manager
Version: 4.4
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: 4.4.0
Assignee: Tomáš Nožička
QA Contact: zhou ying
URL:
Whiteboard:
Depends On: 1821689
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-07 12:24 UTC by Tomáš Nožička
Modified: 2020-05-04 11:49 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of: 1821689
Environment:
Last Closed: 2020-05-04 11:48:34 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-kube-controller-manager-operator pull 391 0 None closed [release-4.4] Bug 1821690: Fix csr-signer update hotloop 2020-05-04 13:04:14 UTC
Red Hat Product Errata RHBA-2020:0581 0 None None None 2020-05-04 11:49:00 UTC

Description Tomáš Nožička 2020-04-07 12:24:59 UTC 
+++ This bug was initially created as a clone of Bug #1821689 +++

CKCMO keeps updating the csr-signer although no change is needed.

I0407 10:54:35.081638  298746 csrcontroller.go:167] CSRController sync done
I0407 10:54:35.081699  298746 csrcontroller.go:128] Starting CSRController sync
I0407 10:54:35.081791  298746 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"840796e5-22d8-479c-b539-05c11be3f958", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretUpdated' Updated Secret/csr-signer -n openshift-kube-controller-manager because it changed
I0407 10:54:46.046043  298746 core.go:281] Secret openshift-kube-controller-manager/csr-signer changes: {"type":null}
I0407 10:54:46.230221  298746 csrcontroller.go:164] Refreshed CSRSigner.
I0407 10:54:46.230237  298746 csrcontroller.go:167] CSRController sync done
I0407 10:54:46.230250  298746 csrcontroller.go:128] Starting CSRController sync
I0407 10:54:46.230249  298746 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"840796e5-22d8-479c-b539-05c11be3f958", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretUpdated' Updated Secret/csr-signer -n openshift-kube-controller-manager because it changed
I0407 10:54:57.144311  298746 core.go:281] Secret openshift-kube-controller-manager/csr-signer changes: {"type":null}
I0407 10:54:57.323193  298746 csrcontroller.go:164] Refreshed CSRSigner.
I0407 10:54:57.323220  298746 csrcontroller.go:167] CSRController sync done
Comment 3 zhou ying 2020-04-13 06:29:19 UTC 
Confirmed with payload :4.4.0-0.nightly-2020-04-09-220855, the issue has fixed:
1) one terminal delete secrets :

`oc delete secrets csr-signer -n  openshift-kube-controller-manager`


2) on second terminal check logs from CKCMO:
oc logs -f po/kube-controller-manager-operator-5f47c4d756-l8p8p  -n openshift-kube-controller-manager-operator

I0413 01:57:25.894786       1 event.go:281] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"22273bd2-2b95-426d-93bc-06f3191ab756", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "NodeControllerDegraded: All master nodes are ready\nStaticPodsDegraded: nodes/ip-10-0-163-185.us-east-2.compute.internal pods/kube-controller-manager-ip-10-0-163-185.us-east-2.compute.internal container=\"cluster-policy-controller\" is not ready" to "NodeControllerDegraded: All master nodes are ready"





I0413 06:26:54.336426       1 event.go:281] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"22273bd2-2b95-426d-93bc-06f3191ab756", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretCreated' Created Secret/csr-signer -n openshift-kube-controller-manager because it was missing
Comment 5 errata-xmlrpc 2020-05-04 11:48:34 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581
Note You need to log in before you can comment on or make changes to this bug.