Bug 1804239 - add dynamic reloading for CSR signing controllers
Summary: add dynamic reloading for CSR signing controllers
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-controller-manager
Version: 4.4
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.5.0
Assignee: Lukasz Szaszkiewicz
QA Contact: zhou ying
URL:
Whiteboard:
Depends On:
Blocks: 1805791
TreeView+ depends on / blocked
 
Reported: 2020-02-18 14:07 UTC by Maciej Szulik
Modified: 2020-07-13 17:16 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 1805791 (view as bug list)
Environment:
Last Closed: 2020-07-13 17:15:41 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-kube-controller-manager-operator pull 351 0 None closed Bug 1804239: Make csr-signer live-reloaded 2021-01-22 06:41:50 UTC
Github openshift origin pull 24577 0 None closed Bug 1804239: picks dynamic reloading for CSR signing controllers 2021-01-22 06:41:50 UTC
Red Hat Product Errata RHBA-2020:2409 0 None None None 2020-07-13 17:16:10 UTC

Description Maciej Szulik 2020-02-18 14:07:09 UTC
Pick https://github.com/kubernetes/kubernetes/pull/86816 into origin to provide  dynamic reloading for CSR signing controllers to match the kube-apiserver for cert/key pair reloading behavior.

Comment 3 zhou ying 2020-03-09 02:36:59 UTC
Confirmed with payload: 4.5.0-0.nightly-2020-03-06-190457, the issue has fixed:

One terminal:
[root@dhcp-140-138 roottest]# oc delete secrets csr-signer -n openshift-kube-controller-manager-operator
secret "csr-signer" deleted
[root@dhcp-140-138 roottest]# oc get secrets -n openshift-kube-controller-manager-operator
NAME                                               TYPE                                  DATA   AGE
builder-dockercfg-fqm9c                            kubernetes.io/dockercfg               1      67m
builder-token-vcgqq                                kubernetes.io/service-account-token   4      68m
builder-token-vnxqt                                kubernetes.io/service-account-token   4      68m
csr-signer                                         kubernetes.io/tls                     2      15s


Another terminal:
[root@dhcp-140-138 roottest]# oc logs -f po/kube-controller-manager-ip-10-0-135-80.us-east-2.compute.internal
Copying system trust bundle
I0309 01:29:59.252226       1 feature_gate.go:244] feature gates: &{map[RotateKubeletServerCertificate:true]}
I0309 01:29:59.252359       1 feature_gate.go:244] feature gates: &{map[RotateKubeletServerCertificate:true SupportPodPidsLimit:true]}
...

I0309 02:31:59.866375       1 tlsconfig.go:179] loaded client CA [0/"client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt,request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt"]: "admin-kubeconfig-signer" [] issuer="<self>" (2020-03-09 01:01:23 +0000 UTC to 2030-03-07 01:01:23 +0000 UTC (now=2020-03-09 02:31:59.86635187 +0000 UTC))
I0309 02:31:59.866409       1 tlsconfig.go:179] loaded client CA [1/"client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt,request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt"]: "kube-csr-signer_@1583716679" [] issuer="kubelet-signer" (2020-03-09 01:17:59 +0000 UTC to 2020-03-10 01:01:37 +0000 UTC (now=2020-03-09 02:31:59.866398885 +0000 UTC))
I0309 02:31:59.866426       1 tlsconfig.go:179] loaded client CA [2/"client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt,request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt"]: "kubelet-signer" [] issuer="<self>" (2020-03-09 01:01:37 +0000 UTC to 2020-03-10 01:01:37 +0000 UTC (now=2020-03-09 02:31:59.866418712 +0000 UTC))
I0309 02:31:59.866443       1 tlsconfig.go:179] loaded client CA [3/"client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt,request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt"]: "kube-csr-signer_@1583721062" [] issuer="kubelet-signer" (2020-03-09 02:31:01 +0000 UTC to 2020-03-10 01:01:37 +0000 UTC (now=2020-03-09 02:31:59.866435024 +0000 UTC))
I0309 02:31:59.866461       1 tlsconfig.go:179] loaded client CA [4/"client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt,request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt"]: "kube-apiserver-to-kubelet-signer" [] issuer="<self>" (2020-03-09 01:01:38 +0000 UTC to 2021-03-09 01:01:38 +0000 UTC (now=2020-03-09 02:31:59.866453375 +0000 UTC))
I0309 02:31:59.866477       1 tlsconfig.go:179] loaded client CA [5/"client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt,request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt"]: "kube-control-plane-signer" [] issuer="<self>" (2020-03-09 01:01:38 +0000 UTC to 2021-03-09 01:01:38 +0000 UTC (now=2020-03-09 02:31:59.866469916 +0000 UTC))
I0309 02:31:59.866493       1 tlsconfig.go:179] loaded client CA [6/"client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt,request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt"]: "kubelet-bootstrap-kubeconfig-signer" [] issuer="<self>" (2020-03-09 01:01:26 +0000 UTC to 2030-03-07 01:01:26 +0000 UTC (now=2020-03-09 02:31:59.866485372 +0000 UTC))
I0309 02:31:59.866510       1 tlsconfig.go:179] loaded client CA [7/"client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt,request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt"]: "aggregator-signer" [] issuer="<self>" (2020-03-09 01:01:33 +0000 UTC to 2020-03-10 01:01:33 +0000 UTC (now=2020-03-09 02:31:59.866501495 +0000 UTC))
I0309 02:31:59.866782       1 tlsconfig.go:201] loaded serving cert ["serving-cert::/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.crt::/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.key"]: "kube-controller-manager.openshift-kube-controller-manager.svc" [serving] validServingFor=[kube-controller-manager.openshift-kube-controller-manager.svc,kube-controller-manager.openshift-kube-controller-manager.svc.cluster.local] issuer="openshift-service-serving-signer@1583716682" (2020-03-09 01:18:15 +0000 UTC to 2022-03-09 01:18:16 +0000 UTC (now=2020-03-09 02:31:59.866767461 +0000 UTC))

Comment 5 errata-xmlrpc 2020-07-13 17:15:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409


Note You need to log in before you can comment on or make changes to this bug.