Hide Forgot
Created attachment 1536541 [details] must-gather-kube-controller-manager-operator-20190219.tar.gz openshift-controller-manager-operator/openshift-controller-manager-operator scrape fails with: Get https://10.130.0.14:8443/metrics: x509: certificate is valid for localhost, not metrics.openshift-controller-manager-operator.svc openshift-kube-controller-manager-operator/kube-controller-manager-operator scrape fails with: Get https://10.129.0.7:8443/metrics: x509: certificate is valid for localhost, not metrics.openshift-kube-controller-manager-operator.svc See attached must-gather.
Michal: looks like reactor may not be working properly which is interesting because CI is showing this working on every CI run. https://gcsweb-ci.svc.ci.openshift.org/gcs/origin-ci-test/pr-logs/pull/openshift_cluster-kube-apiserver-operator/268/pull-ci-openshift-cluster-kube-apiserver-operator-master-e2e-aws/1229/artifacts/e2e-aws/pods/ for example. Seth: anything unusual about the kubelet configuration there? Is there a way for us to check to see if the kubelet is properly providing the new files on disk?
got a candidate fix https://github.com/openshift/cluster-kube-controller-manager-operator/pull/169
*** Bug 1679922 has been marked as a duplicate of this bug. ***
I am not seeing this any more on a cluster that has been up for 2.5hrs (post rotation)
There were numerous cert rotation fixes since this was tested and we also extended the rotation period.
Verified in 4.0.0-0.nightly-2019-03-28-030453 per comment 5: $ prometheus_route=$(oc -n openshift-monitoring get route | grep prometheus-k8s | awk '{print $2}') $ curl -k -H "Authorization: Bearer $(oc sa get-token prometheus-k8s -n openshift-monitoring)" https://${prometheus_route}/targets | grep -i x509 Didn't see error given the env is 7+ hours.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758