Bug 1867494 - webhook authenticator does not have the master name replaced
Summary: webhook authenticator does not have the master name replaced
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-controller-manager
Version: 4.6
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.6.0
Assignee: Tomáš Nožička
QA Contact: RamaKasturi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-08-10 07:39 UTC by RamaKasturi
Modified: 2021-03-04 20:10 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-27 16:26:36 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-kube-controller-manager-operator pull 441 0 None closed Bug 1867494: Explicitly use internal LB for KCM and CPC 2021-02-18 14:14:16 UTC
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 16:26:48 UTC

Description RamaKasturi 2020-08-10 07:39:50 UTC
When verifying bug https://bugzilla.redhat.com/show_bug.cgi?id=1842002 hit error below when kube-apiserver is down.

E0807 13:48:12.608278       1 webhook.go:111] Failed to make webhook authenticator request: Post "https://localhost:6443/apis/authentication.k8s.io/v1/tokenreviews": dial tcp [::1]:6443: connect: connection refused
E0807 13:48:12.608328       1 authentication.go:53] Unable to authenticate the request due to an error: [invalid bearer token, Post "https://localhost:6443/apis/authentication.k8s.io/v1/tokenreviews": dial tcp [::1]:6443: connect: connection refused]

Version-Release number of selected component (if applicable):
Client Version: 4.6.0-202008031851.p0-a695d74
Server Version: 4.6.0-0.nightly-2020-08-07-042746

How Reproducible:
Always

Steps to Reproduce:
1) Install 4.6 cluster
2) on the same master, bring down the local kube-apiserver pod and see if the local KCM fails or keeps working use a master with KCM leader
3) To bring down local kube-apiserver, ssh to the master , mv /etc/kubernetes/manifests/kube-apiserver-pod.yaml /home/kube-apiserver-pod.yaml
4) Terminate it gracefully by running the command "pid=$(ps aux | grep " kube-apiserver " | grep -v grep  | awk 'NR==1 {print $2}');
kill $pid"
5) Now keep checking KCM and api-server logs

Actual Results:
=================
Below errors appear in the KCM logs:
E0807 13:48:12.608278       1 webhook.go:111] Failed to make webhook authenticator request: Post "https://localhost:6443/apis/authentication.k8s.io/v1/tokenreviews": dial tcp [::1]:6443: connect: connection refused
E0807 13:48:12.608328       1 authentication.go:53] Unable to authenticate the request due to an error: [invalid bearer token, Post "https://localhost:6443/apis/authentication.k8s.io/v1/tokenreviews": dial tcp [::1]:6443: connect: connection refused]

Expected Results:
======================
Should not see any errors as above.

Additional Info:
https://coreos.slack.com/archives/CKJR6200N/p1596805775023000 - More info in this thread

Comment 1 RamaKasturi 2020-08-10 07:40:47 UTC
Raising this bug in kcm as i am not sure which component to assign  for , tomas could you please help reassign, thanks !!

Comment 2 Tomáš Nožička 2020-08-10 08:12:47 UTC
kcm is the correct choice ;)

Comment 6 RamaKasturi 2020-09-11 06:50:24 UTC
Verified bug with the payload below and i do not see any messages as reported. So marking the bug verified.

[ramakasturinarra@dhcp35-60 ~]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.nightly-2020-09-10-195619   True        False         49m     Cluster version is 4.6.0-0.nightly-2020-09-10-195619

Followed the same steps as the steps used to reproduce the issue.

Comment 9 errata-xmlrpc 2020-10-27 16:26:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196


Note You need to log in before you can comment on or make changes to this bug.