Bug 1918375

Summary: [calico] rbac-proxy container in kube-proxy fails to create tokenreviews
Product: OpenShift Container Platform Reporter: Cesar Wong <cewong>
Component: NetworkingAssignee: Alexander Constantinescu <aconstan>
Networking sub component: openshift-sdn QA Contact: zhaozhanqi <zzhao>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: aconstan, wking
Version: 4.7   
Target Milestone: ---   
Target Release: 4.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-24 15:55:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1967972    

Description Cesar Wong 2021-01-20 15:10:14 UTC 
Description of problem:
When using Calico as the network provider, the rbac-proxy container in kube-proxy pods fails to create token reviews.

Version-Release number of selected component (if applicable):
4.7

Steps to Reproduce:
1. Install OpenShift with Calico as the network provider
2. Inspect log of the rbac-proxy container in kube-proxy pod(s)

Actual results:
The following error appears in the log:
E0119 18:57:17.512404       1 proxy.go:73] Unable to authenticate the request due to an error: tokenreviews.authentication.k8s.io is forbidden: User "system:serviceaccount:openshift-kube-proxy:openshift-kube-proxy" cannot create resource "tokenreviews" in API group "authentication.k8s.io" at the cluster scope

Expected results:
No errors appear in the log

Additional info:
e2e test: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/15053/rehearse-15053-release-openshift-origin-installer-e2e-aws-calico-4.7/1351588780695359488
rbac-proxy log: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_release/15053/rehearse-15053-release-openshift-origin-installer-e2e-aws-calico-4.7/1351588780695359488/artifacts/e2e-aws/pods/openshift-kube-proxy_openshift-kube-proxy-56vl7_kube-rbac-proxy.log
Comment 1 zhaozhanqi 2021-02-08 07:38:48 UTC 
verified the fixed PR with cluster-bot deploy one cluster by 'launch openshift/cluster-network-operator#960 aws'

and then deploy cluster with Calico plugin, no found above error:


# oc get pod -n openshift-kube-proxy
NAME                         READY   STATUS    RESTARTS   AGE
openshift-kube-proxy-4fmsd   2/2     Running   0          4h22m
openshift-kube-proxy-6gpzn   2/2     Running   0          4h22m
openshift-kube-proxy-glsl4   2/2     Running   0          4h24m
openshift-kube-proxy-kc2g7   2/2     Running   0          4h32m
openshift-kube-proxy-nrxsm   2/2     Running   0          4h32m
openshift-kube-proxy-w68gx   2/2     Running   0          4h32m
# oc logs openshift-kube-proxy-4fmsd -n openshift-kube-proxy
error: a container name must be specified for pod openshift-kube-proxy-4fmsd, choose one of: [kube-proxy kube-rbac-proxy]

[root@preserve-zzhao 207]# oc logs openshift-kube-proxy-4fmsd -n openshift-kube-proxy -c kube-rbac-proxy
I0208 03:12:34.271077       1 main.go:190] Valid token audiences: 
I0208 03:12:34.271170       1 main.go:278] Reading certificate files
I0208 03:12:34.271399       1 main.go:311] Starting TCP socket on :9102
I0208 03:12:34.271713       1 main.go:318] Listening securely on :9102
Comment 3 zhaozhanqi 2021-02-09 03:29:02 UTC 
Move to verified according to comment 1
Comment 6 errata-xmlrpc 2021-02-24 15:55:11 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633