Bug 1853278

Summary: v4-0-config-system-console-config configmap is attempted to be removed for > 10k times during aws-serial
Product: OpenShift Container Platform Reporter: Vadim Rutkovsky <vrutkovs>
Component: apiserver-authAssignee: Venkata Siva Teja Areti <vareti>
Status: CLOSED ERRATA QA Contact: pmali
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.6CC: aos-bugs, mfojtik, pasik, vareti, wking
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:11:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vadim Rutkovsky 2020-07-02 10:45:26 UTC 
AWS serial test: https://deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gcs/origin-ci-test/logs/release-openshift-origin-installer-e2e-aws-serial-4.6/1278444169194377216

`topk(20, apiserver_request_total{verb=~"(UPDATE|PATCH|DELETE)"})` metric shows that >10k requests to delete configmaps. Audit logs show a lot of messages like:

```
{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"152eac88-dbd8-4053-817e-74f603aa3e77","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/openshift-authentication/configmaps/v4-0-config-system-console-config","verb":"delete","user":{"│
username":"system:serviceaccount:openshift-authentication-operator:authentication-operator","uid":"f486fb16-f3de-427c-ad88-16b578ac9ae1","groups":["system:serviceaccounts","system:serviceaccounts:openshift-authentication-operator","system:authenticated"]},"sourceIPs":["1│
0.128.0.5"],"userAgent":"authentication-operator/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"configmaps","namespace":"openshift-authentication","name":"v4-0-config-system-console-config","apiVersion":"v1"},"responseStatus":{"metadata":{},"status":"F│
ailure","reason":"NotFound","code":404},"requestReceivedTimestamp":"2020-07-01T22:31:24.596203Z","stageTimestamp":"2020-07-01T22:31:24.599746Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"sys│
tem:openshift:operator:authentication\" of ClusterRole \"cluster-admin\" to ServiceAccount \"authentication-operator/openshift-authentication-operator\""}}
```
Comment 1 Venkata Siva Teja Areti 2020-07-09 19:18:20 UTC 
changes are complete. can be merged once I finish testing the PR manually.
Comment 6 errata-xmlrpc 2020-10-27 16:11:47 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196