Bug 2032325

Summary: api request counts for current hour are incorrect
Product: OpenShift Container Platform Reporter: Luis Sanchez <sanchezl>
Component: kube-apiserverAssignee: Luis Sanchez <sanchezl>
Status: CLOSED ERRATA QA Contact: Ke Wang <kewang>
Severity: high Docs Contact:
Priority: high    
Version: 4.8CC: andbartl, anmistry, aos-bugs, kewang, mfojtik, oarribas, openshift-bugzilla-robot, wking, xxia
Target Milestone: ---Keywords: FastFix
Target Release: 4.8.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: 2030697 Environment:
Last Closed: 2022-01-05 16:12:09 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2030697    
Bug Blocks:    

Comment 3 Ke Wang 2021-12-15 07:56:07 UTC
1. Setup one 4.8 cluster, use a deprecated API,
$ oc get clusterversion
NAME      VERSION                        AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.ci-2021-12-15-032742   True        False         38m     Cluster version is 4.8.0-0.ci-2021-12-15-032742

2. Get current apirequestcount of ingresses.v1beta1.extensions:
$ oc get apirequestcount/ingresses.v1beta1.extension
NAME                           REMOVEDINRELEASE   REQUESTSINCURRENTHOUR   REQUESTSINLAST24H
ingresses.v1beta1.extensions   1.22               0                       18

3. Try to access the deprecated API ingresses.v1beta1.extensions ...
$ oc get ingresses.v1beta1.extensions
W1215 13:09:34.525757 1619464 warnings.go:70] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
No resources found in default namespace.

----------------------------------
4. After 1 hour(s) later, check the deprecated apirequestcount of ingresses.v1beta1.extensions, no longer has count > 0 for the deprecated API.
$ oc get apirequestcount/ingresses.v1beta1.extensions
NAME                           REMOVEDINRELEASE   REQUESTSINCURRENTHOUR   REQUESTSINLAST24H
ingresses.v1beta1.extensions   1.22               1                       34

----------------------------------
After 2 hour(s) later, check the deprecated apirequestcount of ingresses.v1beta1.extensions ...
$ oc get apirequestcount/ingresses.v1beta1.extensions
NAME                           REMOVEDINRELEASE   REQUESTSINCURRENTHOUR   REQUESTSINLAST24H
ingresses.v1beta1.extensions   1.22               1                       48

----------------------------------
5. Delete apiresourcecounts/ingresses.v1beta1.extensions . Observe that it does not get recreated. (wait at least 5 min)
Wed 15 Dec 2021 03:09:46 PM CST
$ oc delete apirequestcount/ingresses.v1beta1.extensions
apirequestcount.apiserver.openshift.io "ingresses.v1beta1.extensions" deleted

Wed 15 Dec 2021 03:19:47 PM CST
Waiting 10 mins, check ingresses.v1beta1.extensions again,
$ oc get apirequestcount | grep "ingresses.v1beta1.extensions"
ingresses.v1beta1.extensions              1.22               3                       3

6. Invoke the deprecated API again and confirm apirequestcount/ingresses.v1beta1.extensions gets recreated.
Wed 15 Dec 2021 03:19:54 PM CST
$ oc get apirequestcount/ingresses.v1beta1.extensions
NAME                           REMOVEDINRELEASE   REQUESTSINCURRENTHOUR   REQUESTSINLAST24H
ingresses.v1beta1.extensions   1.22               3                       3

7.Identify uses of removed APIs:

$ oc get apirequestcounts ingresses.v1beta1.extensions -o jsonpath='{range ..username}{$}{"\n"}{end}' | sort | uniq
system:kube-controller-manager

We can safely ignore the entries system:kube-controller-manager appears in the results because it walks through all resources to count them while enforcing quotas. 
The REQUESTSINCURRENTHOUR has correct count as expected, so move the bug VERIFIED.

Comment 7 errata-xmlrpc 2022-01-05 16:12:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.25 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:5209