Bug 2076842

Summary: Auth tracker to switch to use flowcontrol.apiserver.k8s.io/v1beta2 instead v1beta1 to be deprecated in k8s 1.26
Product: OpenShift Container Platform Reporter: Xingxing Xia <xxia>
Component: apiserver-authAssignee: Krzysztof Ostrowski <kostrows>
Status: CLOSED WONTFIX QA Contact: Xingxing Xia <xxia>
Severity: medium Docs Contact:
Priority: low    
Version: 4.11CC: aos-bugs, kostrows, mfojtik, surbania
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2076877 (view as bug list) Environment:
Last Closed: 2023-01-16 14:43:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Xingxing Xia 2022-04-20 04:22:49 UTC
Description of problem:
Auth tracker to switch to use flowcontrol.apiserver.k8s.io/v1beta2 instead v1beta1 to be deprecated in k8s 1.26, see:
$ oc get apirequestcounts | grep -e NAME -e flowcontrol.apiserver.k8s.io
NAME                                                                           REMOVEDINRELEASE   REQUESTSINCURRENTHOUR   REQUESTSINLAST24H
flowschemas.v1beta1.flowcontrol.apiserver.k8s.io                               1.26               171                     240
flowschemas.v1beta2.flowcontrol.apiserver.k8s.io                                                  2235                    3084
prioritylevelconfigurations.v1beta1.flowcontrol.apiserver.k8s.io               1.26               14                      26
prioritylevelconfigurations.v1beta2.flowcontrol.apiserver.k8s.io                                  1520                    2079

Using this bug for Auth component. Other components should create separate bugs.

Version-Release number of selected component (if applicable):
4.11.0-0.nightly-2022-04-20-002931

How reproducible:
Always

Steps to Reproduce:
1. $ MASTERS=`oc get no | grep master | grep -o '^[^ ]*'`

2. $ for i in $MASTERS; do oc debug no/$i -- chroot /host bash -c "grep -hE '"'"k8s.io/removed-release":"[^"]+"'"' /var/log/kube-apiserver/audit*.log" ; done > all.log

3. $ grep '"k8s.io/removed-release":"1.26"' all.log > 1.26.log

4. $ jq -r '.user.username+": "+.requestURI' 1.26.log | sed 's/=[0-9][^&]*/=***/g' | sort | uniq -c | sort -n > 1.26.removed.apis

5. $ cat 1.26.removed.apis

Check auth specific requests for this bug to fix:
6. $ cat 1.26.removed.apis | grep auth

Actual results:
5. `cat 1.26.removed.apis` shows below requests that are using apiversions to be deprecated in k8s 1.26, the beginning is the request number, followed by request user and requested URL:
      1 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations
      1 system:serviceaccount:openshift-monitoring:kube-state-metrics: /apis/autoscaling/v2beta2/horizontalpodautoscalers?limit=***&resourceVersion=***
      9 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas
     11 system:serviceaccount:openshift-monitoring:kube-state-metrics: /apis/autoscaling/v2beta2/horizontalpodautoscalers?allowWatchBookmarks=true&resourceVersion=***&timeout=***&timeoutSeconds=***&watch=true
     18 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-authentication-operator
     18 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-oauth-apiserver
     18 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-oauth-apiserver-sar
     18 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-oauth-server
     20 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-apiserver
     20 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-apiserver-operator
     20 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-apiserver-sar
     20 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-controller-manager
     20 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-etcd-operator
     20 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-kube-apiserver-operator
     20 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-monitoring-metrics
     20 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations/openshift-control-plane-operators
     29 system:serviceaccount:openshift-network-operator:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-sdn?fieldManager=cluster-network-operator%2Foperconfig&force=true

6. $ cat 1.26.removed.apis | grep auth
     18 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-authentication-operator
     18 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-oauth-apiserver
     18 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-oauth-apiserver-sar
     18 system:serviceaccount:openshift-cluster-version:default: /apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/openshift-oauth-server

The auth file to fix is: https://github.com/openshift/cluster-authentication-operator/blob/master/manifests/09_flowschema.yaml , as of now this bug is filed, its 4 objects still use:
```
apiVersion: flowcontrol.apiserver.k8s.io/v1beta1
kind: FlowSchema
...
```

Expected results:
The components shown in above requests should switch to v1beta2.flowcontrol.apiserver.k8s.io. This bug tracks auth components.

Additional info:

Comment 2 Michal Fojtik 2023-01-16 14:43:05 UTC
Dear reporter, we greatly appreciate the bug you have reported here. Unfortunately, due to migration to a new issue-tracking system (https://issues.redhat.com/), we cannot continue triaging bugs reported in Bugzilla. Since this bug has been stale for multiple days, we, therefore, decided to close this bug.
If you think this is a mistake or this bug has a higher priority or severity as set today, please feel free to reopen this bug and tell us why. We are going to move every re-opened bug to https://issues.redhat.com. 

Thank you for your patience and understanding.