DescriptionAlexander Constantinescu
2021-04-12 15:26:54 UTC
+++ This bug was initially created as a clone of Bug #1947795 +++
This component accesses APIs that will be removed in 4.9 (Kubernetes 1.22). It is causing the DeprecatedAPIInUse alert to fire in every 4.8 clusters permanently and hence must be fixed in 4.8 (blocker+).
The raw audit data can be found at https://gist.github.com/sttts/50a1429837f2448ce07f30174fa73cdb.
Here are the observed requests for this component:
system:serviceaccount:openshift-cluster-version:default: /apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/networks.config.openshift.io
system:serviceaccount:openshift-cluster-version:default: /apis/rbac.authorization.k8s.io/v1beta1/clusterrolebindings/default-account-cluster-network-operator
+++ This bug was initially created as a clone of Bug #1947719 +++
Created attachment 1770482[details]
alert screen shot
Created attachment 1770482[details]
alert screen shot
Description of problem:
8 DeprecatedAPIInUse info alerts display
Version-Release number of selected component (if applicable):
4.8.0-0.nightly-2021-04-08-200632
How reproducible:
always
Steps to Reproduce:
1. open console-monitoring-alerts
2.
3.
Actual results:
8 DeprecatedAPIInUse info alerts display
Expected results:
No other alerts display except watchdog
Additional info:
alert rule metrics:
group by(group, version, resource) (apiserver_requested_deprecated_apis{removed_release="1.22"}) and (sum by(group, version, resource) (rate(apiserver_request_total[10m]))) > 0
Element Value:
{group="rbac.authorization.k8s.io",resource="roles",version="v1beta1"} 1
{group="admissionregistration.k8s.io",resource="mutatingwebhookconfigurations",version="v1beta1"} 1
{group="admissionregistration.k8s.io",resource="validatingwebhookconfigurations",version="v1beta1"} 1
{group="apiextensions.k8s.io",resource="customresourcedefinitions",version="v1beta1"} 1
{group="certificates.k8s.io",resource="certificatesigningrequests",version="v1beta1"} 1
{group="extensions",resource="ingresses",version="v1beta1"} 1
{group="rbac.authorization.k8s.io",resource="clusterrolebindings",version="v1beta1"} 1
{group="rbac.authorization.k8s.io",resource="rolebindings",version="v1beta1"} 1
----------------
# for i in roles mutatingwebhookconfigurations validatingwebhookconfigurations customresourcedefinitions certificatesigningrequests ingresses clusterrolebindings rolebindings; do oc api-resources | grep $i; echo -e "\n"; done
clusterroles authorization.openshift.io/v1 false ClusterRole
roles authorization.openshift.io/v1 true Role
clusterroles rbac.authorization.k8s.io/v1 false ClusterRole
roles rbac.authorization.k8s.io/v1 true Role
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition
certificatesigningrequests csr certificates.k8s.io/v1 false CertificateSigningRequest
ingresses config.openshift.io/v1 false Ingress
ingresses ing extensions/v1beta1 true Ingress
ingresses ing networking.k8s.io/v1 true Ingress
clusterrolebindings authorization.openshift.io/v1 false ClusterRoleBinding
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
clusterrolebindings authorization.openshift.io/v1 false ClusterRoleBinding
rolebindings authorization.openshift.io/v1 true RoleBinding
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
rolebindings rbac.authorization.k8s.io/v1 true RoleBinding
--- Additional comment from Junqi Zhao on 2021-04-09 05:28:56 CEST ---
alert details
alert:DeprecatedAPIInUse
expr:group by(group, version, resource) (apiserver_requested_deprecated_apis{removed_release="1.22"}) and (sum by(group, version, resource) (rate(apiserver_request_total[10m]))) > 0
for: 1h
labels:
severity: info
annotations:
message: Deprecated API that will be removed in the next version is being used. Removing the workload that is using the {{"{{$labels.group}}"}}.{{"{{$labels.version}}"}}/{{"{{$labels.resource}}"}} API might be necessary for a successful upgrade to the next cluster version. Refer to the audit logs to identify the workload.
--- Additional comment from hongyan li on 2021-04-09 05:37:17 CEST ---
--- Additional comment from hongyan li on 2021-04-09 05:44:46 CEST ---
Different issue from bug 1932165 which is about variable not translated to value
--- Additional comment from Junqi Zhao on 2021-04-09 06:04:30 CEST ---
# oc version
Client Version: 4.8.0-0.nightly-2021-04-08-200632
Server Version: 4.8.0-0.nightly-2021-04-08-200632
Kubernetes Version: v1.21.0-rc.0+6d27558
checked from prometheus, query parameter:
count(apiserver_requested_deprecated_apis{removed_release="1.22"}) by(instance,version,group,resource)
version is v1beta1
{group="certificates.k8s.io", instance="10.0.160.188:6443", resource="certificatesigningrequests", version="v1beta1"} 1
{group="extensions", instance="10.0.160.188:6443", resource="ingresses", version="v1beta1"} 1
{group="rbac.authorization.k8s.io", instance="10.0.160.188:6443", resource="clusterrolebindings", version="v1beta1"} 1
{group="rbac.authorization.k8s.io", instance="10.0.160.188:6443", resource="rolebindings", version="v1beta1"} 1
{group="rbac.authorization.k8s.io", instance="10.0.160.188:6443", resource="roles", version="v1beta1"} 1
{group="admissionregistration.k8s.io", instance="10.0.160.188:6443", resource="mutatingwebhookconfigurations", version="v1beta1"} 1
{group="admissionregistration.k8s.io", instance="10.0.160.188:6443", resource="validatingwebhookconfigurations", version="v1beta1"} 1
{group="apiextensions.k8s.io", instance="10.0.160.188:6443", resource="customresourcedefinitions", version="v1beta1"} 1
but the api versions are all actually v1, which means apiserver_requested_deprecated_apis may post the wrong result
# for i in certificatesigningrequests ingresses clusterrolebindings rolebindings roles mutatingwebhookconfigurations validatingwebhookconfigurations customresourcedefinitions; do oc api-resources | grep $i; echo -e "\n"; done
certificatesigningrequests csr certificates.k8s.io/v1 false CertificateSigningRequest
ingresses config.openshift.io/v1 false Ingress
ingresses ing extensions/v1beta1 true Ingress
ingresses ing networking.k8s.io/v1 true Ingress
clusterrolebindings authorization.openshift.io/v1 false ClusterRoleBinding
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
clusterrolebindings authorization.openshift.io/v1 false ClusterRoleBinding
rolebindings authorization.openshift.io/v1 true RoleBinding
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
rolebindings rbac.authorization.k8s.io/v1 true RoleBinding
clusterroles authorization.openshift.io/v1 false ClusterRole
roles authorization.openshift.io/v1 true Role
clusterroles rbac.authorization.k8s.io/v1 false ClusterRole
roles rbac.authorization.k8s.io/v1 true Role
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition
Comment 1Alexander Constantinescu
2021-04-12 15:27:44 UTC
This bug is used for tracking all OVN-Kubernetes dependencies and making sure they are bumped by code freeze.
Comment 2Alexander Constantinescu
2021-04-12 15:27:57 UTC
This bug is used for tracking all OVN-Kubernetes dependencies and making sure they are bumped by code freeze.
Comment 3Ricardo Carrillo Cruz
2021-04-16 10:54:03 UTC