Description of problem: upi-on-azure OVN 4.10.16 cluster, APIRemovedInNextEUSReleaseInUse alert fired for poddisruptionbudgets, checked the apirequest, it is from openshift-network-operator # token=`oc sa get-token prometheus-k8s -n openshift-monitoring` # oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://thanos-querier.openshift-monitoring.svc:9091/api/v1/query?' --data-urlencode 'query=ALERTS{alertname="APIRemovedInNextEUSReleaseInUse",resource="poddisruptionbudgets"}' | jq { "status": "success", "data": { "resultType": "vector", "result": [ { "metric": { "__name__": "ALERTS", "alertname": "APIRemovedInNextEUSReleaseInUse", "alertstate": "firing", "environment": "Azure", "group": "policy", "namespace": "openshift-kube-apiserver", "prometheus": "openshift-monitoring/k8s", "region": "mtcazs", "resource": "poddisruptionbudgets", "severity": "info", "version": "v1beta1" }, "value": [ 1654055786.192, "1" ] } ] } } # oc get apirequestcounts poddisruptionbudgets.v1beta1.policy -o yaml apiVersion: apiserver.openshift.io/v1 kind: APIRequestCount metadata: creationTimestamp: "2022-05-31T21:56:57Z" generation: 1 name: poddisruptionbudgets.v1beta1.policy resourceVersion: "142700" uid: f38c8fa7-a9da-4945-b9b4-7ff0542ba9af spec: numberOfUsersToReport: 10 status: currentHour: byNode: - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - byUser: - byVerb: - requestCount: 13 verb: get - requestCount: 13 verb: update requestCount: 26 userAgent: cluster-network-operator/v0.0.0 username: system:serviceaccount:openshift-network-operator:default nodeName: 10.0.0.8 requestCount: 26 requestCount: 26 last24h: - byNode: - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - byUser: - byVerb: - requestCount: 20 verb: get - requestCount: 20 verb: update requestCount: 40 userAgent: cluster-network-operator/v0.0.0 username: system:serviceaccount:openshift-network-operator:default nodeName: 10.0.0.8 requestCount: 40 requestCount: 40 - byNode: - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - byUser: - byVerb: - requestCount: 20 verb: get - requestCount: 20 verb: update requestCount: 40 userAgent: cluster-network-operator/v0.0.0 username: system:serviceaccount:openshift-network-operator:default nodeName: 10.0.0.8 requestCount: 40 requestCount: 40 - byNode: - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - byUser: - byVerb: - requestCount: 20 verb: get - requestCount: 20 verb: update requestCount: 40 userAgent: cluster-network-operator/v0.0.0 username: system:serviceaccount:openshift-network-operator:default nodeName: 10.0.0.8 requestCount: 40 requestCount: 40 - byNode: - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - byUser: - byVerb: - requestCount: 13 verb: get - requestCount: 13 verb: update requestCount: 26 userAgent: cluster-network-operator/v0.0.0 username: system:serviceaccount:openshift-network-operator:default nodeName: 10.0.0.8 requestCount: 26 requestCount: 26 - requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - nodeName: 10.0.0.5 requestCount: 0 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 0 - byNode: - byUser: - byVerb: - requestCount: 1 verb: create - requestCount: 30 verb: get - requestCount: 29 verb: update requestCount: 60 userAgent: cluster-network-operator/v0.0.0 username: system:serviceaccount:openshift-network-operator:default nodeName: 10.0.0.5 requestCount: 60 - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - nodeName: 10.0.0.8 requestCount: 0 requestCount: 60 - byNode: - byUser: - byVerb: - requestCount: 19 verb: get - requestCount: 19 verb: update requestCount: 38 userAgent: cluster-network-operator/v0.0.0 username: system:serviceaccount:openshift-network-operator:default nodeName: 10.0.0.6 requestCount: 38 - nodeName: 10.0.0.7 requestCount: 0 - byUser: - byVerb: - requestCount: 16 verb: get - requestCount: 16 verb: update requestCount: 32 userAgent: cluster-network-operator/v0.0.0 username: system:serviceaccount:openshift-network-operator:default nodeName: 10.0.0.8 requestCount: 32 requestCount: 70 - byNode: - nodeName: 10.0.0.6 requestCount: 0 - nodeName: 10.0.0.7 requestCount: 0 - byUser: - byVerb: - requestCount: 20 verb: get - requestCount: 20 verb: update requestCount: 40 userAgent: cluster-network-operator/v0.0.0 username: system:serviceaccount:openshift-network-operator:default nodeName: 10.0.0.8 requestCount: 40 requestCount: 40 removedInRelease: "1.25" requestCount: 316 # oc -n openshift-network-operator get po NAME READY STATUS RESTARTS AGE network-operator-6ddb6d99bc-hlg2v 1/1 Running 0 6h4m # oc -n openshift-network-operator logs network-operator-6ddb6d99bc-hlg2v | grep -i poddisruptionbudget | tail I0601 03:26:58.392985 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard I0601 03:29:59.402334 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard I0601 03:33:00.425388 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard I0601 03:36:01.481744 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard I0601 03:39:02.516711 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard I0601 03:42:03.559903 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard I0601 03:45:04.593258 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard I0601 03:48:05.610182 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard I0601 03:51:06.642777 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard I0601 03:54:07.719012 1 log.go:184] reconciling (policy/v1beta1, Kind=PodDisruptionBudget) openshift-ovn-kubernetes/ovn-raft-quorum-guard Version-Release number of selected component (if applicable): # oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.10.16 True False 5h33m Cluster version is 4.10.16 # oc get network cluster -o jsonpath="{.spec.networkType}" OVNKubernetes How reproducible: always Steps to Reproduce: 1. see the description 2. 3. Actual results: APIRemovedInNextEUSReleaseInUse alert fired for poddisruptionbudgets Expected results: no such alert Additional info: maybe caused by https://github.com/openshift/ovn-kubernetes/blob/release-4.10/go-controller/vendor/modules.txt#L388
Anurag, we are looking to move this bz forward. The code is merged but we are on QA. Can you PTAL and advance this issues status?
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.52 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:0698