Bug 1768417 - Test cluster is firing large number of UsingDeprecatedAPIExtensionsV1Beta1 alerts.
Summary: Test cluster is firing large number of UsingDeprecatedAPIExtensionsV1Beta1 al...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-apiserver
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.3.0
Assignee: Tomáš Nožička
QA Contact: Xingxing Xia
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-04 12:10 UTC by Pawel Krupa
Modified: 2020-01-23 11:10 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1788812 (view as bug list)
Environment:
Last Closed: 2020-01-23 11:10:30 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-kube-apiserver-operator pull 639 0 'None' closed Bug 1768417: Adjust UsingDeprecatedAPIExtensionsV1Beta1 alert 2020-05-20 06:55:10 UTC
Red Hat Product Errata RHBA-2020:0062 0 None None None 2020-01-23 11:10:40 UTC

Description Pawel Krupa 2019-11-04 12:10:03 UTC
Description of problem:
Cluster is firing multiple `UsingDeprecatedAPIExtensionsV1Beta1` alerts.


Version-Release number of selected component (if applicable):


How reproducible:
Running e2e-gcp CI job and checking prometheus dump


Steps to Reproduce:
1. 
2.
3.

Actual results:


Expected results:


Additional info:

Monitoring team refactored tests for checking number of alerts being fired and excluded checking for `UsingDeprecatedAPIExtensionsV1Beta1`. Inclusion of that check is done in https://github.com/openshift/origin/pull/24071 which cannot be merged due to this bug.

Comment 1 Tomáš Nožička 2019-11-04 15:11:24 UTC
looks like client changed for kcm in the meantime when the check was borked

Comment 3 Xingxing Xia 2019-11-11 14:30:03 UTC
Verified in 4.3.0-0.nightly-2019-11-11-115927:
$ cat daemonset-simple-hello-openshift.yaml
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: hello-openshift-daemonset
spec:
  selector:
      matchLabels:
        name: hello-openshift-daemonset
  template:
    metadata:
      labels:
        name: hello-openshift-daemonset
    spec:
      containers:
      - image: openshift/hello-openshift
        name: hello-openshift
$ oc create -f daemonset-simple-hello-openshift.yaml
Query apiserver_request_count{client!~"hyperkube/.*|cluster-policy-controller/.*",group="extensions",resource!~"ingresses|",version="v1beta1"}
Get:
apiserver_request_count{client="oc/v0.0.0 (linux/amd64) kubernetes/$Format",code="201",component="apiserver",contentType="application/json",endpoint="https",group="extensions",instance="10.0.142.221:6443",job="apiserver",namespace="default",resource="daemonsets",scope="namespace",service="kubernetes",verb="POST",version="v1beta1"}	1
Check alerts, get: UsingDeprecatedAPIExtensionsV1Beta1 (1 active)

Comment 4 Pawel Krupa 2019-12-16 14:46:14 UTC
Still happening for 4.4.

https://github.com/openshift/origin/pull/24071 should be green if this would not be happening.

Comment 5 Lili Cosic 2019-12-16 14:49:18 UTC
Can verify, on 4.4.0-0.nightly-2019-12-16-025547 the alert is still firing (it is in fact firing 20 times).

apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.148.182:6443",job="apiserver",namespace="default",resource="replicasets",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.148.182:6443",job="apiserver",namespace="default",resource="podsecuritypolicies",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.148.182:6443",job="apiserver",namespace="default",resource="networkpolicies",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.148.182:6443",job="apiserver",namespace="default",resource="deployments",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.148.182:6443",job="apiserver",namespace="default",resource="daemonsets",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.135.65:6443",job="apiserver",namespace="default",resource="replicasets",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.135.65:6443",job="apiserver",namespace="default",resource="podsecuritypolicies",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.135.65:6443",job="apiserver",namespace="default",resource="networkpolicies",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.135.65:6443",job="apiserver",namespace="default",resource="deployments",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="200",component="apiserver",contentType="application/vnd.kubernetes.protobuf",endpoint="https",group="extensions",instance="10.0.135.65:6443",job="apiserver",namespace="default",resource="daemonsets",scope="cluster",service="kubernetes",verb="LIST",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance="10.0.148.182:6443",job="apiserver",namespace="default",resource="replicasets",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance="10.0.148.182:6443",job="apiserver",namespace="default",resource="podsecuritypolicies",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance="3",job="apiserver",namespace="default",resource="networkpolicies",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance="43",job="apiserver",namespace="default",resource="deployments",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance="43",job="apiserver",namespace="default",resource="daemonsets",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance="3",job="apiserver",namespace="default",resource="replicasets",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance=":6443",job="apiserver",namespace="default",resource="podsecuritypolicies",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance="3",job="apiserver",namespace="default",resource="networkpolicies",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance="",job="apiserver",namespace="default",resource="deployments",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}
✔apiserver_request_count{client="kube-controller-manager/v1.16.2 (linux/amd64) kubernetes/177d316/kube-controller-manager",code="0",component="apiserver",contentType="application/vnd.kubernetes.protobuf;stream=watch",endpoint="https",group="extensions",instance="",job="apiserver",namespace="default",resource="daemonsets",scope="cluster",service="kubernetes",verb="WATCH",version="v1beta1"}

Comment 6 Maciej Szulik 2019-12-16 16:07:44 UTC
Moving to 4.4, per previous comment.

Comment 7 Xingxing Xia 2020-01-08 07:37:31 UTC
Confirmed in latest 4.3.0-0.nightly-2020-01-08-005052 env like above verified comment, 4.3 indeed has no the issue of above new 4.4 comments. Since this bug attached 4.3 advisory which must include right target release bugs, cloning a 4.4 bug 1788812 for the issue of above new comments.

Comment 10 errata-xmlrpc 2020-01-23 11:10:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062


Note You need to log in before you can comment on or make changes to this bug.