Bug 1764491

Summary: CVO uses deprecated API
Product: OpenShift Container Platform Reporter: Tomáš Nožička <tnozicka>
Component: Cloud ComputeAssignee: Alberto <agarcial>
Status: CLOSED ERRATA QA Contact: Jianwei Hou <jhou>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.3.0CC: aos-bugs, jokerman, maszulik, wking
Target Milestone: ---   
Target Release: 4.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-01-23 11:09:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tomáš Nožička 2019-10-23 07:21:46 UTC
Description of problem:

kube 1.16 disabled serving apps/v1beta2 and with the rebase we have temporary enabled it back for this release and we are adding an alert to warn on using the deprecated apis. Serving will be disabled in the next release.

CVO needs to migrate to apps/v1 ASAP.

apiserver_request_count{client="cluster-version-operator/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",component="apiserver",contentType="application/json",endpoint="https",group="apps",instance="10.0.154.241:6443",job="apiserver",namespace="default",resource="deployments",scope="namespace",service="kubernetes",verb="GET",version="v1beta2"}  24

apiserver_request_count{client="cluster-version-operator/v0.0.0 (linux/amd64) kubernetes/$Format",code="200",component="apiserver",contentType="application/json",endpoint="https",group="apps",instance="10.0.154.241:6443",job="apiserver",namespace="default",resource="deployments",scope="namespace",service="kubernetes",verb="PUT",version="v1beta2"}   24




Version-Release number of selected component (if applicable):
oc version
Client Version: version.Info{Major:"", Minor:"", GitVersion:"v4.2.0-alpha.0-56-ge6a379d", GitCommit:"e6a379d50", GitTreeState:"clean", BuildDate:"2019-09-05T10:19:53Z", GoVersion:"go1.13", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.0-beta.2+03b33d2", GitCommit:"03b33d2", GitTreeState:"clean", BuildDate:"2019-10-21T19:38:32Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
OpenShift Version: 4.3.0-0.ci-2019-10-22-062243

Comment 1 Maciej Szulik 2019-10-29 09:32:49 UTC
This is a release blocker, we cannot deliver w/o this being fixed ASAP. We have alerts in place which will be triggered every time deprecated API is being used. 
This alert should not apply to any resources provided by OpenShift itself but should serve customers as an information that in the next version these API won't
be available.

Comment 2 Abhinav Dahiya 2019-10-30 16:43:38 UTC
CVO only uses the APIs that operators define/use in the `manifests` directory. So this is not a cvo bug, please open bugs against operators.

Comment 3 W. Trevor King 2019-10-31 16:17:07 UTC
Also in this space, sometimes we need to use older versions for compatibility during upgrades (e.g. [1]).

[1]: https://github.com/openshift/cluster-version-operator/pull/265

Comment 4 W. Trevor King 2019-10-31 16:29:22 UTC
Looking at 4.3.0-0.nightly-2019-10-22-143646 [1]:

$ oc adm release extract --to manifests quay.io/openshift-release-dev/ocp-release-nightly@sha256:b9eac53ce3fe8f4756fdb1f8858382f77e09ad5e8334f6a00b495583f09684ca
Extracted release payload created at 2019-10-22T18:42:26Z
$ grep -r v1beta2 manifests
...no hits...
$ oc adm release info --commits quay.io/openshift-release-dev/ocp-release-nightly@sha256:b9eac53ce3fe8f4756fdb1f8858382f77e09ad5e8334f6a00b495583f09684ca | grep cluster-version-operator
  cluster-version-operator                      https://github.com/openshift/cluster-version-operator                      qe87726dfabea14903e55341f595b

Looking at the most recent 4.3 nightly:

$ oc adm release extract --to manifests registry.svc.ci.openshift.org/ocp/release:4.3.0-0.nightly-2019-10-31-125230
Extracted release payload from digest sha256:29ac20bafe14bb396ca52ef8f46d97a4a9ab1016137192408c8bb95f4facd42b created at 2019-10-31T12:59:23Z
$ grep -r v1beta2 manifests
manifests/0000_90_kube-apiserver-operator_04_servicemonitor-apiserver.yaml:        message: A client in the cluster is using deprecated apps/v1beta2 API that will be removed soon.
manifests/0000_90_kube-apiserver-operator_04_servicemonitor-apiserver.yaml:        apiserver_request_count{group="apps",version="v1beta2"}

So all of this looks good.  Checking the CVO source:

$ git grep v1beta2 ae1f16b5ee8f -- pkg
...no hits...
$ git grep v1beta2 origin/master -- pkg
$ git describe origin/master
v1.0.0-135-gebb86aa

So I dunno where this would have been coming from.  Are we still seeing it?

[1]: https://mirror.openshift.com/pub/openshift-v4/clients/ocp-dev-preview/4.3.0-0.nightly-2019-10-22-143646/release.txt

Comment 5 W. Trevor King 2019-10-31 16:33:06 UTC
Aha!  From [1]:

$ oc adm release extract --to manifests quay.io/openshift-release-dev/ocp-release-nightly@sha256:101a63f5f7de11bc11fcd9582f810a0ce01e341ff606fb1e03c5ac47de5c074a
Extracted release payload created at 2019-10-18T14:40:13Z
$ grep -r v1beta2 manifests
manifests/0000_30_machine-api-operator_11_deployment.yaml:apiVersion: apps/v1beta2
$ head manifests/0000_30_machine-api-operator_11_deployment.yaml
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: machine-api-operator
  namespace: openshift-machine-api
  labels:
    k8s-app: machine-api-operator
spec:
  replicas: 1
  selector:

So this was fixed by [2].

[1]: https://mirror.openshift.com/pub/openshift-v4/clients/ocp-dev-preview/4.3.0-0.nightly-2019-10-18-120524/release.txt
[2]: https://github.com/openshift/machine-api-operator/pull/422

Comment 7 Jianwei Hou 2019-11-15 06:27:40 UTC
Verified this is fixed in 4.3.0-0.nightly-2019-11-13-233341

Comment 9 errata-xmlrpc 2020-01-23 11:09:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062