Bug 1851693 - The `oc apply` should return errors instead of hanging there when failing to create the CRD
Summary: The `oc apply` should return errors instead of hanging there when failing to ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 4.5
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.7.0
Assignee: Maciej Szulik
QA Contact: RamaKasturi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-06-28 09:31 UTC by Jian Zhang
Modified: 2021-02-24 15:13 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-24 15:13:20 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:13:22 UTC

Description Jian Zhang 2020-06-28 09:31:38 UTC
Description of problem:
Creating CRDs hangs there for hours.

$ wget https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/ocp/manifests/0.15.1/0000_50_olm_00-clusterserviceversions.crd.yaml

$ ./oc --loglevel 8 apply -f 0000_50_olm_00-clusterserviceversions.crd.yaml
...
I0628 04:54:34.485840    1730 request.go:1068] Response Body: {"kind":"CustomResourceDefinition","apiVersion":"apiextensions.k8s.io/v1","metadata":{"name":"clusterserviceversions.operators.coreos.com","selfLink":"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com","uid":"37defd90-d137-43df-98cd-c1ca432c9186","resourceVersion":"30944","generation":1,"creationTimestamp":"2020-06-28T07:39:30Z","annotations":{"controller-gen.kubebuilder.io/version":"v0.3.0"},"managedFields":[{"manager":"kube-apiserver","operation":"Update","apiVersion":"apiextensions.k8s.io/v1","time":"2020-06-28T07:39:30Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:acceptedNames":{"f:categories":{},"f:kind":{},"f:listKind":{},"f:plural":{},"f:shortNames":{},"f:singular":{}},"f:conditions":{}}}},{"manager":"kubectl","operation":"Update","apiVersion":"apiextensions.k8s.io/v1","time":"2020-06-28T07:39:30Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:controller-gen.kubebuilder.io/version":{}}},"f:spec":{"f:conversion":{". [truncated 221575 chars]
Warning: oc apply should be used on resource created by either oc create --save-config or oc apply


Version-Release number of selected component (if applicable):
[root@preserve-olm-env data]# ./oc version
Client Version: 4.5.0-rc.4
Server Version: 4.5.0-0.nightly-2020-06-26-215024
Kubernetes Version: v1.18.3+f291db1

How reproducible:
always

Steps to Reproduce:
1. Install the OCP 4.5.

This clusterserviceversions CRD already exist in the cluster as default.
[root@preserve-olm-env data]# ./oc get crd|grep clusterserviceversions
clusterserviceversions.operators.coreos.com                 2020-06-28T09:15:22Z


2. Install this clusterserviceversions CRD again.
$ wget https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/ocp/manifests/0.15.1/0000_50_olm_00-clusterserviceversions.crd.yaml

Actual results:
It hangs there for a long time.
$ ./oc --loglevel 8 apply -f 0000_50_olm_00-clusterserviceversions.crd.yaml
...
I0628 04:54:34.485840    1730 request.go:1068] Response Body: {"kind":"CustomResourceDefinition","apiVersion":"apiextensions.k8s.io/v1","metadata":{"name":"clusterserviceversions.operators.coreos.com","selfLink":"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com","uid":"37defd90-d137-43df-98cd-c1ca432c9186","resourceVersion":"30944","generation":1,"creationTimestamp":"2020-06-28T07:39:30Z","annotations":{"controller-gen.kubebuilder.io/version":"v0.3.0"},"managedFields":[{"manager":"kube-apiserver","operation":"Update","apiVersion":"apiextensions.k8s.io/v1","time":"2020-06-28T07:39:30Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:acceptedNames":{"f:categories":{},"f:kind":{},"f:listKind":{},"f:plural":{},"f:shortNames":{},"f:singular":{}},"f:conditions":{}}}},{"manager":"kubectl","operation":"Update","apiVersion":"apiextensions.k8s.io/v1","time":"2020-06-28T07:39:30Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:controller-gen.kubebuilder.io/version":{}}},"f:spec":{"f:conversion":{". [truncated 221575 chars]
Warning: oc apply should be used on resource created by either oc create --save-config or oc apply


Expected results:
The `oc apply` should return errors immediately if anything wrong instead of hanging there. For example,
#  ./oc create -f 0000_50_olm_00-clusterserviceversions.crd.yaml 
Error from server (AlreadyExists): error when creating "0000_50_olm_00-clusterserviceversions.crd.yaml": customresourcedefinitions.apiextensions.k8s.io "clusterserviceversions.operators.coreos.com" already exists

Additional info:
1, The `oc create` works well.
2, The `oc apply` works well after removing the exist clusterserviceversions CRD

3, Check the kube-controller-manager logs but didn't get any errors:
[root@preserve-olm-env data]#  oc logs kube-controller-manager-ip-10-0-186-87.us-east-2.compute.internal -n openshift-kube-controller-manager -c kube-controller-manager |grep clusterserviceversions.operators.coreos.com
I0628 07:23:25.734631       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for clusterserviceversions.operators.coreos.com
...
 [], removed: [operators.coreos.com/v1, Resource=operatorgroups operators.coreos.com/v1alpha1, Resource=catalogsources operators.coreos.com/v1alpha1, Resource=clusterserviceversions operators.coreos.com/v1alpha1, Resource=installplans operators.coreos.com/v1alpha1, Resource=subscriptions]
I0628 07:39:29.660946       1 garbagecollector.go:204] syncing garbage collector with updated resources from discovery (attempt 1): added: [], removed: [operators.coreos.com/v1, Resource=operatorgroups operators.coreos.com/v1alpha1, Resource=catalogsources operators.coreos.com/v1alpha1, Resource=clusterserviceversions operators.coreos.com/v1alpha1, Resource=installplans operators.coreos.com/v1alpha1, Resource=subscriptions]
I0628 07:39:59.524013       1 resource_quota_controller.go:434] syncing resource quota controller with updated resources from discovery: added: [operators.coreos.com/v1, Resource=operatorgroups operators.coreos.com/v1alpha1, Resource=catalogsources operators.coreos.com/v1alpha1, Resource=clusterserviceversions operators.coreos.com/v1alpha1, Resource=installplans operators.coreos.com/v1alpha1, Resource=subscriptions], removed: []
I0628 07:39:59.684951       1 garbagecollector.go:204] syncing garbage collector with updated resources from discovery (attempt 1): added: [operators.coreos.com/v1, Resource=operatorgroups operators.coreos.com/v1alpha1, Resource=catalogsources operators.coreos.com/v1alpha1, Resource=clusterserviceversions operators.coreos.com/v1alpha1, Resource=installplans operators.coreos.com/v1alpha1, Resource=subscriptions], removed: []

Comment 1 Maciej Szulik 2020-06-30 15:07:25 UTC
From my quick debugging session it looks like github.com/evanphx/json-patch library is stuck when creating a merge patch
on a CRD specifically. I have a few possible suspicions: 
- a bug in a library, we might need to upgrade it
- the size of data which might be causing the merge patch calculation complicated 
- semi-related, the managed fields are affecting the previous

Comment 2 Maciej Szulik 2020-07-09 11:08:41 UTC
I’m adding UpcomingSprint, because I was occupied by fixing bugs with higher priority/severity, developing new features with higher priority, or developing new features to improve stability at a macro level. I will revisit this bug next sprint.

Comment 3 Maciej Szulik 2020-07-09 11:11:15 UTC
Actually https://github.com/kubernetes/kubernetes/pull/92069 will be part of the next k8s bump so we should get it then.

Comment 4 Maciej Szulik 2020-08-21 13:59:26 UTC
This was fixed in https://github.com/openshift/oc/pull/491

Comment 9 RamaKasturi 2020-08-25 12:59:18 UTC
Below is the oc version with which the bug was verified.

[ramakasturinarra@dhcp35-60 ~]$ oc version -o yaml
clientVersion:
  buildDate: "2020-08-21T02:37:08Z"
  compiler: gc
  gitCommit: ea0d54068621ec0f95973068729f739f3dacfef7
  gitTreeState: clean
  gitVersion: 4.6.0-202008210209.p0-ea0d540
  goVersion: go1.14.4
  major: ""
  minor: ""
  platform: linux/amd64
openshiftVersion: 4.6.0-0.nightly-2020-08-23-214712
serverVersion:
  buildDate: "2020-08-20T16:46:57Z"
  compiler: gc
  gitCommit: 3e083ac29409923906267ebcc5f8e0aa13072c72
  gitTreeState: dirty
  gitVersion: v1.19.0-rc.2+3e083ac-dirty
  goVersion: go1.14.4
  major: "1"
  minor: 19+
  platform: linux/amd64

Comment 10 Maciej Szulik 2020-08-26 10:38:56 UTC
With

$ oc version
Client Version: 4.6.0-0.ci-2020-08-24-152549
Server Version: 4.6.0-0.ci-2020-08-24-152549
Kubernetes Version: v1.19.0-rc.2.467+3e083ac2940992-dirty

this is working as expected, w/o any errors or delays.

Comment 13 Maciej Szulik 2020-09-11 11:21:57 UTC
I’m adding UpcomingSprint, because I was occupied by fixing bugs with higher priority/severity, developing new features with higher priority, or developing new features to improve stability at a macro level. I will revisit this bug next sprint.

Comment 14 Maciej Szulik 2020-09-15 11:23:09 UTC
From my tests it looks like only this particular CRD is having issues, I'm pushing this over to 4.7

Comment 15 Maciej Szulik 2020-10-01 14:49:20 UTC
I’m adding UpcomingSprint, because I was occupied by fixing bugs with higher priority/severity, developing new features with higher priority, or developing new features to improve stability at a macro level. I will revisit this bug next sprint.

Comment 17 Maciej Szulik 2020-10-23 10:38:08 UTC
I’m adding UpcomingSprint, because I was occupied by fixing bugs with higher priority/severity, developing new features with higher priority, or developing new features to improve stability at a macro level. I will revisit this bug next sprint.

Comment 18 Maciej Szulik 2020-11-05 12:26:36 UTC
I've re-checked this bug again just now. Tested both below versions - original and latest available:
- https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/ocp/manifests/0.15.1/0000_50_olm_00-clusterserviceversions.crd.yaml
- https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/ocp/manifests/0.17.0/0000_50_olm_00-clusterserviceversions.crd.yaml

I've tried running multiple times each and works as expected. Moving to qa for verification.

Comment 20 RamaKasturi 2020-11-09 13:46:02 UTC
Verified bug with the payload below and i see that oc apply does not hang anymore when run more than once.

[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2020-11-08-225909]$ ./oc version
Client Version: 4.7.0-0.nightly-2020-11-08-225909
Server Version: 4.7.0-0.nightly-2020-10-27-051128
Kubernetes Version: v1.19.0+e67f5dc

Below are the steps followed to verify:
=====================================
1) Install 4.7 cluster
2) Now run the command below which says Already Exists as expected

[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2020-11-08-225909]$ ./oc create -f 0000_50_olm_00-clusterserviceversions.crd.yaml 
Error from server (AlreadyExists): error when creating "0000_50_olm_00-clusterserviceversions.crd.yaml": customresourcedefinitions.apiextensions.k8s.io "clusterserviceversions.operators.coreos.com" already exists

3) Now run the command oc apply thirce and i did not any hang as reported in the description.

[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2020-11-08-225909]$ ./oc --loglevel 8 apply -f 0000_50_olm_00-clusterserviceversions.crd.yaml
I1109 19:11:33.990035   12019 loader.go:375] Config loaded from file:  /home/knarra/Downloads/kubeconfig_47
I1109 19:11:33.990652   12019 round_trippers.go:420] GET https://api.knarra1109.qe.devcluster.openshift.com:6443/openapi/v2?timeout=32s
I1109 19:11:33.990658   12019 round_trippers.go:427] Request Headers:
I1109 19:11:33.990661   12019 round_trippers.go:431]     Accept: application/com.github.proto-openapi.spec.v2+protobuf
I1109 19:11:33.990664   12019 round_trippers.go:431]     User-Agent: oc/4.7.0 (linux/amd64) kubernetes/9f54c1d
I1109 19:11:33.990667   12019 round_trippers.go:431]     Authorization: Bearer <masked>
I1109 19:11:35.894003   12019 round_trippers.go:446] Response Status: 200 OK in 1903 milliseconds
I1109 19:11:35.894035   12019 round_trippers.go:449] Response Headers:
I1109 19:11:35.894040   12019 round_trippers.go:452]     Audit-Id: 618d5d2d-0d48-4b34-9509-0c2ec0c64f10
I1109 19:11:35.894043   12019 round_trippers.go:452]     Cache-Control: no-cache, private
I1109 19:11:35.894059   12019 round_trippers.go:452]     Content-Type: application/octet-stream
I1109 19:11:35.894062   12019 round_trippers.go:452]     X-Kubernetes-Pf-Prioritylevel-Uid: 1517604c-7276-4328-ad28-1565abb05931
I1109 19:11:35.894065   12019 round_trippers.go:452]     Date: Mon, 09 Nov 2020 13:41:35 GMT
I1109 19:11:35.894067   12019 round_trippers.go:452]     Accept-Ranges: bytes
I1109 19:11:35.894070   12019 round_trippers.go:452]     X-Varied-Accept: application/com.github.proto-openapi.spec.v2+protobuf
I1109 19:11:35.894073   12019 round_trippers.go:452]     Etag: "C6440351D4A49BB350D54AEED20945D882975D7BC61165A5A2A43E38AD2A9AA7F8937FB5CF25A11A222B178AEB8C5B3B2221164CFBE6295408A3EDB11CDFD14C"
I1109 19:11:35.894075   12019 round_trippers.go:452]     Last-Modified: Mon, 09 Nov 2020 10:34:38 GMT
I1109 19:11:35.894078   12019 round_trippers.go:452]     Vary: Accept-Encoding
I1109 19:11:35.894085   12019 round_trippers.go:452]     Vary: Accept
I1109 19:11:35.894089   12019 round_trippers.go:452]     X-Kubernetes-Pf-Flowschema-Uid: 202c5c05-54a2-4b02-ade2-8f30ecb59d3e
I1109 19:11:40.958146   12019 request.go:1095] Response Body:
00000000  0a 03 32 2e 30 12 1d 0a  0a 4b 75 62 65 72 6e 65  |..2.0....Kuberne|
00000010  74 65 73 12 0f 76 31 2e  31 39 2e 30 2b 65 36 37  |tes..v1.19.0+e67|
00000020  66 35 64 63 42 b7 a3 bc  02 12 fa 55 0a 2c 2f 61  |f5dcB......U.,/a|
00000030  70 69 73 2f 63 6f 6e 66  69 67 2e 6f 70 65 6e 73  |pis/config.opens|
00000040  68 69 66 74 2e 69 6f 2f  76 31 2f 69 6e 66 72 61  |hift.io/v1/infra|
00000050  73 74 72 75 63 74 75 72  65 73 12 c9 55 12 d3 25  |structures..U..%|
00000060  0a 14 63 6f 6e 66 69 67  4f 70 65 6e 73 68 69 66  |..configOpenshif|
00000070  74 49 6f 5f 76 31 1a 23  6c 69 73 74 20 6f 62 6a  |tIo_v1.#list obj|
00000080  65 63 74 73 20 6f 66 20  6b 69 6e 64 20 49 6e 66  |ects of kind Inf|
00000090  72 61 73 74 72 75 63 74  75 72 65 2a 25 6c 69 73  |rastructure*%lis|
000000a0  74 43 6f 6e 66 69 67 4f  70 65 6e 73 68 69 66 74  |tConfigOpenshift|
000000b0  49 6f 56 31 49 6e 66 72  61 73 74 72 75 63 74 75  |IoV1Infrastructu|
000000c0  72 65 32 10 61 70 70 6c  69 63 61 74 69 6f 6e 2f  |re2.application [truncated 37022160 chars]
I1109 19:11:41.074655   12019 round_trippers.go:420] GET https://api.knarra1109.qe.devcluster.openshift.com:6443/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com
I1109 19:11:41.074670   12019 round_trippers.go:427] Request Headers:
I1109 19:11:41.074693   12019 round_trippers.go:431]     Accept: application/json
I1109 19:11:41.074698   12019 round_trippers.go:431]     Authorization: Bearer <masked>
I1109 19:11:41.074702   12019 round_trippers.go:431]     User-Agent: oc/4.7.0 (linux/amd64) kubernetes/9f54c1d
I1109 19:11:41.627646   12019 round_trippers.go:446] Response Status: 200 OK in 552 milliseconds
I1109 19:11:41.627659   12019 round_trippers.go:449] Response Headers:
I1109 19:11:41.627663   12019 round_trippers.go:452]     Cache-Control: no-cache, private
I1109 19:11:41.627666   12019 round_trippers.go:452]     Content-Type: application/json
I1109 19:11:41.627669   12019 round_trippers.go:452]     Vary: Accept-Encoding
I1109 19:11:41.627672   12019 round_trippers.go:452]     X-Kubernetes-Pf-Flowschema-Uid: 202c5c05-54a2-4b02-ade2-8f30ecb59d3e
I1109 19:11:41.627677   12019 round_trippers.go:452]     X-Kubernetes-Pf-Prioritylevel-Uid: 1517604c-7276-4328-ad28-1565abb05931
I1109 19:11:41.627681   12019 round_trippers.go:452]     Date: Mon, 09 Nov 2020 13:41:41 GMT
I1109 19:11:41.627684   12019 round_trippers.go:452]     Audit-Id: ec36d70e-ec26-44e9-a497-7315bb1b0459
I1109 19:11:41.629078   12019 request.go:1097] Response Body: {"kind":"CustomResourceDefinition","apiVersion":"apiextensions.k8s.io/v1","metadata":{"name":"clusterserviceversions.operators.coreos.com","selfLink":"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com","uid":"0bc8b63f-7424-4650-bf93-b3c9103e8bec","resourceVersion":"898","generation":1,"creationTimestamp":"2020-11-09T10:11:23Z","annotations":{"controller-gen.kubebuilder.io/version":"v0.3.0"},"managedFields":[{"manager":"cluster-version-operator","operation":"Update","apiVersion":"apiextensions.k8s.io/v1","time":"2020-11-09T10:11:23Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:controller-gen.kubebuilder.io/version":{}}},"f:spec":{"f:conversion":{".":{},"f:strategy":{}},"f:group":{},"f:names":{"f:categories":{},"f:kind":{},"f:listKind":{},"f:plural":{},"f:shortNames":{},"f:singular":{}},"f:scope":{},"f:versions":{}},"f:status":{"f:storedVersions":{}}}},{"manager":"kube-apiserver","operation":"Update","apiVersion":"apiextensions.k [truncated 221908 chars]
Warning: oc apply should be used on resource created by either oc create --save-config or oc apply
I1109 19:11:41.741674   12019 request.go:1097] Request Body: {"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apiextensions.k8s.io/v1\",\"kind\":\"CustomResourceDefinition\",\"metadata\":{\"annotations\":{\"controller-gen.kubebuilder.io/version\":\"v0.3.0\"},\"creationTimestamp\":null,\"name\":\"clusterserviceversions.operators.coreos.com\"},\"spec\":{\"group\":\"operators.coreos.com\",\"names\":{\"categories\":[\"olm\"],\"kind\":\"ClusterServiceVersion\",\"listKind\":\"ClusterServiceVersionList\",\"plural\":\"clusterserviceversions\",\"shortNames\":[\"csv\",\"csvs\"],\"singular\":\"clusterserviceversion\"},\"scope\":\"Namespaced\",\"versions\":[{\"additionalPrinterColumns\":[{\"description\":\"The name of the CSV\",\"jsonPath\":\".spec.displayName\",\"name\":\"Display\",\"type\":\"string\"},{\"description\":\"The version of the CSV\",\"jsonPath\":\".spec.version\",\"name\":\"Version\",\"type\":\"string\"},{\"description\":\"The name of a CSV that this one replaces\",\"jsonPath\":\".spec.replaces\",\"name\":\"Replaces\", [truncated 455864 chars]
I1109 19:11:41.741734   12019 round_trippers.go:420] PATCH https://api.knarra1109.qe.devcluster.openshift.com:6443/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com?fieldManager=kubectl-client-side-apply
I1109 19:11:41.741738   12019 round_trippers.go:427] Request Headers:
I1109 19:11:41.741741   12019 round_trippers.go:431]     Content-Type: application/merge-patch+json
I1109 19:11:41.741746   12019 round_trippers.go:431]     User-Agent: oc/4.7.0 (linux/amd64) kubernetes/9f54c1d
I1109 19:11:41.741750   12019 round_trippers.go:431]     Authorization: Bearer <masked>
I1109 19:11:41.741753   12019 round_trippers.go:431]     Accept: application/json
I1109 19:11:48.127176   12019 round_trippers.go:446] Response Status: 200 OK in 6385 milliseconds
I1109 19:11:48.127195   12019 round_trippers.go:449] Response Headers:
I1109 19:11:48.127200   12019 round_trippers.go:452]     Content-Type: application/json
I1109 19:11:48.127222   12019 round_trippers.go:452]     Vary: Accept-Encoding
I1109 19:11:48.127227   12019 round_trippers.go:452]     X-Kubernetes-Pf-Flowschema-Uid: 202c5c05-54a2-4b02-ade2-8f30ecb59d3e
I1109 19:11:48.127246   12019 round_trippers.go:452]     X-Kubernetes-Pf-Prioritylevel-Uid: 1517604c-7276-4328-ad28-1565abb05931
I1109 19:11:48.127251   12019 round_trippers.go:452]     Date: Mon, 09 Nov 2020 13:41:47 GMT
I1109 19:11:48.127256   12019 round_trippers.go:452]     Audit-Id: 2524efe4-e7d1-4abb-9c30-d915ea5d9e41
I1109 19:11:48.127261   12019 round_trippers.go:452]     Cache-Control: no-cache, private
I1109 19:11:49.207545   12019 request.go:1097] Response Body: {"kind":"CustomResourceDefinition","apiVersion":"apiextensions.k8s.io/v1","metadata":{"name":"clusterserviceversions.operators.coreos.com","selfLink":"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com","uid":"0bc8b63f-7424-4650-bf93-b3c9103e8bec","resourceVersion":"78886","generation":2,"creationTimestamp":"2020-11-09T10:11:23Z","annotations":{"controller-gen.kubebuilder.io/version":"v0.3.0","kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apiextensions.k8s.io/v1\",\"kind\":\"CustomResourceDefinition\",\"metadata\":{\"annotations\":{\"controller-gen.kubebuilder.io/version\":\"v0.3.0\"},\"creationTimestamp\":null,\"name\":\"clusterserviceversions.operators.coreos.com\"},\"spec\":{\"group\":\"operators.coreos.com\",\"names\":{\"categories\":[\"olm\"],\"kind\":\"ClusterServiceVersion\",\"listKind\":\"ClusterServiceVersionList\",\"plural\":\"clusterserviceversions\",\"shortNames\":[\"csv\",\"csvs\"],\"singular\":\"clusterserviceversion\"},\"sc [truncated 458122 chars]
customresourcedefinition.apiextensions.k8s.io/clusterserviceversions.operators.coreos.com configured
I1109 19:11:49.214419   12019 apply.go:390] Running apply post-processor function
[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2020-11-08-225909]$ ./oc --loglevel 8 apply -f 0000_50_olm_00-clusterserviceversions.crd.yaml
I1109 19:11:51.461551   12045 loader.go:375] Config loaded from file:  /home/knarra/Downloads/kubeconfig_47
I1109 19:11:51.462176   12045 round_trippers.go:420] GET https://api.knarra1109.qe.devcluster.openshift.com:6443/openapi/v2?timeout=32s
I1109 19:11:51.462181   12045 round_trippers.go:427] Request Headers:
I1109 19:11:51.462184   12045 round_trippers.go:431]     Accept: application/com.github.proto-openapi.spec.v2+protobuf
I1109 19:11:51.462187   12045 round_trippers.go:431]     User-Agent: oc/4.7.0 (linux/amd64) kubernetes/9f54c1d
I1109 19:11:51.462190   12045 round_trippers.go:431]     Authorization: Bearer <masked>
I1109 19:11:57.351712   12045 round_trippers.go:446] Response Status: 200 OK in 5889 milliseconds
I1109 19:11:57.351728   12045 round_trippers.go:449] Response Headers:
I1109 19:11:57.351734   12045 round_trippers.go:452]     X-Varied-Accept: application/com.github.proto-openapi.spec.v2+protobuf
I1109 19:11:57.351738   12045 round_trippers.go:452]     Accept-Ranges: bytes
I1109 19:11:57.351742   12045 round_trippers.go:452]     Content-Type: application/octet-stream
I1109 19:11:57.351746   12045 round_trippers.go:452]     Etag: "A85F76D2788113642E47D3786CBC0A9D007BF06CE007706AA0C4FE14A8DD9C8728A17057ABFFA813614446BF24F5468C30A4C9F85422948D00A0474E8621D54D"
I1109 19:11:57.351751   12045 round_trippers.go:452]     Last-Modified: Mon, 09 Nov 2020 13:41:52 GMT
I1109 19:11:57.351756   12045 round_trippers.go:452]     X-Kubernetes-Pf-Flowschema-Uid: 202c5c05-54a2-4b02-ade2-8f30ecb59d3e
I1109 19:11:57.351760   12045 round_trippers.go:452]     X-Kubernetes-Pf-Prioritylevel-Uid: 1517604c-7276-4328-ad28-1565abb05931
I1109 19:11:57.351764   12045 round_trippers.go:452]     Audit-Id: e4ff55e2-5386-4c12-b925-8823710d52a5
I1109 19:11:57.351769   12045 round_trippers.go:452]     Cache-Control: no-cache, private
I1109 19:11:57.351772   12045 round_trippers.go:452]     Vary: Accept-Encoding
I1109 19:11:57.351776   12045 round_trippers.go:452]     Vary: Accept
I1109 19:11:57.351779   12045 round_trippers.go:452]     Date: Mon, 09 Nov 2020 13:41:56 GMT
E1109 19:12:23.462574   12045 request.go:1001] Unexpected error when reading response body: context deadline exceeded (Client.Timeout or context cancellation while reading body)
I1109 19:12:23.492906   12045 round_trippers.go:420] GET https://api.knarra1109.qe.devcluster.openshift.com:6443/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com
I1109 19:12:23.492923   12045 round_trippers.go:427] Request Headers:
I1109 19:12:23.492927   12045 round_trippers.go:431]     Authorization: Bearer <masked>
I1109 19:12:23.492930   12045 round_trippers.go:431]     Accept: application/json
I1109 19:12:23.492932   12045 round_trippers.go:431]     User-Agent: oc/4.7.0 (linux/amd64) kubernetes/9f54c1d
I1109 19:12:26.830988   12045 round_trippers.go:446] Response Status: 200 OK in 3338 milliseconds
I1109 19:12:26.831004   12045 round_trippers.go:449] Response Headers:
I1109 19:12:26.831008   12045 round_trippers.go:452]     X-Kubernetes-Pf-Flowschema-Uid: 202c5c05-54a2-4b02-ade2-8f30ecb59d3e
I1109 19:12:26.831012   12045 round_trippers.go:452]     X-Kubernetes-Pf-Prioritylevel-Uid: 1517604c-7276-4328-ad28-1565abb05931
I1109 19:12:26.831014   12045 round_trippers.go:452]     Date: Mon, 09 Nov 2020 13:42:23 GMT
I1109 19:12:26.831018   12045 round_trippers.go:452]     Audit-Id: 35ddb16e-052e-45d9-a5e8-75a6c9b18bc1
I1109 19:12:26.831021   12045 round_trippers.go:452]     Cache-Control: no-cache, private
I1109 19:12:26.831024   12045 round_trippers.go:452]     Content-Type: application/json
I1109 19:12:26.831027   12045 round_trippers.go:452]     Vary: Accept-Encoding
I1109 19:12:30.372487   12045 request.go:1097] Response Body: {"kind":"CustomResourceDefinition","apiVersion":"apiextensions.k8s.io/v1","metadata":{"name":"clusterserviceversions.operators.coreos.com","selfLink":"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com","uid":"0bc8b63f-7424-4650-bf93-b3c9103e8bec","resourceVersion":"79023","generation":3,"creationTimestamp":"2020-11-09T10:11:23Z","annotations":{"controller-gen.kubebuilder.io/version":"v0.3.0","kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apiextensions.k8s.io/v1\",\"kind\":\"CustomResourceDefinition\",\"metadata\":{\"annotations\":{\"controller-gen.kubebuilder.io/version\":\"v0.3.0\"},\"creationTimestamp\":null,\"name\":\"clusterserviceversions.operators.coreos.com\"},\"spec\":{\"group\":\"operators.coreos.com\",\"names\":{\"categories\":[\"olm\"],\"kind\":\"ClusterServiceVersion\",\"listKind\":\"ClusterServiceVersionList\",\"plural\":\"clusterserviceversions\",\"shortNames\":[\"csv\",\"csvs\"],\"singular\":\"clusterserviceversion\"},\"sc [truncated 458429 chars]
I1109 19:12:30.476594   12045 request.go:1097] Request Body: {"spec":{"versions":[{"additionalPrinterColumns":[{"description":"The name of the CSV","jsonPath":".spec.displayName","name":"Display","type":"string"},{"description":"The version of the CSV","jsonPath":".spec.version","name":"Version","type":"string"},{"description":"The name of a CSV that this one replaces","jsonPath":".spec.replaces","name":"Replaces","type":"string"},{"jsonPath":".status.phase","name":"Phase","type":"string"}],"name":"v1alpha1","schema":{"openAPIV3Schema":{"description":"ClusterServiceVersion is a Custom Resource of type `ClusterServiceVersionSpec`.","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object repres [truncated 219580 chars]
I1109 19:12:30.476661   12045 round_trippers.go:420] PATCH https://api.knarra1109.qe.devcluster.openshift.com:6443/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com?fieldManager=kubectl-client-side-apply
I1109 19:12:30.476666   12045 round_trippers.go:427] Request Headers:
I1109 19:12:30.476671   12045 round_trippers.go:431]     Authorization: Bearer <masked>
I1109 19:12:30.476675   12045 round_trippers.go:431]     Accept: application/json
I1109 19:12:30.476696   12045 round_trippers.go:431]     Content-Type: application/merge-patch+json
I1109 19:12:30.476700   12045 round_trippers.go:431]     User-Agent: oc/4.7.0 (linux/amd64) kubernetes/9f54c1d
I1109 19:12:35.810922   12045 round_trippers.go:446] Response Status: 200 OK in 5334 milliseconds
I1109 19:12:35.810936   12045 round_trippers.go:449] Response Headers:
I1109 19:12:35.810940   12045 round_trippers.go:452]     X-Kubernetes-Pf-Flowschema-Uid: 202c5c05-54a2-4b02-ade2-8f30ecb59d3e
I1109 19:12:35.810943   12045 round_trippers.go:452]     X-Kubernetes-Pf-Prioritylevel-Uid: 1517604c-7276-4328-ad28-1565abb05931
I1109 19:12:35.810945   12045 round_trippers.go:452]     Date: Mon, 09 Nov 2020 13:42:35 GMT
I1109 19:12:35.810947   12045 round_trippers.go:452]     Audit-Id: 38687b5c-1b1b-483a-91be-f029a5311035
I1109 19:12:35.810950   12045 round_trippers.go:452]     Cache-Control: no-cache, private
I1109 19:12:35.810952   12045 round_trippers.go:452]     Content-Type: application/json
I1109 19:12:35.810954   12045 round_trippers.go:452]     Vary: Accept-Encoding
I1109 19:12:46.633881   12045 request.go:1097] Response Body: {"kind":"CustomResourceDefinition","apiVersion":"apiextensions.k8s.io/v1","metadata":{"name":"clusterserviceversions.operators.coreos.com","selfLink":"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com","uid":"0bc8b63f-7424-4650-bf93-b3c9103e8bec","resourceVersion":"79122","generation":4,"creationTimestamp":"2020-11-09T10:11:23Z","annotations":{"controller-gen.kubebuilder.io/version":"v0.3.0","kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apiextensions.k8s.io/v1\",\"kind\":\"CustomResourceDefinition\",\"metadata\":{\"annotations\":{\"controller-gen.kubebuilder.io/version\":\"v0.3.0\"},\"creationTimestamp\":null,\"name\":\"clusterserviceversions.operators.coreos.com\"},\"spec\":{\"group\":\"operators.coreos.com\",\"names\":{\"categories\":[\"olm\"],\"kind\":\"ClusterServiceVersion\",\"listKind\":\"ClusterServiceVersionList\",\"plural\":\"clusterserviceversions\",\"shortNames\":[\"csv\",\"csvs\"],\"singular\":\"clusterserviceversion\"},\"sc [truncated 458122 chars]
customresourcedefinition.apiextensions.k8s.io/clusterserviceversions.operators.coreos.com configured
I1109 19:12:46.639320   12045 apply.go:390] Running apply post-processor function
[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2020-11-08-225909]$ ./oc --loglevel 8 apply -f 0000_50_olm_00-clusterserviceversions.crd.yaml
I1109 19:12:55.600182   12114 loader.go:375] Config loaded from file:  /home/knarra/Downloads/kubeconfig_47
I1109 19:12:55.600774   12114 round_trippers.go:420] GET https://api.knarra1109.qe.devcluster.openshift.com:6443/openapi/v2?timeout=32s
I1109 19:12:55.600780   12114 round_trippers.go:427] Request Headers:
I1109 19:12:55.600782   12114 round_trippers.go:431]     Accept: application/com.github.proto-openapi.spec.v2+protobuf
I1109 19:12:55.600785   12114 round_trippers.go:431]     User-Agent: oc/4.7.0 (linux/amd64) kubernetes/9f54c1d
I1109 19:12:55.600788   12114 round_trippers.go:431]     Authorization: Bearer <masked>
I1109 19:12:57.219528   12114 round_trippers.go:446] Response Status: 200 OK in 1618 milliseconds
I1109 19:12:57.219544   12114 round_trippers.go:449] Response Headers:
I1109 19:12:57.219548   12114 round_trippers.go:452]     Accept-Ranges: bytes
I1109 19:12:57.219551   12114 round_trippers.go:452]     Cache-Control: no-cache, private
I1109 19:12:57.219554   12114 round_trippers.go:452]     Content-Type: application/octet-stream
I1109 19:12:57.219556   12114 round_trippers.go:452]     X-Kubernetes-Pf-Flowschema-Uid: 202c5c05-54a2-4b02-ade2-8f30ecb59d3e
I1109 19:12:57.219559   12114 round_trippers.go:452]     X-Kubernetes-Pf-Prioritylevel-Uid: 1517604c-7276-4328-ad28-1565abb05931
I1109 19:12:57.219562   12114 round_trippers.go:452]     Audit-Id: d3876b69-bee6-42c9-b87c-9112444afa0a
I1109 19:12:57.219564   12114 round_trippers.go:452]     Etag: "31BCD824419712B67F395B83AABF97584195BE57C27D7206DD012381242A141840AF8D52FECE53BA31D6B4DB3F29FFE1ECB848972F6BE0686D3B9C4A53F1A80A"
I1109 19:12:57.219569   12114 round_trippers.go:452]     Last-Modified: Mon, 09 Nov 2020 13:42:39 GMT
I1109 19:12:57.219571   12114 round_trippers.go:452]     Vary: Accept-Encoding
I1109 19:12:57.219574   12114 round_trippers.go:452]     Vary: Accept
I1109 19:12:57.219578   12114 round_trippers.go:452]     Date: Mon, 09 Nov 2020 13:42:57 GMT
I1109 19:12:57.219580   12114 round_trippers.go:452]     X-Varied-Accept: application/com.github.proto-openapi.spec.v2+protobuf
I1109 19:13:02.234474   12114 request.go:1095] Response Body:
00000000  0a 03 32 2e 30 12 1d 0a  0a 4b 75 62 65 72 6e 65  |..2.0....Kuberne|
00000010  74 65 73 12 0f 76 31 2e  31 39 2e 30 2b 65 36 37  |tes..v1.19.0+e67|
00000020  66 35 64 63 42 b7 a3 bc  02 12 80 03 0a 1c 2f 61  |f5dcB........./a|
00000030  70 69 73 2f 6f 61 75 74  68 2e 6f 70 65 6e 73 68  |pis/oauth.opensh|
00000040  69 66 74 2e 69 6f 2f 76  31 2f 12 df 02 12 dc 02  |ift.io/v1/......|
00000050  0a 13 6f 61 75 74 68 4f  70 65 6e 73 68 69 66 74  |..oauthOpenshift|
00000060  49 6f 5f 76 31 1a 17 67  65 74 20 61 76 61 69 6c  |Io_v1..get avail|
00000070  61 62 6c 65 20 72 65 73  6f 75 72 63 65 73 2a 21  |able resources*!|
00000080  67 65 74 4f 61 75 74 68  4f 70 65 6e 73 68 69 66  |getOauthOpenshif|
00000090  74 49 6f 56 31 41 50 49  52 65 73 6f 75 72 63 65  |tIoV1APIResource|
000000a0  73 32 10 61 70 70 6c 69  63 61 74 69 6f 6e 2f 6a  |s2.application/j|
000000b0  73 6f 6e 32 10 61 70 70  6c 69 63 61 74 69 6f 6e  |son2.application|
000000c0  2f 79 61 6d 6c 32 23 61  70 70 6c 69 63 61 74 69  |/yaml2#applicat [truncated 37021055 chars]
I1109 19:13:02.346865   12114 round_trippers.go:420] GET https://api.knarra1109.qe.devcluster.openshift.com:6443/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com
I1109 19:13:02.346881   12114 round_trippers.go:427] Request Headers:
I1109 19:13:02.346884   12114 round_trippers.go:431]     Accept: application/json
I1109 19:13:02.346887   12114 round_trippers.go:431]     User-Agent: oc/4.7.0 (linux/amd64) kubernetes/9f54c1d
I1109 19:13:02.346890   12114 round_trippers.go:431]     Authorization: Bearer <masked>
I1109 19:13:02.756675   12114 round_trippers.go:446] Response Status: 200 OK in 409 milliseconds
I1109 19:13:02.756691   12114 round_trippers.go:449] Response Headers:
I1109 19:13:02.756697   12114 round_trippers.go:452]     Vary: Accept-Encoding
I1109 19:13:02.756701   12114 round_trippers.go:452]     X-Kubernetes-Pf-Flowschema-Uid: 202c5c05-54a2-4b02-ade2-8f30ecb59d3e
I1109 19:13:02.756704   12114 round_trippers.go:452]     X-Kubernetes-Pf-Prioritylevel-Uid: 1517604c-7276-4328-ad28-1565abb05931
I1109 19:13:02.756706   12114 round_trippers.go:452]     Date: Mon, 09 Nov 2020 13:43:02 GMT
I1109 19:13:02.756709   12114 round_trippers.go:452]     Audit-Id: 203ea072-29df-4c9a-b17e-800ea9890f25
I1109 19:13:02.756712   12114 round_trippers.go:452]     Cache-Control: no-cache, private
I1109 19:13:02.756714   12114 round_trippers.go:452]     Content-Type: application/json
I1109 19:13:03.140004   12114 request.go:1097] Response Body: {"kind":"CustomResourceDefinition","apiVersion":"apiextensions.k8s.io/v1","metadata":{"name":"clusterserviceversions.operators.coreos.com","selfLink":"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com","uid":"0bc8b63f-7424-4650-bf93-b3c9103e8bec","resourceVersion":"79122","generation":4,"creationTimestamp":"2020-11-09T10:11:23Z","annotations":{"controller-gen.kubebuilder.io/version":"v0.3.0","kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apiextensions.k8s.io/v1\",\"kind\":\"CustomResourceDefinition\",\"metadata\":{\"annotations\":{\"controller-gen.kubebuilder.io/version\":\"v0.3.0\"},\"creationTimestamp\":null,\"name\":\"clusterserviceversions.operators.coreos.com\"},\"spec\":{\"group\":\"operators.coreos.com\",\"names\":{\"categories\":[\"olm\"],\"kind\":\"ClusterServiceVersion\",\"listKind\":\"ClusterServiceVersionList\",\"plural\":\"clusterserviceversions\",\"shortNames\":[\"csv\",\"csvs\"],\"singular\":\"clusterserviceversion\"},\"sc [truncated 458122 chars]
I1109 19:13:03.164265   12114 request.go:1097] Request Body: {"status":{"acceptedNames":{"kind":"","plural":""},"conditions":[],"storedVersions":[]}}
I1109 19:13:03.164312   12114 round_trippers.go:420] PATCH https://api.knarra1109.qe.devcluster.openshift.com:6443/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com?fieldManager=kubectl-client-side-apply
I1109 19:13:03.164316   12114 round_trippers.go:427] Request Headers:
I1109 19:13:03.164319   12114 round_trippers.go:431]     Accept: application/json
I1109 19:13:03.164323   12114 round_trippers.go:431]     Content-Type: application/merge-patch+json
I1109 19:13:03.164346   12114 round_trippers.go:431]     User-Agent: oc/4.7.0 (linux/amd64) kubernetes/9f54c1d
I1109 19:13:03.164351   12114 round_trippers.go:431]     Authorization: Bearer <masked>
I1109 19:13:03.678476   12114 round_trippers.go:446] Response Status: 200 OK in 514 milliseconds
I1109 19:13:03.678495   12114 round_trippers.go:449] Response Headers:
I1109 19:13:03.678498   12114 round_trippers.go:452]     Date: Mon, 09 Nov 2020 13:43:03 GMT
I1109 19:13:03.678503   12114 round_trippers.go:452]     Audit-Id: 72832cf9-3670-4e10-ae06-d65bdb689563
I1109 19:13:03.678508   12114 round_trippers.go:452]     Cache-Control: no-cache, private
I1109 19:13:03.678510   12114 round_trippers.go:452]     Content-Type: application/json
I1109 19:13:03.678513   12114 round_trippers.go:452]     Vary: Accept-Encoding
I1109 19:13:03.678516   12114 round_trippers.go:452]     X-Kubernetes-Pf-Flowschema-Uid: 202c5c05-54a2-4b02-ade2-8f30ecb59d3e
I1109 19:13:03.678520   12114 round_trippers.go:452]     X-Kubernetes-Pf-Prioritylevel-Uid: 1517604c-7276-4328-ad28-1565abb05931
I1109 19:13:03.698983   12114 request.go:1097] Response Body: {"kind":"CustomResourceDefinition","apiVersion":"apiextensions.k8s.io/v1","metadata":{"name":"clusterserviceversions.operators.coreos.com","selfLink":"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusterserviceversions.operators.coreos.com","uid":"0bc8b63f-7424-4650-bf93-b3c9103e8bec","resourceVersion":"79231","generation":4,"creationTimestamp":"2020-11-09T10:11:23Z","annotations":{"controller-gen.kubebuilder.io/version":"v0.3.0","kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apiextensions.k8s.io/v1\",\"kind\":\"CustomResourceDefinition\",\"metadata\":{\"annotations\":{\"controller-gen.kubebuilder.io/version\":\"v0.3.0\"},\"creationTimestamp\":null,\"name\":\"clusterserviceversions.operators.coreos.com\"},\"spec\":{\"group\":\"operators.coreos.com\",\"names\":{\"categories\":[\"olm\"],\"kind\":\"ClusterServiceVersion\",\"listKind\":\"ClusterServiceVersionList\",\"plural\":\"clusterserviceversions\",\"shortNames\":[\"csv\",\"csvs\"],\"singular\":\"clusterserviceversion\"},\"sc [truncated 458122 chars]
customresourcedefinition.apiextensions.k8s.io/clusterserviceversions.operators.coreos.com configured
I1109 19:13:03.707079   12114 apply.go:390] Running apply post-processor function


Based on the above moving bug to verified state.

Comment 23 errata-xmlrpc 2021-02-24 15:13:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.