Description of problem: Deploy local volume on OCP 4.3 cluster by following https://github.com/openshift/local-storage-operator/blob/master/docs/deploy-with-olm.md, the local-storage-operator is throwing RBAC permission issue. Version-Release number of selected component (if applicable): 4.3.0-0.nightly-2019-10-28-222147 How reproducible: Always Steps to Reproduce: 1. Following https://github.com/openshift/local-storage-operator/blob/master/docs/deploy-with-olm.md 2. After CR is created, check all in namespace local-storage 3. Actual results: There are no storage class and PV created, even no provisioner and disk-maker pod exist/running. $ oc get all NAME READY STATUS RESTARTS AGE pod/local-storage-operator-7759795457-9mh7r 1/1 Running 0 15h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/local-storage-operator ClusterIP 172.30.166.181 <none> 60000/TCP 15h NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/local-storage-operator 1/1 1 1 15h NAME DESIRED CURRENT READY AGE replicaset.apps/local-storage-operator-7759795457 1 1 1 15h Expected results: The local volume is functioning. Additional info: $ oc logs pod/local-storage-operator-7759795457-9mh7r I1029 11:19:53.682709 1 main.go:18] Go Version: go1.10.8 I1029 11:19:53.683050 1 main.go:19] Go OS/Arch: linux/amd64 I1029 11:19:53.683074 1 main.go:20] operator-sdk Version: 0.0.7 time="2019-10-29T11:19:53Z" level=info msg="Metrics service local-storage-operator created" I1029 11:19:53.861347 1 main.go:36] Watching local.storage.openshift.io/v1, LocalVolume I1029 11:19:53.861356 1 main.go:41] Watching local.storage.openshift.io/v1, LocalVolume, local-storage, 180000000000 E1029 22:07:53.986662 1 memcache.go:147] couldn't get resource list for packages.operators.coreos.com/v1: the server is currently unable to handle the request E1029 22:08:43.372004 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=95079, ErrCode=NO_ERROR, debug="" W1029 22:08:43.797110 1 reflector.go:341] github.com/openshift/local-storage-operator/vendor/github.com/operator-framework/operator-sdk/pkg/sdk/informer.go:91: watch of *unstructured.Unstructured ended with: unexpected object: &{map[apiVersion:v1 metadata:map[] status:Failure message:too old resource version: 143222 (345197) reason:Gone code:410 kind:Status]} E1029 22:08:53.957243 1 memcache.go:147] couldn't get resource list for packages.operators.coreos.com/v1: the server is currently unable to handle the request E1029 22:12:31.513069 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=589, ErrCode=NO_ERROR, debug="" W1029 22:12:31.814700 1 reflector.go:341] github.com/openshift/local-storage-operator/vendor/github.com/operator-framework/operator-sdk/pkg/sdk/informer.go:91: watch of *unstructured.Unstructured ended with: unexpected object: &{map[metadata:map[] status:Failure message:too old resource version: 345197 (345967) reason:Gone code:410 kind:Status apiVersion:v1]} E1030 02:56:16.995443 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=41547, ErrCode=NO_ERROR, debug="" W1030 02:56:17.111693 1 reflector.go:341] github.com/openshift/local-storage-operator/vendor/github.com/operator-framework/operator-sdk/pkg/sdk/informer.go:91: watch of *unstructured.Unstructured ended with: unexpected object: &{map[status:Failure message:too old resource version: 345967 (346702) reason:Gone code:410 kind:Status apiVersion:v1 metadata:map[]]} W1030 02:58:12.819994 1 reflector.go:341] github.com/openshift/local-storage-operator/vendor/github.com/operator-framework/operator-sdk/pkg/sdk/informer.go:91: watch of *unstructured.Unstructured ended with: unexpected object: &{map[code:410 kind:Status apiVersion:v1 metadata:map[] status:Failure message:too old resource version: 346702 (434280) reason:Gone]} I1030 03:07:37.704347 1 api_updater.go:75] Updating localvolume local-storage/local-disks E1030 03:07:37.871331 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]} time="2019-10-30T03:07:37Z" level=error msg="error syncing key (local-storage/local-disks): error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io \"local-storage-provisioner-pv-binding\" is forbidden: user \"system:serviceaccount:local-storage:local-storage-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:local-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"events.k8s.io\"], Resources:[\"events\"], Verbs:[\"create\" \"patch\" \"update\"]}" E1030 03:07:38.005282 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]} time="2019-10-30T03:07:38Z" level=error msg="error syncing key (local-storage/local-disks): error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io \"local-storage-provisioner-pv-binding\" is forbidden: user \"system:serviceaccount:local-storage:local-storage-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:local-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"events.k8s.io\"], Resources:[\"events\"], Verbs:[\"create\" \"patch\" \"update\"]}" E1030 03:07:38.187667 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]} time="2019-10-30T03:07:38Z" level=error msg="error syncing key (local-storage/local-disks): error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io \"local-storage-provisioner-pv-binding\" is forbidden: user \"system:serviceaccount:local-storage:local-storage-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:local-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"events.k8s.io\"], Resources:[\"events\"], Verbs:[\"create\" \"patch\" \"update\"]}" E1030 03:07:38.978857 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]} time="2019-10-30T03:07:38Z" level=error msg="error syncing key (local-storage/local-disks): error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io \"local-storage-provisioner-pv-binding\" is forbidden: user \"system:serviceaccount:local-storage:local-storage-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:local-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"events.k8s.io\"], Resources:[\"events\"], Verbs:[\"create\" \"patch\" \"update\"]}" E1030 03:07:39.785121 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]} time="2019-10-30T03:07:39Z" level=error msg="error syncing key (local-storage/local-disks): error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io \"local-storage-provisioner-pv-binding\" is forbidden: user \"system:serviceaccount:local-storage:local-storage-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:local-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"events.k8s.io\"], Resources:[\"events\"], Verbs:[\"create\" \"patch\" \"update\"]}" E1030 03:07:40.581366 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]} time="2019-10-30T03:07:40Z" level=error msg="error syncing key (local-storage/local-disks): error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io \"local-storage-provisioner-pv-binding\" is forbidden: user \"system:serviceaccount:local-storage:local-storage-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:local-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"events.k8s.io\"], Resources:[\"events\"], Verbs:[\"create\" \"patch\" \"update\"]}" E1030 03:07:41.380715 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]} time="2019-10-30T03:07:41Z" level=error msg="error syncing key (local-storage/local-disks): error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io \"local-storage-provisioner-pv-binding\" is forbidden: user \"system:serviceaccount:local-storage:local-storage-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:local-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"events.k8s.io\"], Resources:[\"events\"], Verbs:[\"create\" \"patch\" \"update\"]}" E1030 03:07:42.179644 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]} time="2019-10-30T03:07:42Z" level=error msg="error syncing key (local-storage/local-disks): error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io \"local-storage-provisioner-pv-binding\" is forbidden: user \"system:serviceaccount:local-storage:local-storage-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:local-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"events.k8s.io\"], Resources:[\"events\"], Verbs:[\"create\" \"patch\" \"update\"]}" E1030 03:07:42.982532 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]} time="2019-10-30T03:07:42Z" level=error msg="error syncing key (local-storage/local-disks): error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io \"local-storage-provisioner-pv-binding\" is forbidden: user \"system:serviceaccount:local-storage:local-storage-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:local-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"events.k8s.io\"], Resources:[\"events\"], Verbs:[\"create\" \"patch\" \"update\"]}" E1030 03:07:43.784049 1 controller.go:135] error applying pv cluster role binding local-storage-provisioner-pv-binding: clusterrolebindings.rbac.authorization.k8s.io "local-storage-provisioner-pv-binding" is forbidden: user "system:serviceaccount:local-storage:local-storage-operator" (groups=["system:serviceaccounts" "system:serviceaccounts:local-storage" "system:authenticated"]) is attempting to grant RBAC permissions not currently held: {APIGroups:["events.k8s.io"], Resources:["events"], Verbs:["create" "patch" "update"]}
Tested with 4.3.0-0.nightly-2019-11-10-185106 using quay.io/hekumar as app registry, no RBAC issue now. $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.3.0-0.nightly-2019-11-10-185106 True False 5h4m Cluster version is 4.3.0-0.nightly-2019-11-10-185106 $ oc get all -n local-storage NAME READY STATUS RESTARTS AGE pod/local-disks-local-diskmaker-b442v 1/1 Running 0 3m58s pod/local-disks-local-diskmaker-zxmz8 1/1 Running 0 3m58s pod/local-disks-local-provisioner-df62f 1/1 Running 0 3m58s pod/local-disks-local-provisioner-kgb6d 1/1 Running 0 3m58s pod/local-storage-operator-b6b7c94b6-jwjzh 1/1 Running 0 37m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/local-storage-operator ClusterIP 172.30.117.98 <none> 60000/TCP 37m NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/local-disks-local-diskmaker 2 2 2 2 2 <none> 3m59s daemonset.apps/local-disks-local-provisioner 2 2 2 2 2 <none> 3m59s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/local-storage-operator 1/1 1 1 37m NAME DESIRED CURRENT READY AGE replicaset.apps/local-storage-operator-b6b7c94b6 1 1 1 37m $ oc get sc,pv NAME PROVISIONER AGE storageclass.storage.k8s.io/local-sc kubernetes.io/no-provisioner 4m44s storageclass.storage.k8s.io/standard (default) kubernetes.io/gce-pd 5h16m NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/local-pv-cc199c3e 10Gi RWO Delete Available local-sc 4m20s $ pwd ; git diff catalog-create-subscribe.yaml /home/lxia/github.com/local-storage-operator/examples/olm diff --git a/examples/olm/catalog-create-subscribe.yaml b/examples/olm/catalog-create-subscribe.yaml index 68c088c..1ca7fb2 100644 --- a/examples/olm/catalog-create-subscribe.yaml +++ b/examples/olm/catalog-create-subscribe.yaml @@ -19,7 +19,7 @@ metadata: namespace: openshift-marketplace spec: type: appregistry - endpoint: https://quay.io/cnr + endpoint: https://quay.io/hekumar registryNamespace: hekumar displayName: "Red Hat Storage operators" publisher: "Red Hat Storage"
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0062