Description of problem (please be detailed as possible and provide log snippests): When deploying the OCS 4.6 on IBM Cloud ROKS on OCP 4.5 when creating storageCluster CR I see that nothing happen and in ocs-operator log I see those errors: {"level":"error","ts":"2020-10-14T21:19:01.380Z","logger":"controller_storagecluster","msg":"failed to 'Get' SnapshotClass \"ocs-storagecluster-rbdplugin-snapclass\"","Request.Namespace":"openshift-storage","Request.Name":"ocs-storagecluster","error":"no matches for kind \"VolumeSnapshotClass\" in version \"snapshot.storage.k8s.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/remote-source/app/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/openshift/ocs-operator/pkg/controller/storagecluster.(*ReconcileStorageCluster).createSnapshotClasses\n\t/remote-source/app/pkg/controller/storagecluster/volumesnapshotterclasses.go:73\ngithub.com/openshift/ocs-operator/pkg/controller/storagecluster.(*ReconcileStorageCluster).ensureSnapshotClasses\n\t/remote-source/app/pkg/controller/storagecluster/volumesnapshotterclasses.go:102\ngithub.com/openshift/ocs-operator/pkg/controller/storagecluster.(*ReconcileStorageCluster).Reconcile\n\t/remote-source/app/pkg/controller/storagecluster/reconcile.go:231\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:256\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} {"level":"error","ts":"2020-10-14T21:19:01.435Z","logger":"controller-runtime.controller","msg":"Reconciler error","controller":"storagecluster-controller","request":"openshift-storage/ocs-storagecluster","error":"no matches for kind \"VolumeSnapshotClass\" in version \"snapshot.storage.k8s.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/remote-source/app/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:258\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} {"level":"info","ts":"2020-10-14T21:19:02.436Z","logger":"controller_storagecluster","msg":"Reconciling StorageCluster","Request.Namespace":"openshift-storage","Request.Name":"ocs-storagecluster"} Version of all relevant components (if applicable): OCP Server Version: 4.5.13 4.6.0-131.ci Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? Yes, cannot deploy on ROKS cluster Is there any workaround available to the best of your knowledge? No Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 1 Can this issue reproducible? Yes Can this issue reproduce from the UI? Haven't tried If this is a regression, please provide more details to justify this: Steps to Reproduce: 1. Instell OCS on ROKS OCP 4.5 cluster 2. After creating storageCluster CR nothing happens 3. In the ocs-operator logs I see the error mentioned above Actual results: No deployment success Expected results: After create storageCluster see starting spinning up all needed pods Additional info:
Must gather: http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/pbalogh-ind2/pbalogh-ind2_20200618T133943/logs/bz-logs/
Madhu, PTAL
As a workaround this helped to create CRDs from: https://github.com/kubernetes-csi/external-snapshotter/tree/e562c2df7427d9e28ed170ccd9a0aad06717f061/client/config/crd oc create -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/e562c2df7427d9e28ed170ccd9a0aad06717f061/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml oc create -f oc create -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/e562c2df7427d9e28ed170ccd9a0aad06717f061/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml oc create -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/e562c2df7427d9e28ed170ccd9a0aad06717f061/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
Petr, it is more of a configuration issue specific to this platform. We have a workaround also, can we remove the blocker flag?
Hey Mudit, maybe we can do the clone of this BZ to the OCP and keep this as a tracker for OCS? We need to discuss with IBM guys and confirm with them, and maybe Eran if this is the blocker for them for 4.6 release. Meanwhile I am running OCP 4.5 deployment job with OCS 4.6 here: https://ocs4-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/qe-deploy-ocs-cluster/13544/ To confirm that we don't have the same issue in other platform as well. So at this stage I am not the one who can decide about removing blocker flag as of now.
Akash, will these configurations mentioned in Comment 7 be taken care of as part of ROKS installer?
Sahina, Petr: I have raised an internal ticket within IBM Cloud to created then 3 CRDs of external-snapshotter in ROKS 4.5 version onwards. Will keep posted on the updates from that ticket.
Since this is just a tracker and there's nothing for us to do in OCS 4.6, moving this to OCS 4.7.
After the discussion with IBM cloud guys we can close this as not a bug as this will be handled in IBM Cloud OCP deployment automation and it's not OCS bug.