Description of problem (please be detailed as possible and provide log
snippests):
When upgrading from 4.6 internal build to 4.7 build we see CSV in failed state:
NAME                         DISPLAY                       VERSION        REPLACES                     PHASE
ocs-operator.v4.6.0-156.ci   OpenShift Container Storage   4.6.0-156.ci                                Replacing
ocs-operator.v4.7.0-158.ci   OpenShift Container Storage   4.7.0-158.ci   ocs-operator.v4.6.0-156.ci   Failed

From operator logs I see a lot of this errors:
2020-11-10T02:28:25.853965298Z {"level":"error","ts":"2020-11-10T02:28:25.853Z","logger":"controller_storagecluster","msg":"prometheus rules file not found","error":"'/ocs-prometheus-rules/prometheus-ocs-rules.yaml' not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/remote-source/deps/gomod/pkg/mod/github.com/go-logr/zapr.1/zapr.go:128\ngithub.com/openshift/ocs-operator/pkg/controller/storagecluster.(*ReconcileStorageCluster).enablePrometheusRules\n\t/remote-source/app/pkg/controller/storagecluster/prometheus.go:29\ngithub.com/openshift/ocs-operator/pkg/controller/storagecluster.(*ReconcileStorageCluster).Reconcile\n\t/remote-source/app/pkg/controller/storagecluster/reconcile.go:359\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:256\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.4/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.4/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.4/pkg/util/wait/wait.go:88"}
2020-11-10T02:28:25.854009133Z {"level":"error","ts":"2020-11-10T02:28:25.853Z","logger":"controller_storagecluster","msg":"unable to deploy Prometheus rules","error":"failed while creating PrometheusRule: expected pointer, but got nil","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/remote-source/deps/gomod/pkg/mod/github.com/go-logr/zapr.1/zapr.go:128\ngithub.com/openshift/ocs-operator/pkg/controller/storagecluster.(*ReconcileStorageCluster).enablePrometheusRules\n\t/remote-source/app/pkg/controller/storagecluster/prometheus.go:33\ngithub.com/openshift/ocs-operator/pkg/controller/storagecluster.(*ReconcileStorageCluster).Reconcile\n\t/remote-source/app/pkg/controller/storagecluster/reconcile.go:359\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:256\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.4/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.4/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.4/pkg/util/wait/wait.go:88"}




Version of all relevant components (if applicable):
OCS:  4.6.0-156.ci  upgrade to: 4.7.0-158.ci

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Yep, it's blocking upgrade to new Y version.


Is there any workaround available to the best of your knowledge?
No

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue reproducible?
Haven't tried yet


Can this issue reproduce from the UI?
Haven't tried.


If this is a regression, please provide more details to justify this:
Yes, this worked before.


Steps to Reproduce:
1. Install OCS 4.6 internal build mentioned above
2. Upgrade to 4.7 internal build
3. Upgrade will not complete and fail


Actual results:
Upgrade failed


Expected results:
Have successful upgrade


Additional info:
Must gather:
http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/j006vu1cs33-uan/j006vu1cs33-uan_20201109T221607/logs/failed_testcase_ocs_logs_1604963919/test_upgrade_ocs_logs/

Job:
https://ocs4-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/qe-deploy-ocs-cluster/14430/consoleFull