Bug 2257694

Description of problem (please be detailed as possible and provide log
snippests):

This BZ is raised to address following ocs-metrics-exporter issues (as noted by Umanga in BZ#2255036):

1. ocs-metrics-exporter deployed on `openshift-extended-storage` namespace does not have access to collect PV data. It causes some PV metrics to be missing.
```
{"level":"error","ts":1704795344.3449461,"caller":"cache/reflector.go:147","msg":"/remote-source/app/metrics/internal/collectors/registry.go:72: Failed to watch *v1.PersistentVolume: unable to sync list result: failed to initialize ceph: failed to get client key from secret in namespace \"openshift-storage-extended\""}
```

2. ocs-metrics-exporter deployed on `openshift-extended-storage` namespace can not find CSI config required to connect to Ceph to retrieve data. It causes metrics depending on Ceph commands to be missing.
   We might need to disable this exporter for external mode since we can not assume access to external clusters to execute ceph commands.
```
{"level":"info","ts":1704795373.87813,"caller":"cache/reflector.go:458","msg":"/remote-source/app/metrics/internal/collectors/registry.go:95: watch of *v1.CephBlockPool ended with: failed to initialize ceph: expected 1 or more CSI cluster config but found 0 from configmap in namespace \"openshift-storage-extended\""}
```

3. Some Prometheus query functions are not working as expected. There are multiple jobs exporting metrics. These should be considered when updating queries.

Above errors are part of the issues faced in populating 'openshift-storage-extended' namespace, thus blocking BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2255036

Version of all relevant components (if applicable):


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
NA

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1.
2.
3.


Actual results:
ocs-metrics-exporter is currently facing the above errors

Expected results:
working of ocs-metrics-exporter should be smooth and there should not be any error messages in the operator logs.

Additional info: