Bug 2011784

Summary: ServiceAccount ocs-metrics-exporter fails to get 'prometheus-user' secret
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Sonia Garudi <sgarudi>
Component: ocs-operatorAssignee: Jiffin <jthottan>
Status: CLOSED CURRENTRELEASE QA Contact: Yosi Ben Shimon <ybenshim>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.9CC: ebenahar, jthottan, madam, mbukatov, muagarwa, ocs-bugs, odf-bz-bot, sostapov
Target Milestone: ---   
Target Release: ODF 4.9.0   
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: v4.9.0-189.ci Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-01-07 17:46:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sonia Garudi 2021-10-07 11:44:47 UTC 
Description of problem (please be detailed as possible and provide log
snippests):
ServiceAccount ocs-metrics-exporter tries to get secret for 'prometheus-user' for OBC related alerts, but fails with error below:

E0914 13:19:44.481105       1 ceph-block-pool.go:137] Invalid image health for pool ocs-storagecluster-cephblockpool. Must be OK, UNKNOWN, WARNING or ERROR
E0914 13:19:44.492203       1 object-bucket.go:165] Secret for prometheus-user not found. secrets "rook-ceph-object-user-ocs-storagecluster-cephobjectstore-prometheus-user" is forbidden: User "system:serviceaccount:openshift-storage:ocs-metrics-exporter" cannot get resource "secrets" in API group "" in the namespace "openshift-storage"
W0914 13:19:44.492221       1 object-bucket.go:121] CephObjectStore "ocs-storagecluster-cephobjectstore" in namespace "openshift-storage" was skipped


The ServiceAccount does not have the required permission:
# kubectl auth can-i get secrets --as=system:serviceaccount:openshift-storage:ocs-metrics-exporter
no



Version of all relevant components (if applicable):
4.9


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
Yes

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Check logs for pod ocs-metrics-exporter-*
2.
3.


Actual results:


Expected results:


Additional info:
BZ https://bugzilla.redhat.com/show_bug.cgi?id=1999952 is open to automate creation of the prometheus-user for OBC alerts
Comment 6 Yosi Ben Shimon 2021-11-18 07:48:04 UTC 
Tested on ODF 4.9.0:
# oc get csv -n openshift-storage
NAME                  DISPLAY                       VERSION   REPLACES   PHASE
mcg-operator.v4.9.0   NooBaa Operator               4.9.0                Succeeded
ocs-operator.v4.9.0   OpenShift Container Storage   4.9.0                Succeeded
odf-operator.v4.9.0   OpenShift Data Foundation     4.9.0                Succeeded

# kubectl auth can-i get secrets --as=system:serviceaccount:openshift-storage:ocs-metrics-exporter
yes

Moving to VERIFIED