Bug 1819252

Summary: kubevirt-ssp-operator cannot create ServiceMonitor object
Product: Container Native Virtualization (CNV) Reporter: Oren Cohen <ocohen>
Component: SSPAssignee: Karel Šimon <ksimon>
Status: CLOSED ERRATA QA Contact: Israel Pinto <ipinto>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 2.3.0CC: cnv-qe-bugs, ksimon, ncredi, nunnatsa, ocohen, rnetser, stirabos
Target Milestone: ---   
Target Release: 2.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: kubevirt-ssp-operator-container-v2.4.0-45 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-07-28 19:09:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Oren Cohen 2020-03-31 14:51:07 UTC
Description of problem:
While running e2e CI tests, SSP operator failed on the following error:

{"level":"info","ts":1585653625.6926363,"logger":"metrics","msg":"Metrics Service object updated","Service.Name":"kubevirt-ssp-operator-metrics","Service.Namespace":"kubevirt-hyperconverged"}
{"level":"info","ts":1585653628.3454573,"logger":"cmd","msg":"Could not create ServiceMonitor object","Namespace":"","error":"servicemonitors.monitoring.coreos.com is forbidden: User \"system:serviceaccount:kubevirt-hyperconverged:kubevirt-ssp-operator\" cannot create resource \"servicemonitors\" in API group \"monitoring.coreos.com\" in the namespace \"kubevirt-hyperconverged\""}

Version-Release number of selected component (if applicable):
quay.io/fromani/kubevirt-ssp-operator-container:v1.0.28

How reproducible:
Nondeterministic


Actual results:
https://storage.googleapis.com/origin-ci-test/pr-logs/pull/kubevirt_hyperconverged-cluster-operator/516/pull-ci-kubevirt-hyperconverged-cluster-operator-master-hco-e2e-upgrade-aws/720/artifacts/hco-e2e-upgrade-aws/pods/kubevirt-hyperconverged_kubevirt-ssp-operator-5d9fbdfb94-lc5lr_kubevirt-ssp-operator.log

Expected results:
ssp-operator should create its required objects and report ready.

Additional info:
Its seems like a permission for servicemonitors is missing on the ssp-operator service account in the CSV.
Refer to:
https://github.com/MarSik/kubevirt-ssp-operator/blob/master/manifests/generated/kubevirt-ssp-operator.vVERSION.clusterserviceversion.yaml#L83
which is missing "servicemonitors" under resources of apiGroup "monitoring.coreos.com"

Comment 1 Simone Tiraboschi 2020-04-01 10:00:53 UTC
Simply adding a create rule for servicemonitors.monitoring.coreos.com is not enough,
then SSP operator will fail creating it with:

{"level":"info","ts":1585732384.3286846,"logger":"cmd","msg":"Could not create ServiceMonitor object","Namespace":"","error":"servicemonitors.monitoring.coreos.com \"kubevirt-ssp-operator-metrics\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"}

Comment 2 Simone Tiraboschi 2020-04-08 15:30:28 UTC
After that change it fails with a similar issue on services/finalizers and then list on clusterroles.

Comment 3 Karel Šimon 2020-05-06 07:21:53 UTC
fixed by: https://github.com/MarSik/kubevirt-ssp-operator/pull/161

Comment 4 Nelly Credi 2020-06-08 09:49:27 UTC
@Karel, can you please set the fixed in version, so we will know which version contains the fix?

Comment 5 Ruth Netser 2020-06-08 10:24:50 UTC
Verified no ServiceMonitor errors in ssp operator log.
Version: kubevirt-ssp-operator-container-v2.4.0-46

Comment 8 errata-xmlrpc 2020-07-28 19:09:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:3194