Bug 1819252 - kubevirt-ssp-operator cannot create ServiceMonitor object
Summary: kubevirt-ssp-operator cannot create ServiceMonitor object
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: SSP
Version: 2.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 2.4.0
Assignee: Karel Šimon
QA Contact: Israel Pinto
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-31 14:51 UTC by Oren Cohen
Modified: 2020-07-28 19:09 UTC (History)
7 users (show)

Fixed In Version: kubevirt-ssp-operator-container-v2.4.0-45
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-07-28 19:09:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github MarSik kubevirt-ssp-operator pull 159 0 None closed Adding missing rules on servicemonitors.monitoring.coreos.com 2020-12-07 12:18:52 UTC
Github MarSik kubevirt-ssp-operator pull 161 0 None closed fix services/finalizers in cluster roles 2020-12-07 12:19:21 UTC
Github MarSik kubevirt-ssp-operator pull 162 0 None closed Add a rule required on k8s 1.17 2020-12-07 12:19:21 UTC
Github kubevirt hyperconverged-cluster-operator pull 527 0 None closed Enable conditions checks on SSP components 2020-12-07 12:19:20 UTC
Red Hat Product Errata RHSA-2020:3194 0 None None None 2020-07-28 19:09:55 UTC

Description Oren Cohen 2020-03-31 14:51:07 UTC
Description of problem:
While running e2e CI tests, SSP operator failed on the following error:

{"level":"info","ts":1585653625.6926363,"logger":"metrics","msg":"Metrics Service object updated","Service.Name":"kubevirt-ssp-operator-metrics","Service.Namespace":"kubevirt-hyperconverged"}
{"level":"info","ts":1585653628.3454573,"logger":"cmd","msg":"Could not create ServiceMonitor object","Namespace":"","error":"servicemonitors.monitoring.coreos.com is forbidden: User \"system:serviceaccount:kubevirt-hyperconverged:kubevirt-ssp-operator\" cannot create resource \"servicemonitors\" in API group \"monitoring.coreos.com\" in the namespace \"kubevirt-hyperconverged\""}

Version-Release number of selected component (if applicable):
quay.io/fromani/kubevirt-ssp-operator-container:v1.0.28

How reproducible:
Nondeterministic


Actual results:
https://storage.googleapis.com/origin-ci-test/pr-logs/pull/kubevirt_hyperconverged-cluster-operator/516/pull-ci-kubevirt-hyperconverged-cluster-operator-master-hco-e2e-upgrade-aws/720/artifacts/hco-e2e-upgrade-aws/pods/kubevirt-hyperconverged_kubevirt-ssp-operator-5d9fbdfb94-lc5lr_kubevirt-ssp-operator.log

Expected results:
ssp-operator should create its required objects and report ready.

Additional info:
Its seems like a permission for servicemonitors is missing on the ssp-operator service account in the CSV.
Refer to:
https://github.com/MarSik/kubevirt-ssp-operator/blob/master/manifests/generated/kubevirt-ssp-operator.vVERSION.clusterserviceversion.yaml#L83
which is missing "servicemonitors" under resources of apiGroup "monitoring.coreos.com"

Comment 1 Simone Tiraboschi 2020-04-01 10:00:53 UTC
Simply adding a create rule for servicemonitors.monitoring.coreos.com is not enough,
then SSP operator will fail creating it with:

{"level":"info","ts":1585732384.3286846,"logger":"cmd","msg":"Could not create ServiceMonitor object","Namespace":"","error":"servicemonitors.monitoring.coreos.com \"kubevirt-ssp-operator-metrics\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"}

Comment 2 Simone Tiraboschi 2020-04-08 15:30:28 UTC
After that change it fails with a similar issue on services/finalizers and then list on clusterroles.

Comment 3 Karel Šimon 2020-05-06 07:21:53 UTC
fixed by: https://github.com/MarSik/kubevirt-ssp-operator/pull/161

Comment 4 Nelly Credi 2020-06-08 09:49:27 UTC
@Karel, can you please set the fixed in version, so we will know which version contains the fix?

Comment 5 Ruth Netser 2020-06-08 10:24:50 UTC
Verified no ServiceMonitor errors in ssp operator log.
Version: kubevirt-ssp-operator-container-v2.4.0-46

Comment 8 errata-xmlrpc 2020-07-28 19:09:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:3194


Note You need to log in before you can comment on or make changes to this bug.