Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1782061

Summary: template-service-broker-operator doesn't notify users and admins via alerts in prometheus
Product: OpenShift Container Platform Reporter: Jesus M. Rodriguez <jesusr>
Component: Service BrokerAssignee: Jesus M. Rodriguez <jesusr>
Status: CLOSED ERRATA QA Contact: Cuiping HUO <chuo>
Severity: high Docs Contact:
Priority: high    
Version: 4.4CC: aos-bugs, chezhang, chuo, jesusr, jfan, jiazha
Target Milestone: ---Keywords: UpcomingSprint
Target Release: 4.4.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1776133 Environment:
Last Closed: 2020-06-02 11:17:47 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1800645    
Bug Blocks: 1776133    

Comment 2 Cuiping HUO 2019-12-17 06:45:40 UTC
No packagemanifest is ready for 4.4 verification.

Comment 3 Jesus M. Rodriguez 2020-02-07 16:12:43 UTC
This bug depends on PR https://github.com/openshift/template-service-broker-operator/pull/68 to re-enabled TSBO

Comment 4 Jesus M. Rodriguez 2020-02-25 21:17:29 UTC
Now that PR 68 merged, this should be ready to test.

Comment 6 Cuiping HUO 2020-03-05 09:46:19 UTC
Verification failed.
cluster version:4.4.0-0.nightly-2020-03-04-230603
templateservicebroker_info metrics works, but 
1.Alert: TemplateServiceBrokerEnabled is not firing.
2.prometheus rule with alert name  TemplateServiceBrokerEnabled should have a message field.

$ $ oc get ep -n openshift-template-service-broker
NAME                                                 ENDPOINTS                           AGE
apiserver                                            10.128.2.12:8443                    79m
openshift-template-service-broker-operator-metrics   10.131.0.32:8383,10.131.0.32:8686   79m

$ token=`oc -n openshift-monitoring sa get-token prometheus-k8s`
$ oc -n openshift-monitoring exec -c prometheus prometheus-k8s-1  -- curl -k -H "Authorization: Bearer $token" 'https://10.128.2.12:8443/metrics' | grep template_service_broker
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 17482    0 17482    0     0   116k      0 --:--:-- --:--:-- --:--:--  116k
# HELP template_service_broker_build_info A metric with a constant '1' value labeled by major, minor, git commit & git version from which Template Service Broker was built.
# TYPE template_service_broker_build_info gauge
template_service_broker_build_info{gitCommit="58203d1a763764c7eec567473304e831946ab8c3",gitVersion="v0.0.0-alpha.0-5-g58203d1a",major="",minor=""} 1

$ oc get csv -n openshift-template-service-broker
NAME                                                        DISPLAY                                      VERSION              REPLACES   PHASE
openshifttemplateservicebrokeroperator.4.4.0-202003021217   OpenShift Template Service Broker Operator   4.4.0-202003021217              Succeeded

Comment 8 Jesus M. Rodriguez 2020-03-06 00:53:08 UTC
I initially thought it was searching for the wrong metric, but it doesn't look like the metric is being exposed from the operator properly. The one seen is the build-info which wasn't the one I expected. This was not the quick fix I was hoping for so punting for now.

Comment 10 Jesus M. Rodriguez 2020-05-15 03:33:32 UTC
Bug has been reviewed this sprint.

Comment 13 Fan Jia 2020-05-28 11:06:34 UTC
test env:
cv: 4.4.5
template broker version:openshifttemplateservicebrokeroperator.4.4.0-202005252114

test result:
1)$ oc -n openshift-monitoring exec -c prometheus prometheus-k8s-1  -- curl -k -H "Authorization: Bearer $token" http://10xxxxx6:8686/metrics | grep templateservicebroker_info
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   259  100   259    0     0  20734      0 --:--:-- --:--:-- --:--:-- 21583
# HELP templateservicebroker_info Information about the TemplateServiceBroker custom resource.
# TYPE templateservicebroker_info gauge
templateservicebroker_info{namespace="openshift-template-service-broker",templateservicebroker="template-service-broker"} 1

2)$ oc -n openshift-template-service-broker get prometheusrules
NAME                      AGE
template-service-broker   42m

3)TemplateServiceBrokerEnabled is firing in the Alerting page. But the alert doesn't have "message" to describe the content about the alert. Will open one more bug to track this problem.

Comment 15 errata-xmlrpc 2020-06-02 11:17:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2310