Bug 2039321

Summary:	SDN controller metrics are not being consumed by prometheus
Product:	OpenShift Container Platform	Reporter:	Martin Kennelly <mkennell>
Component:	Networking	Assignee:	Martin Kennelly <mkennell>
Networking sub component:	openshift-sdn	QA Contact:	Weibin Liang <weliang>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	medium	CC:	weliang, zzhao
Version:	4.10
Target Milestone:	---
Target Release:	4.10.0
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-03-12 04:40:34 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Martin Kennelly 2022-01-11 13:42:21 UTC

Description of problem:
SDN controller metrics are exposed but are not being scraped by prometheus via servicemonitor+service [1] pattern.

Version-Release number of selected component (if applicable):
4.10

How reproducible:
always

Steps to Reproduce:
1. Spin up 4.10 cluster
2. Search for SDN controller metric "sdn_controller_num_egress_ips" via prometheus query interface. It will not exist.

Actual results:
SDN controller metrics are not available via prometheus but are exposed via SDN controller.

Expected results:
SDN controller metrics are available via prometheus query interface.

Additional info:
[1] https://docs.openshift.com/container-platform/4.7/monitoring/managing-metrics.html#setting-up-metrics-collection-for-user-defined-projects_managing-metrics

Comment 4 Weibin Liang 2022-01-27 18:55:00 UTC

Tested and verified in 4.10.0-0.nightly-2022-01-27-104747

[weliang@weliang ~]$ oc -n openshift-monitoring exec -it prometheus-k8s-1 curl http://localhost:9090/api/v1/query?query=sdn_controller_num_egress_ips
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
{"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"sdn_controller_num_egress_ips","container":"kube-rbac-proxy","endpoint":"controller-metrics","instance":"172.31.249.102:9106","job":"sdn-controller","namespace":"openshift-sdn","pod":"sdn-controller-5kmkm","service":"sdn-controller"},"value":[1643309580.205,"1"]}]}}

Comment 7 errata-xmlrpc 2022-03-12 04:40:34 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056