Bug 1809699 - No controller metrics reported from 4.3 openshift-controller-manager
Summary: No controller metrics reported from 4.3 openshift-controller-manager
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: openshift-controller-manager
Version: 4.3.z
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.5.0
Assignee: Adam Kaplan
QA Contact: wewang
Depends On:
Blocks: 1810304
TreeView+ depends on / blocked
Reported: 2020-03-03 17:23 UTC by Clayton Coleman
Modified: 2020-07-13 17:18 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: openshift-controller-manager metrics were not properly registered with the 1.16 kubernetes Prometheus registry Consequence: metrics for openshift control plane controllers were missing Fix: properly registered openshift-controller-manager metrics with Prometheus Result: restored openshift control plane metrics
Clone Of:
: 1810304 (view as bug list)
Last Closed: 2020-07-13 17:17:45 UTC
Target Upstream Version:

Attachments (Terms of Use)
Metrics from 4.3 cluster OCM leader (12.49 KB, text/plain)
2020-03-03 17:23 UTC, Clayton Coleman
no flags Details

System ID Priority Status Summary Last Updated
Github openshift openshift-controller-manager pull 69 None closed Bug 1809699: Restore openshift-controller-manager metrics 2020-08-21 14:00:21 UTC
Red Hat Product Errata RHBA-2020:2409 None None None 2020-07-13 17:18:07 UTC

Description Clayton Coleman 2020-03-03 17:23:49 UTC
Created attachment 1667268 [details]
Metrics from 4.3 cluster OCM leader

On a 4.3 cluster experiencing issues we tried to pull metrics from the openshift controllers and realized none of them were being reported.  The pods are being scraped, but the metrics endpoint is only returning a very small set of metrics.  It looks like we are failing to correctly register our metrics.

The linked attachment contains all the metrics reported from this 4.3 cluster, but metrics that should be there are missing (like image import controller metrics).

Urgent because we can't alert or debug if these are missing.

Comment 6 errata-xmlrpc 2020-07-13 17:17:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.