Bug 1825324
| Summary: | Work queue metrics from openshift-controller-manager missing | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Adam Kaplan <adam.kaplan> | ||||||
| Component: | openshift-controller-manager | Assignee: | Adam Kaplan <adam.kaplan> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | wewang <wewang> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | urgent | ||||||||
| Version: | 4.3.z | CC: | aaleman, aos-bugs, mfojtik, obulatov, skuznets | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | 4.5.0 | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: |
Cause: controllers within the openshift-controller-manager did not use named work queues
Consequence: useful metrics like workqueue_depth did not appear in Prometheus
Fix: refactored controllers to use named work queues
Result: workqueue_depth metrics appear in Prometheus
|
Story Points: | --- | ||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2020-07-13 17:28:35 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 1832839 | ||||||||
| Attachments: |
|
||||||||
The queue to check for regression: ImageStreamController. We have it in 3.11, but it disappeared in 4.x Created attachment 1683219 [details]
metrics for ocm
Verified in version: 4.5.0-0.nightly-2020-04-30-042256 Bumping to urgent, we need this backported at least to 4.4 before we can migrate again. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2409 |
Created attachment 1679716 [details] workqueue_depth metrics Description of problem: Work queue metrics for controllers within OCM are missing in Prometheus Version-Release number of selected component (if applicable): 4.3.z How reproducible: Always Steps to Reproduce: 1. Open the prometheus UI from the web console 2. Query the workqueue_depth metric by job - ex `sum(workqueue_depth) by (job)` 3. View the results Actual results: No metrics appear from the "openshift-controller-manager" job Expected results: Metrics from the "openshift-controller-manager" job should appear, with work queue metrics for each named controller