Bug 1690951
| Summary: | [3.11.z] KubeletTooManyPods statically compares against 100 instead of --max-pods (-10) | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Matthew Robson <mrobson> |
| Component: | Monitoring | Assignee: | Frederic Branczyk <fbranczy> |
| Status: | CLOSED ERRATA | QA Contact: | Junqi Zhao <juzhao> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 3.11.0 | CC: | anpicker, cvogel, erooth, fbranczy, jupierce, juzhao, mloibl, mrobson, ronny.pettersen, surbania |
| Target Milestone: | --- | ||
| Target Release: | 3.11.z | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1633387 | Environment: | |
| Last Closed: | 2019-06-06 02:00:29 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1633387 | ||
| Bug Blocks: | |||
|
Comment 4
Frederic Branczyk
2019-04-04 07:52:21 UTC
Tested with cluster-monitoring-operator-v3.11.104-1, KubeletTooManyPods rule is under below, since the maximum pods number is 250, this alert will not be triggered, and message "close to the limit of 110" is also wrong, should be 250
***************************************
- alert: KubeletTooManyPods
annotations:
message: Kubelet {{$labels.instance}} is running {{$value}} pods, close to
the limit of 110.
expr: |
kubelet_running_pod_count{job="kubelet"} > 250
for: 15m
labels:
severity: warning
***************************************
Should be the same with 4.1 KubeletTooManyPods rule
***************************************
- alert: KubeletTooManyPods
annotations:
message: Kubelet {{ $labels.instance }} is running {{ $value }} Pods, close
to the limit of 250.
expr: |
kubelet_running_pod_count{job="kubelet"} > 250 * 0.9
for: 15m
labels:
severity: warning
***************************************
Good catch! https://github.com/openshift/cluster-monitoring-operator/pull/317 fixes this. https://github.com/openshift/cluster-monitoring-operator/pull/317 is merged, so moving to modified again. Tested with cluster-monitoring-operator-v3.11.105-1, issue is fixed
alert: KubeletTooManyPods
expr: kubelet_running_pod_count{job="kubelet"}
> 250 * 0.9
for: 15m
labels:
severity: warning
annotations:
message: Kubelet {{ $labels.instance }} is running {{ $value }} Pods, close to the
limit of 250.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0794 |