Bug 1817597 - KubeCPUOvercommit and KubeMemOvercommit alerts can be triggered in two different scenarios.
Summary: KubeCPUOvercommit and KubeMemOvercommit alerts can be triggered in two differ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: 4.5.0
Assignee: Pawel Krupa
QA Contact: hongyan li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-26 16:02 UTC by German Parente
Modified: 2023-10-06 19:30 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-07-13 17:23:46 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github kubernetes-monitoring kubernetes-mixin pull 386 0 None closed alerts: rename duplicate Kube*Overcommit alerts 2020-12-24 02:00:45 UTC
Github openshift cluster-monitoring-operator pull 727 0 None closed Bug 1817597: Fix alert duplicates 2020-12-24 02:01:18 UTC
Red Hat Product Errata RHBA-2020:2409 0 None None None 2020-07-13 17:24:15 UTC

Description German Parente 2020-03-26 16:02:40 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 German Parente 2020-03-26 16:12:26 UTC
Description of problem:

There are two different scenarios for each of these alerts to be triggered: KubeCPUOvercommit and KubeMemOvercommit


Version-Release number of selected component (if applicable): ocp 3.11.Z / ocp 4.X


How reproducible: always

Additional info:

    - alert: KubeCPUOvercommit
      annotations:
        message: Cluster has overcommitted CPU resource requests for Pods and cannot
          tolerate node failure.
      expr: |
        sum(namespace:kube_pod_container_resource_requests_cpu_cores:sum)
          /
        sum(kube_node_status_allocatable_cpu_cores)
          >
        (count(kube_node_status_allocatable_cpu_cores)-1) / count(kube_node_status_allocatable_cpu_cores)
      for: 5m
      labels:
        severity: warning
    - alert: KubeMemOvercommit
      annotations:
        message: Cluster has overcommitted memory resource requests for Pods and cannot
          tolerate node failure.
      expr: |
        sum(namespace:kube_pod_container_resource_requests_memory_bytes:sum)
          /
        sum(kube_node_status_allocatable_memory_bytes)
          >
        (count(kube_node_status_allocatable_memory_bytes)-1)
          /
        count(kube_node_status_allocatable_memory_bytes)
      for: 5m
      labels:
        severity: warning
    - alert: KubeCPUOvercommit
      annotations:
        message: Cluster has overcommitted CPU resource requests for Namespaces.
      expr: |
        sum(kube_resourcequota{namespace=~"(openshift-.*|kube-.*|default|logging)",job="kube-state-metrics", type="hard", resource="cpu"})
          /
        sum(kube_node_status_allocatable_cpu_cores)
          > 1.5
      for: 5m
      labels:
        severity: warning
    - alert: KubeMemOvercommit
      annotations:
        message: Cluster has overcommitted memory resource requests for Namespaces.
      expr: |
        sum(kube_resourcequota{namespace=~"(openshift-.*|kube-.*|default|logging)",job="kube-state-metrics", type="hard", resource="memory"})
          /
        sum(kube_node_status_allocatable_memory_bytes{job="node-exporter"})
          > 1.5
      for: 5m
      labels:

The first one seems to be more cluster wided and the second one related to quotas in namespaces.

Comment 8 errata-xmlrpc 2020-07-13 17:23:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409


Note You need to log in before you can comment on or make changes to this bug.