Bug 1667331 - Duplicate alert for pod which have single container
Summary: Duplicate alert for pod which have single container
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.1.0
Assignee: Frederic Branczyk
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-01-18 07:51 UTC by Junqi Zhao
Modified: 2019-06-04 10:42 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:42:02 UTC
Target Upstream Version:


Attachments (Terms of Use)
CPUThrottlingHigh alerts for cluster-monitoring-operator pod (80.30 KB, image/png)
2019-01-18 07:51 UTC, Junqi Zhao
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 None None None 2019-06-04 10:42:09 UTC

Description Junqi Zhao 2019-01-18 07:51:08 UTC
Created attachment 1521429 [details]
CPUThrottlingHigh alerts for cluster-monitoring-operator pod

Description of problem:
Cloned from https://jira.coreos.com/browse/MON-521
See the attached picture, there are 2 firing CPUThrottlingHigh alerts for cluster-monitoring-operator pod

The first one is: 39% throttling of CPU in namespace openshift-monitoring for container cluster-monitoring-operator in pod cluster-monitoring-operator-96ff8b5c9-d8cd7.

The second one is: 35% throttling of CPU in namespace openshift-monitoring for container in pod cluster-monitoring-operator-96ff8b5c9-d8cd7.

The second one missed the container name

*******************************************************************************

cluster-monitoring-operator pod only has one container

$oc -n openshift-monitoring get pod cluster-monitoring-operator-96ff8b5c9-d8cd7 -o jsonpath="{.spec.containers[*].name}"
cluster-monitoring-operator

also checked the alerts by API

********************************************************************************

{
            "labels": {
                "alertname": "CPUThrottlingHigh",
                "container_name": "cluster-monitoring-operator",
                "namespace": "openshift-monitoring",
                "pod_name": "cluster-monitoring-operator-96ff8b5c9-d8cd7",
                "severity": "warning"
            },
            "annotations": {
                "message": "36% throttling of CPU in namespace openshift-monitoring for container cluster-monitoring-operator in pod cluster-monitoring-operator-96ff8b5c9-d8cd7."
            },
            "state": "firing",
            "activeAt": "2019-01-18T06:56:01.534253519Z",
            "value": 36.43564356435644
        }, {
            "labels": {
                "alertname": "CPUThrottlingHigh",
                "namespace": "openshift-monitoring",
                "pod_name": "cluster-monitoring-operator-96ff8b5c9-d8cd7",
                "severity": "warning"
            },
            "annotations": {
                "message": "46% throttling of CPU in namespace openshift-monitoring for container  in pod cluster-monitoring-operator-96ff8b5c9-d8cd7."
            },
            "state": "firing",
            "activeAt": "2019-01-17T09:52:01.534253519Z",
            "value": 45.78544061302682
        }

 

********************************************************************************

I think the second one should not be appeared, there is something wrong in code to find out the container name




Version-Release number of selected component (if applicable):
payload: registry.svc.ci.openshift.org/ocp/release@sha256:85736576e39221daf368cc82be51cdb2509a77d2446ed98734286fc0ea99656c

cluster-monitoring-operator image: registry.svc.ci.openshift.org/ocp/4.0-art-latest-2019-01-15-184339@sha256:7b88121a3c893297c1be75261bae142b6312227c72bc72a0a64c68363a96601f

How reproducible:
Always

Steps to Reproduce:
1. login web console with cluster-admin and check alerts under "Monitoring -> Alerts"
2.
3.

Actual results:
one alert missed container name

Expected results:
the alert missed container name should not be shown

Additional info:

Comment 1 Frederic Branczyk 2019-02-06 13:16:11 UTC
This was fixed with https://github.com/openshift/cluster-monitoring-operator/pull/221

Comment 3 Junqi Zhao 2019-02-19 01:40:44 UTC
Set it to VERIFIED since the cloned one https://jira.coreos.com/browse/MON-521 is fixed

Comment 6 errata-xmlrpc 2019-06-04 10:42:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.