Bug 1986983 - Revise Alert Severity in OCP 4.8
Summary: Revise Alert Severity in OCP 4.8
Status: CLOSED DUPLICATE of bug 1986981
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.9
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: ---
Assignee: Haoyu Sun
QA Contact: Junqi Zhao
Depends On:
TreeView+ depends on / blocked
Reported: 2021-07-28 16:11 UTC by Haoyu Sun
Modified: 2021-08-10 07:13 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2021-08-10 07:13:43 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1986981 1 high CLOSED Revise Alert Severity in OCP 4.9 2021-10-18 17:43:10 UTC

Description Haoyu Sun 2021-07-28 16:11:13 UTC
This is a backport of modifications that we are going to make to fix bug 1986981.

Description of problem:

After reviewing critical alerts in OCP, we find out the 21 alerts that need adjustments:
- Recommend changing Critical to Warning:  13
  - KubePersistentVolumeErrors
  - PrometheusBadConfig
  - PrometheusRemoteStorageFailures
  - PrometheusRuleFailures
  - AlertmanagerMembersInconsistent
  - AlertmanagerClusterFailedToSendAlerts
  - AlertmanagerConfigInconsistent
  - AlertmanagerClusterDown
  - KubeStateMetricsListErrors
  - KubeStateMetricsWatchErrors
  - ThanosRuleSenderIsFailingAlerts
  - ThanosRuleHighRuleEvaluationFailures
  - ThanosNoRuleEvaluations

- Recommend removing alert:  2
  - PrometheusErrorSendingAlertsToAnyAlertmanager
  - AlertmanagerClusterCrashlooping

- Recommend changing Critical to Info:  1
  - PrometheusRemoteWriteBehind
- Threshold Tweaks:  5
  - KubePersistentVolumeFillingUp
  - KubeletDown
  - NodeFilesystemFilesFillingUp
  - NodeFilesystemSpaceFillingUp
  - PrometheusRemoteStorageFailures

Please refer to this table for details(proposed modification are in column F "Comments") : https://docs.google.com/spreadsheets/d/10rL3loHz6a8lBfKsU2W9TVZSrSqndrnVmkzDeA3Z2kI/edit?usp=sharing
This table can be also found in the attachment.

Version-Release number of selected component (if applicable): 4.8

How reproducible:

Steps to Reproduce:

Actual results:

Expected results:

Additional info:

Comment 1 Damien Grisonnet 2021-08-10 07:13:43 UTC

*** This bug has been marked as a duplicate of bug 1986981 ***

Note You need to log in before you can comment on or make changes to this bug.