Bug 2014167

Summary: MGR related alerts are not triggered when Ceph Manager is down
Product: [Red Hat Storage] Red Hat OpenShift Container Storage Reporter: Filip Balák <fbalak>
Component: ceph-monitoringAssignee: arun kumar mohan <amohan>
Status: CLOSED ERRATA QA Contact: Filip Balák <fbalak>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.8CC: amohan, muagarwa, ocs-bugs, ratamir, shan, tnielsen
Target Milestone: ---Keywords: Regression, TestBlocker, ZStream
Target Release: OCS 4.8.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.8.3-14 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 2014391 (view as bug list) Environment:
Last Closed: 2021-10-18 12:16:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2014391    

Description Filip Balák 2021-10-14 14:44:13 UTC
Description of problem (please be detailed as possible and provide log
snippests):
Alerts CephMgrIsAbsent and CephMgrIsMissingReplicas are not triggered when deployment rook-ceph-mgr-a has no pods.

Version of all relevant components (if applicable):
OCS 4.8.3-10
OCP 4.8.0-0.nightly-2021-10-09-130601

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Ceph alerts are not working when Ceph Manager is not working. This can hide any Ceph problem in the cluster by not informing user.

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
2

Can this issue reproducible?
yes

Can this issue reproduce from the UI?
yes

If this is a regression, please provide more details to justify this:
The alert worked in previous versions.

Steps to Reproduce:
1. Downscale rook-ceph-mgr-a deployment (Workloads -> Deployments -> rook-ceph-mgr-a in OCP Console).
2. Check alerts (Monitoring -> Alerting in OCP Console).


Actual results:
It seems that alerts CephMgrIsAbsent and CephMgrIsMissingReplicas are raised for few seconds but are immediately resolved even if the problem is not resolved yet. Those alerts are not triggered anymore.

Expected results:
Alerts CephMgrIsAbsent and CephMgrIsMissingReplicas should work.

Additional info:

Comment 1 Mudit Agarwal 2021-10-15 06:42:05 UTC
We **MUST** always open a new bug against ODF and current release (which is 4.9 currently)
It does not matter where the bug was found, it will be decided that where the fix will go but we must always start with the current release.

Comment 3 arun kumar mohan 2021-10-15 07:40:58 UTC
Added the following PR: https://github.com/rook/rook/pull/8985

Sebastian, Travis please take a look

Comment 4 arun kumar mohan 2021-10-15 07:43:00 UTC
Adding RAC (same as commit message)

CephMgrIsAbsent
----------------
This alert initially had the following query

absent(up{job="rook-ceph-mgr"})

which will fire when the 'up' query is not present, but had two flows
  a. it will not be fired if 'up' provides a result with ZERO value
  b. it will not give any fields in the metric, so 'namespace' was missing

when the above query was replaced with the following,

up{job="rook-ceph-mgr"} == 0

query had the following shortage
  a. whenever mgr pod is completely down (like 'replicas' set to ZERO
and 'mgr' is not coming up), 'up' query will not give any result.

Thus we had to combine both the queries to get results in both the scenarios.

CephMgrIsMissingReplicas
------------------------
This query previously was,

sum(up{job="rook-ceph-mgr"}) < 1

had the same structure as the above (Absent) query, but it's
intention was to check the no: of 'replicas' count for ceph mgr.
Now it is changed to a kube query which handles the replicas count.

Comment 7 Filip Balák 2021-10-18 08:17:21 UTC
The fix looks ok. -> VERIFIED

Tested with:
OCS 4.8.3-14
OCP 4.8.0-0.nightly-2021-10-16-024756

Comment 12 errata-xmlrpc 2021-10-18 12:16:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OCS 4.8.3 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3881