Bug 2032969 - No RBD mirroring daemon down alert when daemon is down
Summary: No RBD mirroring daemon down alert when daemon is down
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: ocs-operator
Version: 4.9
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ODF 4.10.0
Assignee: gowtham
QA Contact: Aman Agrawal
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-15 15:15 UTC by gowtham
Modified: 2023-08-09 17:00 UTC (History)
8 users (show)

Fixed In Version: 4.10.0-147
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-04-13 18:50:40 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github red-hat-storage ocs-operator pull 1442 0 None open Bug 2032969: Fix no RBD mirroring daemon alert when daemon is down issue 2021-12-17 20:25:35 UTC
Github red-hat-storage ocs-operator pull 1487 0 None open Bug 2032969: [release-4.10] Fix no RBD mirroring daemon alert when daemon is down issue 2022-02-03 11:59:02 UTC
Red Hat Product Errata RHSA-2022:1372 0 None None None 2022-04-13 18:51:13 UTC

Description gowtham 2021-12-15 15:15:42 UTC
No RBD mirror daemon-related alert when the daemon is down. 


Steps to Reproduce:
1. Enable mirroring at cluster level using the below CLI command:
         oc patch StorageCluster $(oc get StorageCluster -n openshift-storage -o=jsonpath='{.items[0].metadata.name}')  -n openshift-storage --type json --patch  '[{ "op": "replace", "path": "/spec/mirroring", "value": {"enabled": true} }]'

2. Use below script to down rbd mirroring container:

start=$(date +%s.%N)

for i in {1..10}
    do
         x=`oc get pods -n openshift-storage | grep rook-ceph-rbd-mirror-a | awk '{print $1}'`
         oc -n openshift-storage exec $x -c rbd-mirror -- kill 1
         result=`oc get pod $x -n openshift-storage -o=jsonpath='{.status.containerStatuses[1].state.running}'`
         until [ -n "$result" ]
	 do
             echo "Container is down"
             sleep 2s
             result=`oc get pod $x -n openshift-storage -o=jsonpath='{.status.containerStatuses[1].state.running}'`
         done
         echo "Container is running"
         sleep 2s
     done

end=$(date +%s.%N)    
runtime=$(python -c "print(${end} - ${start})")

echo "Runtime was $runtime"

3. (or) another way is to decrease the instance count to 0 in rbd mirror daemon deployment CR (not recommended) 



Actual results:
No RBD mirroring daemon alert was raised even after the mirroring daemon is down. (alert was introduced in 4.9) 
Expected results:

Raise RBd mirroring mirroring daemon alert when daemon is down
Additional info:

Comment 11 errata-xmlrpc 2022-04-13 18:50:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.10.0 enhancement, security & bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1372


Note You need to log in before you can comment on or make changes to this bug.