Tested with ODF 4.14.0-132.stable OCP 4.14.0-0.nightly-2023-09-02-132842 ACM 2.9.0-DOWNSTREAM-2023-08-24-09-30-12 subctl version: v0.16.0 ceph version 17.2.6-138.el9cp (b488c8dad42b2ecffcd96f3d76eeeecce48b8590) quincy (stable) When steps to reproduce are repeated, maintainence mode gets activated on the failover cluster which bring down rbd-mirror daemon deployment. From failover cluster- ========================================================================================================================================================================== amagrawa:~$ while true; do mm; sleep 5; echo "********************"; done No resources found ******************** NAME AGE e3f84faf9e8077c06e0a94ca290c447e37d5cf3 3s ******************** NAME AGE e3f84faf9e8077c06e0a94ca290c447e37d5cf3 9s ******************** NAME AGE e3f84faf9e8077c06e0a94ca290c447e37d5cf3 15s ******************** NAME AGE e3f84faf9e8077c06e0a94ca290c447e37d5cf3 21s ******************** NAME AGE e3f84faf9e8077c06e0a94ca290c447e37d5cf3 28s ========================================================================================================================================================================== amagrawa:c1-acm$ mm -o yaml apiVersion: v1 items: - apiVersion: ramendr.openshift.io/v1alpha1 kind: MaintenanceMode metadata: creationTimestamp: "2023-09-20T12:15:33Z" finalizers: - maintenance.multicluster.odf.openshift.io generation: 1 name: e3f84faf9e8077c06e0a94ca290c447e37d5cf3 ownerReferences: - apiVersion: work.open-cluster-management.io/v1 kind: AppliedManifestWork name: f9e723bc6ad94f8af2fac0602a31f48e84899168c4a8dd245cf24ac09430ec13-e3f84faf9e8077c06e0a94ca290c447e37d5cf3-mmode-mw uid: 625c837e-7279-49ff-b1b5-09216c3c7747 resourceVersion: "18537721" uid: 16e74788-1130-4377-854d-bd0155ada4e0 spec: modes: - Failover storageProvisioner: openshift-storage.rbd.csi.ceph.com targetID: e3f84faf9e8077c06e0a94ca290c447e37d5cf3 status: conditions: - lastTransitionTime: "2023-09-20T12:15:33Z" message: Maintenance(mode=Failover) of cluster has completed successfully observedGeneration: 1 reason: MaintenanceCompleted status: "True" type: FailoverActivated observedGeneration: 1 state: Completed kind: List metadata: resourceVersion: "" ========================================================================================================================================================================== amagrawa:c1-acm$ pods| grep mirror enable-rbd-mirror-debug-logging-8t74r 0/1 Completed 0 8d <none> compute-1 <none> <none> rook-ceph-rbd-mirror-a-5c6b58fdcb-z6q5c 2/2 Running 2 7d18h 10.128.2.3 compute-0 <none> <none> amagrawa:c1-acm$ pods| grep mirror enable-rbd-mirror-debug-logging-8t74r 0/1 Completed 0 8d <none> compute-1 <none> <none> Tested twice, fix LGTM.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.14.0 security, enhancement & bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:6832