Description of problem (please be detailed as possible and provide log snippests): Steps to reproduce:- ------------------- 1) Keep the workload in RDR setup running for more than a week Additional Info: ---------------- Not able to execute some rbd commands. Getting the below message while executing the rbd cmd rbd mirror snapshot schedule list --recursive rbd: rbd mirror snapshot schedule list failed: (11) Resource temporarily unavailable Actual results: --------------- observing the error message 'rados: ret=-11, Resource temporarily unavailable' Because of this the snapshot scheduling stops Expected results: ------------------ -> Snapshot scheduling should not stop vr yaml: ------- oc get vr busybox-pvc-61 -o yaml apiVersion: replication.storage.openshift.io/v1alpha1 kind: VolumeReplication metadata: creationTimestamp: "2023-07-10T08:04:25Z" finalizers: - replication.storage.openshift.io generation: 1 name: busybox-pvc-61 namespace: appset-busybox-4 ownerReferences: - apiVersion: ramendr.openshift.io/v1alpha1 blockOwnerDeletion: true controller: true kind: VolumeReplicationGroup name: busybox-4-placement-drpc uid: 6f21ad83-16e0-4eb9-98bf-e43b9fb9bdf0 resourceVersion: "36486402" uid: a85e701c-4109-49a5-9dd6-fcb682a818bf spec: autoResync: false dataSource: apiGroup: "" kind: PersistentVolumeClaim name: busybox-pvc-61 replicationHandle: "" replicationState: primary volumeReplicationClass: rbd-volumereplicationclass-2263283542 status: conditions: - lastTransitionTime: "2023-07-10T08:04:26Z" message: "" observedGeneration: 1 reason: FailedToPromote status: "False" type: Completed - lastTransitionTime: "2023-07-10T08:04:26Z" message: "" observedGeneration: 1 reason: Error status: "True" type: Degraded - lastTransitionTime: "2023-07-10T08:04:26Z" message: "" observedGeneration: 1 reason: NotResyncing status: "False" type: Resyncing message: 'rados: ret=-11, Resource temporarily unavailable' observedGeneration: 1 state: Unknown Must gather logs ---------------- c1 - http://rhsqe-repo.lab.eng.blr.redhat.com/OCS/ocs-qe-bugs/bz-2219628/july10/c1/ c2 - http://rhsqe-repo.lab.eng.blr.redhat.com/OCS/ocs-qe-bugs/bz-2219628/july10/c2/ hub - http://rhsqe-repo.lab.eng.blr.redhat.com/OCS/ocs-qe-bugs/bz-2219628/july10/hub/ Version of all relevant components (if applicable): OCP Version - 4.13.0-0.nightly-2023-06-05-164816 ODF - ODF 4.13.0-219.snaptrim SUBMARINER version:- v0.15.1 VOLSYNC version:- volsync-product.v0.7.1 ceph version - ceph version 17.2.6-70.0.TEST.bz2119217.el9cp (6d74fefa15d1216867d1d112b47bb83c4913d28f) quincy (stable) Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? Is there any workaround available to the best of your knowledge? Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? Can this issue reproducible? Can this issue reproduce from the UI? If this is a regression, please provide more details to justify this: