Description of problem (please be detailed as possible and provide log
snippests):
[DR] Mirror Peer stuck in ExchangingSecret State


Version of all relevant components (if applicable):

ODF version:- full_version=4.10.0-137
OCP version:- 4.10.0-0.nightly-2022-01-31-012936

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue reproducible?
yes

Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Deploy DR cluster
2. Deploy ODF over them
3. Create multiclusteroperator
4. Create mirror peer


Actual results:
2022-02-02T09:09:24.738Z	INFO	controller-runtime.manager.controller.mirrorpeer	Source secret not found	{"reconciler group": "multicluster.odf.openshift.io", "reconciler kind": "MirrorPeer", "name": "mirrorpeer-sample", "namespace": "", "Secret": "2e9583b9a3f92f843095ddf0c1893ffec536e48"}
2022-02-02T09:09:24.739Z	ERROR	controller-runtime.manager.controller.mirrorpeer	Failed to find valid source secret	{"reconciler group": "multicluster.odf.openshift.io", "reconciler kind": "MirrorPeer", "name": "mirrorpeer-sample", "namespace": "", "PeerRef": {"clusterName":"vmware-dccp-one","storageClusterRef":{"name":"ocs-storagecluster","namespace":"openshift-storage"}}, "error": "Secret \"2e9583b9a3f92f843095ddf0c1893ffec536e48\" not found"}
github.com/red-hat-storage/odf-multicluster-orchestrator/controllers.(*MirrorPeerReconciler).Reconcile
	/remote-source/app/controllers/mirrorpeer_controller.go:146
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.5/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.5/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.5/pkg/internal/controller/controller.go:214
2022-02-02T09:09:24.739Z	INFO	controller-runtime.manager.controller.mirrorpeer	Secrets not found; Attempting to reconcile again	{"reconciler group": "multicluster.odf.openshift.io", "reconciler kind": "MirrorPeer", "name": "mirrorpeer-sample", "namespace": ""}

Expected results:


Additional info: