Description of problem (please be detailed as possible and provide log
snippests):
[RDR][CEPHFS] After performing Failover replicationsource and replicationdestination are not getting updated

Version of all relevant components (if applicable):

OCP version:- 4.12.0-0.nightly-2022-10-05-053337
ODF version:- 4.12.0-70
CEPH version:- ceph version 16.2.10-41.el8cp (26bc3d938546adfb098168b7b565d4f9fa377775) pacific (stable)
ACM version:- 2.6.1
SUBMARINER version:- v0.13.0
VOLSYNC version:- volsync-product.v0.5.0

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
2

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1.Deploy RDR cluster
2.Deploy CEPH DR workload
3.After sometime perform Failover


Actual results:

######Current primary after Failover

 ~  22-10-12 19:27  ✝  ~  oc get replicationdestination        
No resources found in busybox-workloads-8 namespace.
 ~  22-10-12 19:27  ✝  ~  oc get replicationsource     
NAME          SOURCE        LAST SYNC   DURATION   NEXT SYNC
dd-io-pvc-1   dd-io-pvc-1                          
dd-io-pvc-2   dd-io-pvc-2                          
dd-io-pvc-3   dd-io-pvc-3                          
dd-io-pvc-4   dd-io-pvc-4                          
dd-io-pvc-5   dd-io-pvc-5                          
dd-io-pvc-6   dd-io-pvc-6                          
dd-io-pvc-7   dd-io-pvc-7                          
 ~  22-10-12 19:27  ✝  ~  oc get pods                                       
NAME                                  READY   STATUS        RESTARTS   AGE
dd-io-1-5857bfdcd9-wbx8r              1/1     Running       0          27m
dd-io-2-bcd6d9f65-9tzml               1/1     Running       0          27m
dd-io-3-5d6b4b84df-v2855              1/1     Running       0          27m
dd-io-4-6f6db89fbf-hgdq7              1/1     Running       0          27m
dd-io-5-7868bc6b5c-wbx2p              1/1     Running       0          27m
dd-io-6-58c98598d5-88fhz              1/1     Running       0          27m
dd-io-7-694958ff97-5cmql              1/1     Running       0          27m
volsync-rsync-src-dd-io-pvc-1-czkpp   1/1     Terminating   0          34s
volsync-rsync-src-dd-io-pvc-1-g5sfn   1/1     Running       0          34s
volsync-rsync-src-dd-io-pvc-2-8tpvc   1/1     Terminating   0          33s
volsync-rsync-src-dd-io-pvc-2-xlr72   1/1     Running       0          33s
volsync-rsync-src-dd-io-pvc-3-kdmsp   1/1     Running       0          13s
volsync-rsync-src-dd-io-pvc-3-rrrb5   1/1     Terminating   0          13s
volsync-rsync-src-dd-io-pvc-4-2wr92   1/1     Running       0          13s
volsync-rsync-src-dd-io-pvc-4-szr4j   1/1     Terminating   0          13s
volsync-rsync-src-dd-io-pvc-5-m4nhh   1/1     Running       0          13s
volsync-rsync-src-dd-io-pvc-5-vmzxb   1/1     Terminating   0          13s
volsync-rsync-src-dd-io-pvc-6-6npgg   1/1     Running       0          33s
volsync-rsync-src-dd-io-pvc-6-sj8qd   1/1     Terminating   0          33s
volsync-rsync-src-dd-io-pvc-7-hwx24   1/1     Running       0          33s
volsync-rsync-src-dd-io-pvc-7-pwpz2   1/1     Terminating   0          33s


##### Seconday cluster after Failover
 ~  22-10-12 19:27  ✝  ~  oc get replicationsource
NAME          SOURCE        LAST SYNC              DURATION             NEXT SYNC
dd-io-pvc-1   dd-io-pvc-1   2022-10-12T13:14:21Z   1h24m21.496481698s   2022-10-12T13:20:00Z
dd-io-pvc-2   dd-io-pvc-2   2022-10-12T13:29:34Z   1h19m34.382411017s   2022-10-12T13:30:00Z
dd-io-pvc-3   dd-io-pvc-3   2022-10-12T12:59:51Z   1h19m51.364323228s   2022-10-12T13:00:00Z
dd-io-pvc-4   dd-io-pvc-4   2022-10-12T13:12:45Z   1h12m45.131768462s   2022-10-12T13:20:00Z
dd-io-pvc-5   dd-io-pvc-5   2022-10-12T12:37:38Z   1h7m38.332564644s    2022-10-12T12:40:00Z
dd-io-pvc-6   dd-io-pvc-6   2022-10-12T12:28:01Z   1h8m1.008806092s     2022-10-12T12:30:00Z
dd-io-pvc-7   dd-io-pvc-7   2022-10-12T12:23:38Z   1h23m38.015112898s   2022-10-12T12:30:00Z
 scripts  22-10-12 19:28  ✝  ~/scripts  oc get replicationdestination
No resources found in busybox-workloads-8 namespace.
 scripts  22-10-12 19:28  ✝  ~/scripts  oc get pods                  
NAME                                  READY   STATUS    RESTARTS   AGE
volsync-rsync-src-dd-io-pvc-1-7tcp4   0/1     Pending   0          38m
volsync-rsync-src-dd-io-pvc-2-7ckzj   0/1     Pending   0          28m
volsync-rsync-src-dd-io-pvc-3-lsdr8   0/1     Pending   0          58m
volsync-rsync-src-dd-io-pvc-4-g9ldq   0/1     Pending   0          38m
volsync-rsync-src-dd-io-pvc-5-dbkxm   0/1     Error     0          78m
volsync-rsync-src-dd-io-pvc-5-jpt7g   0/1     Error     0          5m25s
volsync-rsync-src-dd-io-pvc-5-zqqvr   0/1     Error     0          6m38s
volsync-rsync-src-dd-io-pvc-6-56qmw   0/1     Error     0          13m
volsync-rsync-src-dd-io-pvc-6-g2bzq   0/1     Error     0          15m
volsync-rsync-src-dd-io-pvc-6-wqbl7   0/1     Error     0          88m
volsync-rsync-src-dd-io-pvc-7-g9hxf   0/1     Error     0          88m
volsync-rsync-src-dd-io-pvc-7-rkn9m   0/1     Error     0          3m52s
volsync-rsync-src-dd-io-pvc-7-t44rc   0/1     Error     0          5m6s



Expected results:


Additional info: