Description of problem (please be detailed as possible and provide log
snippests):


Version of all relevant components (if applicable):
OCP 4.17.0-0.nightly-2024-10-20-231827
ODF 4.17.0-126
ACM 2.12.0-DOWNSTREAM-2024-10-18-21-57-41
OpenShift Virtualization 4.17.1-19
Submariner 0.19 unreleased downstream image 846949
ceph version 18.2.1-229.el9cp (ef652b206f2487adfc86613646a4cac946f6b4e0) reef (stable)
OADP 1.4.1
OpenShift GitOps 1.14.0
VolSync 0.10.1


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?


Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Deploy a RBD CNV workload on a RDR setup using discovered apps. Create a clone of the PVC.
2. Delete the workload. 
3. Now deploy a workload in such a way that it consumes the cloned PVC.
4. DR protect this workload with a drpolicy where flattening is not enabled.
5. The VR will go to primary and sync and backup would initially look fine for the workload but the RBD image will not undergo flattening.
6. After a while, sync wouldn't continue for this workload and it's hard to debug the root cause because proper error messages are missing in the VR/DRPC resource.


Actual results:
VR-

oc describe vr -n busybox-workloads-100
Name:         root-disk
Namespace:    busybox-workloads-100
Labels:       ramendr.openshift.io/owner-name=busybox-100
              ramendr.openshift.io/owner-namespace-name=openshift-dr-ops
Annotations:  <none>
API Version:  replication.storage.openshift.io/v1alpha1
Kind:         VolumeReplication
Metadata:
  Creation Timestamp:  2024-10-27T18:04:31Z
  Finalizers:
    replication.storage.openshift.io
  Generation:        1
  Resource Version:  9855180
  UID:               c4ae8511-9fa1-4a53-8374-8b87288255d1
Spec:
  Auto Resync:  false
  Data Source:
    API Group:
    Kind:                    PersistentVolumeClaim
    Name:                    root-disk
  Replication Handle:
  Replication State:         primary
  Volume Replication Class:  rbd-volumereplicationclass-1625360775
Status:
  Conditions:
    Last Transition Time:  2024-10-27T18:04:35Z
    Message:
    Observed Generation:   1
    Reason:                Promoted
    Status:                True
    Type:                  Completed
    Last Transition Time:  2024-10-27T18:04:35Z
    Message:
    Observed Generation:   1
    Reason:                Healthy
    Status:                False
    Type:                  Degraded
    Last Transition Time:  2024-10-27T18:04:35Z
    Message:
    Observed Generation:   1
    Reason:                NotResyncing
    Status:                False
    Type:                  Resyncing
  Last Completion Time:    2024-10-27T18:47:06Z
  Last Sync Duration:      0s
  Last Sync Time:          2024-10-27T18:45:00Z
  Message:                 volume is marked primary
  Observed Generation:     1
  State:                   Primary
Events:                    <none>


DRPC-

oc get drpc busybox-100 -oyaml -n openshift-dr-ops
apiVersion: ramendr.openshift.io/v1alpha1
kind: DRPlacementControl
metadata:
  annotations:
    drplacementcontrol.ramendr.openshift.io/app-namespace: openshift-dr-ops
    drplacementcontrol.ramendr.openshift.io/last-app-deployment-cluster: amagrawa-21o-1
  creationTimestamp: "2024-10-27T18:04:31Z"
  finalizers:
  - drpc.ramendr.openshift.io/finalizer
  generation: 2
  labels:
    cluster.open-cluster-management.io/backup: ramen
  name: busybox-100
  namespace: openshift-dr-ops
  ownerReferences:
  - apiVersion: cluster.open-cluster-management.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: Placement
    name: busybox-100-placement-1
    uid: e36cc23e-b6ad-4e24-ab76-0b8f2332aa9e
  resourceVersion: "8573969"
  uid: 552aaddd-3376-4550-ba3d-b7150e27ac91
spec:
  drPolicyRef:
    apiVersion: ramendr.openshift.io/v1alpha1
    kind: DRPolicy
    name: odr-policy-5m
  kubeObjectProtection:
    captureInterval: 5m0s
    kubeObjectSelector:
      matchExpressions:
      - key: appname
        operator: In
        values:
        - vm
  placementRef:
    apiVersion: cluster.open-cluster-management.io/v1beta1
    kind: Placement
    name: busybox-100-placement-1
    namespace: openshift-dr-ops
  preferredCluster: amagrawa-21o-1
  protectedNamespaces:
  - busybox-workloads-100
  pvcSelector:
    matchExpressions:
    - key: appname
      operator: In
      values:
      - vm
status:
  actionDuration: 15.045573062s
  actionStartTime: "2024-10-27T18:04:46Z"
  conditions:
  - lastTransitionTime: "2024-10-27T18:04:31Z"
    message: Initial deployment completed
    observedGeneration: 2
    reason: Deployed
    status: "True"
    type: Available
  - lastTransitionTime: "2024-10-27T18:04:31Z"
    message: Ready
    observedGeneration: 2
    reason: Success
    status: "True"
    type: PeerReady
  - lastTransitionTime: "2024-10-27T18:07:31Z"
    message: VolumeReplicationGroup (openshift-dr-ops/busybox-100) on cluster amagrawa-21o-1
      is protecting required resources and data
    observedGeneration: 2
    reason: Protected
    status: "True"
    type: Protected
  lastGroupSyncDuration: 0s
  lastGroupSyncTime: "2024-10-27T18:10:00Z"
  lastKubeObjectProtectionTime: "2024-10-27T18:54:38Z"
  lastUpdateTime: "2024-10-27T18:59:33Z"
  observedGeneration: 2
  phase: Deployed
  preferredDecision:
    clusterName: amagrawa-21o-1
    clusterNamespace: amagrawa-21o-1
  progression: Completed
  resourceConditions:
    conditions:
    - lastTransitionTime: "2024-10-27T18:04:35Z"
      message: PVCs in the VolumeReplicationGroup are ready for use
      observedGeneration: 1
      reason: Ready
      status: "True"
      type: DataReady
    - lastTransitionTime: "2024-10-27T18:04:32Z"
      message: VolumeReplicationGroup is replicating
      observedGeneration: 1
      reason: Replicating
      status: "False"
      type: DataProtected
    - lastTransitionTime: "2024-10-27T18:04:31Z"
      message: Nothing to restore
      observedGeneration: 1
      reason: Restored
      status: "True"
      type: ClusterDataReady
    - lastTransitionTime: "2024-10-27T18:04:39Z"
      message: Cluster data of all PVs are protected
      observedGeneration: 1
      reason: Uploaded
      status: "True"
      type: ClusterDataProtected
    resourceMeta:
      generation: 1
      kind: VolumeReplicationGroup
      name: busybox-100
      namespace: openshift-dr-ops
      protectedpvcs:
      - root-disk
      resourceVersion: "9869528"


Although it fires the VolumeSyncronizationDelay alert on the hub cluster if cluster monitoring labelling is done, please note that it's optional and doesn't highlight where the root cause is. There could be n number of reasons why sync isn't progressing?

Also, to check if image under went flattening or not, one has to rsh into toolbox pod and run ceph progress command which isn't recommended for customers. 



Expected results:[RDR] [Flatten] Proper error messages should be shown in VR and DRPC resource when a drpolicy without flattening is applied to cloned/snapshot restored PVC and sync doesn't resume/rbd-image doesn't undergo flattening.


Additional info: