Bug 2283633

Description of problem (please be detailed as possible and provide log
snippests):

[RDR][Discovered Apps] Failover for cephfs workloads are stuck 


Version of all relevant components (if applicable):
OCP version:- 4.16.0-0.nightly-2024-05-21-221942
ODF version:- 4.16.0-108
CEPH version:- ceph version 18.2.1-188.el9cp (b1ae9c989e2f41dcfec0e680c11d1d9465b1db0e) reef (stable)
ACM version:- 2.10.3
SUBMARINER version:- v0.17.1
VOLSYNC version:- volsync-product.v0.9.1
VOLSYNC method:- destinationCopyMethod: Direct
Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue be reproduced?
yes

Can this issue be reproduced from the UI?
yes

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Deploy RDR cluster
2. Deploy cephgs discovered app workload
3. Perform failover 


Actual results:
vrg reports 

failed to add owner/label to snapshot busybox-pvc-16-20240528120125
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)


Expected results:

Failover should work 


Additional info:

status:
  conditions:
  - lastTransitionTime: "2024-05-28T10:31:35Z"
    message: Volsync based PVC protection does not report DataReady condition as Secondary.
      No PVCs are protected using VolumeReplication scheme
    observedGeneration: 3
    reason: Unused
    status: "True"
    type: DataReady
  - lastTransitionTime: "2024-05-28T10:31:35Z"
    message: Volsync based PVC protection does not report DataProtected/ClusterDataProtected
      conditions as Secondary. No PVCs are protected using VolumeReplication scheme
    observedGeneration: 3
    reason: Unused
    status: "True"
    type: DataProtected
  - lastTransitionTime: "2024-05-28T10:31:35Z"
    message: Volsync based PVC protection does not report DataProtected/ClusterDataProtected
      conditions as Secondary. No PVCs are protected using VolumeReplication scheme
    observedGeneration: 3
    reason: Unused
    status: "True"
    type: ClusterDataProtected
  - lastTransitionTime: "2024-05-28T10:32:51Z"
    message: 'Failed to restore PVs/PVCs: failed to restore PV/PVC for VolSync (failed
      to restore all PVCs using RDSpec ([{{app-busybox-cephfs-1 busybox-pvc-2 true
      { { []} { []}} 0xc0092bf940 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{47244640256 0} {<nil>} 44Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-20 true { { []}
      { []}} 0xc0092bf9a0 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{35433480192 0} {<nil>} 33Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-1 true { { []}
      { []}} 0xc0092bfa00 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{100931731456 0} {<nil>} 94Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-13 true { { []}
      { []}} 0xc0092bfa60 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{82678120448 0} {<nil>} 77Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-17 true { { []}
      { []}} 0xc0092bfac0 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{62277025792 0} {<nil>} 58Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-14 true { { []}
      { []}} 0xc0092bfb20 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{75161927680 0} {<nil>} 70Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-18 true { { []}
      { []}} 0xc0092bfb80 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{132070244352 0} {<nil>} 123Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-8 true { { []}
      { []}} 0xc0092bfbe0 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{97710505984 0} {<nil>} 91Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-7 true { { []}
      { []}} 0xc0092bfc40 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{96636764160 0} {<nil>} 90Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-11 true { { []}
      { []}} 0xc0092bfca0 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{35433480192 0} {<nil>} 33Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-10 true { { []}
      { []}} 0xc0092bfd00 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{93415538688 0} {<nil>} 87Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-15 true { { []}
      { []}} 0xc0092bfd60 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{140660178944 0} {<nil>} 131Gi BinarySI}]}
      [{ReplicationSourceSetup True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized
      Ready}] <nil> nil <nil>}} {{app-busybox-cephfs-1 busybox-pvc-4 true { { []}
      { []}} 0xc0092bfdc0 map[] map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs]
      [ReadWriteMany] {map[] map[storage:{{154618822656 0} {<nil>}  BinarySI}]} [{ReplicationSourceSetup
      True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized Ready}] <nil> nil <nil>}}
      {{app-busybox-cephfs-1 busybox-pvc-5 true { { []} { []}} 0xc0092bfe20 map[]
      map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs] [ReadWriteMany]
      {map[] map[storage:{{114890375168 0} {<nil>} 107Gi BinarySI}]} [{ReplicationSourceSetup
      True 1 2024-05-28 07:37:06 +0000 UTC SourceInitialized Ready}] <nil> nil <nil>}}
      {{app-busybox-cephfs-1 busybox-pvc-19 true { { []} { []}} 0xc0092bfe80 map[]
      map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs] [ReadWriteMany]
      {map[] map[storage:{{65498251264 0} {<nil>} 61Gi BinarySI}]} [{ReplicationSourceSetup
      True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized Ready}] <nil> nil <nil>}}
      {{app-busybox-cephfs-1 busybox-pvc-6 true { { []} { []}} 0xc0092bfee0 map[]
      map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs] [ReadWriteMany]
      {map[] map[storage:{{132070244352 0} {<nil>} 123Gi BinarySI}]} [{ReplicationSourceSetup
      True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized Ready}] <nil> nil <nil>}}
      {{app-busybox-cephfs-1 busybox-pvc-12 true { { []} { []}} 0xc0092bff40 map[]
      map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs] [ReadWriteMany]
      {map[] map[storage:{{157840048128 0} {<nil>} 147Gi BinarySI}]} [{ReplicationSourceSetup
      True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized Ready}] <nil> nil <nil>}}
      {{app-busybox-cephfs-1 busybox-pvc-16 true { { []} { []}} 0xc0092bffa0 map[]
      map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs] [ReadWriteMany]
      {map[] map[storage:{{136365211648 0} {<nil>} 127Gi BinarySI}]} [{ReplicationSourceSetup
      True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized Ready}] <nil> nil <nil>}}
      {{app-busybox-cephfs-1 busybox-pvc-3 true { { []} { []}} 0xc00472c000 map[]
      map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs] [ReadWriteMany]
      {map[] map[storage:{{81604378624 0} {<nil>} 76Gi BinarySI}]} [{ReplicationSourceSetup
      True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized Ready}] <nil> nil <nil>}}
      {{app-busybox-cephfs-1 busybox-pvc-9 true { { []} { []}} 0xc00472c060 map[]
      map[app-name:app-busybox-cephfs-1 appname:busybox_app1_cephfs] [ReadWriteMany]
      {map[] map[storage:{{119185342464 0} {<nil>} 111Gi BinarySI}]} [{ReplicationSourceSetup
      True 1 2024-05-28 07:37:05 +0000 UTC SourceInitialized Ready}] <nil> nil <nil>}}]))'
    observedGeneration: 4
    reason: Error
    status: "False"
    type: ClusterDataReady
  kubeObjectProtection: {}
  lastUpdateTime: "2024-05-28T12:04:03Z"
  observedGeneration: 4
  protectedPVCs:
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-2-20240528120120
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-2
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 44Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-20-20240528120119
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-20
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 33Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-1-20240528120149
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-1
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 94Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-13-20240528120148
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-13
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 77Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-17-20240528120117
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-17
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 58Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-14-20240528120117
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-14
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 70Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-18-20240528120119
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-18
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 123Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-8-20240528120136
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-8
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 91Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-7-20240528120148
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-7
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 90Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-11-20240528120113
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-11
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 33Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-10-20240528120149
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-10
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 87Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-15-20240528120112
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-15
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 131Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-4-20240528120112
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-4
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 144Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:06Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-5-20240528120149
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-5
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 107Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-19-20240528120155
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-19
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 61Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-6-20240528120115
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-6
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 123Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-12-20240528120102
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-12
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 147Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-16-20240528120125
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-16
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 127Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-3-20240528120123
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-3
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 76Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  - accessModes:
    - ReadWriteMany
    conditions:
    - lastTransitionTime: "2024-05-28T07:37:05Z"
      message: Ready
      observedGeneration: 1
      reason: SourceInitialized
      status: "True"
      type: ReplicationSourceSetup
    - lastTransitionTime: "2024-05-28T10:32:52Z"
      message: failed to add owner/label to snapshot busybox-pvc-9-20240528120128
        (cross-namespace owner references are disallowed, owner's namespace openshift-dr-ops,
        obj's namespace app-busybox-cephfs-1)
      observedGeneration: 4
      reason: Error
      status: "False"
      type: PVsRestored
    labels:
      app-name: app-busybox-cephfs-1
      appname: busybox_app1_cephfs
    name: busybox-pvc-9
    namespace: app-busybox-cephfs-1
    protectedByVolSync: true
    replicationID:
      id: ""
    resources:
      requests:
        storage: 111Gi
    storageClassName: ocs-storagecluster-cephfs
    storageID:
      id: ""
  state: Unknown