Bug 2264446

Summary: [4.14 clone][ODF Hackathon] Regional DR cephfs based application failover show warning about subscription
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Raghavendra Talur <rtalur>
Component: odf-drAssignee: Benamar Mekhissi <bmekhiss>
odf-dr sub component: ramen QA Contact: krishnaram Karthick <kramdoss>
Status: ASSIGNED --- Docs Contact:
Severity: low    
Priority: unspecified CC: bmekhiss, ddomingu, etamir, kramdoss, kseeger, muagarwa, rgowdege, rtalur, sheggodu, suvmitra
Version: unspecified   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
.Regional DR Cephfs based application failover show warning about subscription After the application is failed over or relocated, the hub subscriptions show up errors stating, "Some resources failed to deploy. Use View status YAML link to view the details." This is because the application persistent volume claims (PVCs) that use CephFS as the backing storage provisioner, deployed using Red Hat Advanced Cluster Management for Kubernetes (RHACM) subscriptions, and are DR protected are owned by the respective DR controllers. Workaround: There are no workarounds to rectify the errors in the subscription status. However, the subscription resources that failed to deploy can be checked to make sure they are PVCs. This ensures that the other resources do not have problems. If the only resources in the Subscription that fail to deploy are the ones that are DR protected, the error can be ignored.
Story Points: ---
Clone Of: 2257820 Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2257820    

Description Raghavendra Talur 2024-02-15 16:40:53 UTC
+++ This bug was initially created as a clone of Bug #2257820 +++

Description of problem (please be detailed as possible and provide log
snippests):

Used sample application busybox  https://github.com/red-hat-storage/ocm-ramen-samples
Failover and fallback work as expected. We observed one minor warning as below

Error: Some resources failed to deploy. Use View status YAML link to view the 
details.

apiVersion: apps.open-cluster-management.io/v1alpha1
kind: SubscriptionStatus
metadata:
  creationTimestamp: '2024-01-10T15:48:55Z'
  generation: 1
  labels:
    apps.open-cluster-management.io/cluster: primary
    apps.open-cluster-management.io/hosting-subscription: busybox-cephfs.busybox-cephfs-subscription-1
  managedFields:
    - apiVersion: apps.open-cluster-management.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:labels:
            .: {}
            f:apps.open-cluster-management.io/cluster: {}
            f:apps.open-cluster-management.io/hosting-subscription: {}
        f:statuses:
          .: {}
          f:packages: {}
      manager: multicluster-operators-subscription
      operation: Update
      time: '2024-01-10T15:48:55Z'
  name: busybox-cephfs-subscription-1
  namespace: busybox-cephfs
  resourceVersion: '774893'
  uid: 3fa3d5b0-3518-4d9f-8ec9-6f16e8d4a442
statuses:
  packages:
    - apiVersion: v1
      kind: PersistentVolumeClaim
      lastUpdateTime: '2024-01-10T15:48:55Z'
      message: Obj busybox-cephfs/busybox-pvc exists and owned by others, backoff
      name: busybox-pvc
      namespace: busybox-cephfs
      phase: Failed
    - apiVersion: apps/v1
      kind: Deployment
      lastUpdateTime: '2024-01-10T15:48:55Z'
      name: busybox
      namespace: busybox-cephfs
      phase: Deployed


Type: Subscription
API Version: apps.open-cluster-management.io/v1
Namespace: busybox-cephfs
Labels: app=busybox-cephfs,app.kubernetes.io/part-of=busybox-cephfs,apps.open-cluster-management.io/reconcile-rate=medium
Channel: ggithubcom-red-hat-storage-ocm-ramen-samples-ns/ggithubcom-red-hat-storage-ocm-ramen-samples
Placement Ref: kind=Placement,name=busybox-cephfs-placement-1
Git branch: release-4.14
Git path: busybox-odr-cephfs
Cluster deploy status
primary: Subscribed
Error: Some resources failed to deploy. Use View status YAML link to view the details.

Version of all relevant components (if applicable):
ODF 4.14.3-rhodf
ACM 2.9.1
ODF Multicluster Orchestrator 4.14.3


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
No

Is there any workaround available to the best of your knowledge?
NO

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?

Yes
Can this issue reproduce from the UI?
Yes

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Followed instruction in customer documentation 
https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.14/html-single/configuring_openshift_data_foundation_disaster_recovery_for_openshift_workloads/index#subscription-based-apps_manage-rdr

cephfs subscription based application

2.
3.


Actual results:
Minor warning shown

Expected results:
No Warning

Additional info:

--- Additional comment from RHEL Program Management on 2024-01-11 02:42:33 EST ---

This bug having no release flag set previously, is now set with release flag 'odf‑4.15.0' to '?', and so is being proposed to be fixed at the ODF 4.15.0 release. Note that the 3 Acks (pm_ack, devel_ack, qa_ack), if any previously set while release flag was missing, have now been reset since the Acks are to be set against a release flag.

--- Additional comment from Suvendu Mitra on 2024-01-11 03:56:47 EST ---

must-gather uploaded to following google drive
https://drive.google.com/drive/folders/1SzM9q6OU2jlrQNA-rxPyO3kqRjUTXbms?usp=drive_link

--- Additional comment from Benamar Mekhissi on 2024-01-12 09:27:38 EST ---

The warning has been around since 4.13. This occurs after we (Ramen) restore the PV/PVC. Right after that action, the PVC is owned by Ramen for various reasons. As of now, this warning has no effect. We'll investigate later to decide how to make that message disappear.

--- Additional comment from Suvendu Mitra on 2024-01-15 03:12:12 EST ---

@Benamar Mekhissi
It would be helpful to list this as know issue. During failure of any DC customers are already in panic and any error or warning will not be pleasant experience for them.

--- Additional comment from Daniel Dominguez on 2024-02-13 10:31:38 EST ---

Hi Eran,

This issue was discovered during the ODF 4.14 EMEA hackathon but we see little progress. Can you help us?

Thanks! Regards

--- Additional comment from Eran Tamir on 2024-02-14 04:54:48 EST ---

Karolin, can you please make sure we update 4.14 and 4.15 with this known issue in the release notes?