Bug 2276052

+++ This bug was initially created as a clone of Bug #2272528 +++

Description of problem (please be detailed as possible and provide log
snippests):


Version of all relevant components (if applicable):

ACM 2.10 GA'ed
ODF 4.15 GA'ed
ceph version 17.2.6-196.el9cp (cbbf2cfb549196ca18c0c9caff9124d83ed681a4) quincy (stable)
OCP 4.15.0-0.nightly-2024-03-24-023440
VolSync 0.9.0
Submariner 0.17 (GA'ed alongside ACM 2.10)


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?


Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
*****Active hub co-situated with primary managed cluster*****

1. Deployed DR protected 6 RBD and 6 CephFS workloads on C1 over a RDR setup of both subscription and appset types (1 each) and failedover (with all clusters up and running) and relocated them in such a way that they are finally running on C1 and maintain a unique state such as Deployed, FailedOver and Relocated (check drpc output below). Such as if busybox-1 is failedover to C2, it is failedover back to C1 and so on (with all clusters up and running).

We also have 4 workloads (2 RBD and 2 CephFS 1 each of subscription and appset types) on C2 and they remain as it is in the Deployed state.


2. After 2nd operation when workloads are finally running on C1, let IOs continue for some time and configure hub recovery by bringing another OCP cluster which would act as passive hub.
3. Post successful backup creation on both sides, bring C1 and active hub down (basically perform site failure for co-situated hub recovery scenario).
4. Restore backups and move to passive hub.
5. Ensure C2 is successfully imported on passive hub.
6. Check if drpolicy is validated and drpc details for all the workloads are available.
7. Now refresh OCP console of passive hub for Data policies page to appear under Data Services.
8. Now on Data policies --> Disaster recovery, check if the correct app count is shown attached to the drpolicy you have on the setup. If everything looks good, click on hyperlinks under ""Connected applications"" and verify all the apps running on C1 and C2, then move to Data policies --> Overview page.
9. Now select cluster C1 from cluster dropdown and then check if all the applications running on C1 is listed under application dropdown or not.

You will find all appset based apps but not subscription apps. However, everything looks good on C2.


Actual results: [RDR] [UI] Subscription based apps go missing on passive hub from application dropdown of Data policies page after site-failure


Refer attached screencast.


Expected results: Subscription based apps running on C1 cluster (which goes down during site failure, and remains down) should also be listed under application dropdown on the Data policies page.


Additional info:

--- Additional comment from RHEL Program Management on 2024-04-02 00:11:12 IST ---

This bug having no release flag set previously, is now set with release flag 'odf‑4.16.0' to '?', and so is being proposed to be fixed at the ODF 4.16.0 release. Note that the 3 Acks (pm_ack, devel_ack, qa_ack), if any previously set while release flag was missing, have now been reset since the Acks are to be set against a release flag.

--- Additional comment from RHEL Program Management on 2024-04-02 00:11:12 IST ---

The 'Target Release' is not to be set manually at the Red Hat OpenShift Data Foundation product.

The 'Target Release' will be auto set appropriately, after the 3 Acks (pm,devel,qa) are set to "+" for a specific release flag and that release flag gets auto set to "+".

--- Additional comment from RHEL Program Management on 2024-04-02 00:13:06 IST ---

The 'Target Release' is not to be set manually at the Red Hat OpenShift Data Foundation product.

The 'Target Release' will be auto set appropriately, after the 3 Acks (pm,devel,qa) are set to "+" for a specific release flag and that release flag gets auto set to "+".

--- Additional comment from Aman Agrawal on 2024-04-02 00:16:00 IST ---

DRPC from passive hub- 


amanagrawal@Amans-MacBook-Pro hub % drpc
NAMESPACE              NAME                                    AGE   PREFERREDCLUSTER   FAILOVERCLUSTER   DESIREDSTATE   CURRENTSTATE   PROGRESSION            START TIME             DURATION       PEER READY
busybox-workloads-10   rbd-sub-busybox10-placement-1-drpc      28m   amagrawa-c1-29m    amagrawa-c2-29m   Relocate       WaitForUser    Paused                                                       True
busybox-workloads-11   rbd-sub-busybox11-placement-1-drpc      28m   amagrawa-c1-29m                                     WaitForUser    Paused                                                       True
busybox-workloads-12   rbd-sub-busybox12-placement-1-drpc      28m   amagrawa-c1-29m                                     WaitForUser    Paused                                                       True
busybox-workloads-13   rbd-sub-busybox13-placement-1-drpc      28m   amagrawa-c2-29m                                     Deployed       Completed              2024-04-01T18:18:55Z   915.873615ms   True
busybox-workloads-14   cephfs-sub-busybox14-placement-1-drpc   28m   amagrawa-c2-29m    amagrawa-c1-29m   Failover       WaitForUser    Paused                                                       True
busybox-workloads-15   cephfs-sub-busybox15-placement-1-drpc   28m   amagrawa-c1-29m    amagrawa-c2-29m   Relocate       WaitForUser    Paused                                                       True
busybox-workloads-16   cephfs-sub-busybox16-placement-1-drpc   28m   amagrawa-c1-29m                                     WaitForUser    Paused                                                       True
busybox-workloads-17   cephfs-sub-busybox17-placement-1-drpc   28m   amagrawa-c2-29m                                     Deployed       EnsuringVolSyncSetup   2024-04-01T18:18:55Z                  True
busybox-workloads-9    rbd-sub-busybox9-placement-1-drpc       28m   amagrawa-c2-29m    amagrawa-c1-29m   Failover       WaitForUser    Paused                                                       True
openshift-gitops       cephfs-appset-busybox5-placement-drpc   28m   amagrawa-c2-29m    amagrawa-c1-29m   Failover       WaitForUser    Paused                                                       True
openshift-gitops       cephfs-appset-busybox6-placement-drpc   28m   amagrawa-c1-29m    amagrawa-c2-29m   Relocate       WaitForUser    Paused                                                       True
openshift-gitops       cephfs-appset-busybox7-placement-drpc   28m   amagrawa-c1-29m                                     WaitForUser    Paused                                                       True
openshift-gitops       cephfs-appset-busybox8-placement-drpc   28m   amagrawa-c2-29m                                     Deployed       EnsuringVolSyncSetup   2024-04-01T18:18:55Z                  True
openshift-gitops       rbd-appset-busybox1-placement-drpc      28m   amagrawa-c2-29m    amagrawa-c1-29m   Failover       WaitForUser    Paused                                                       True
openshift-gitops       rbd-appset-busybox2-placement-drpc      28m   amagrawa-c1-29m    amagrawa-c2-29m   Relocate       WaitForUser    Paused                                                       True
openshift-gitops       rbd-appset-busybox3-placement-drpc      28m   amagrawa-c1-29m                                     WaitForUser    Paused                                                       True
openshift-gitops       rbd-appset-busybox4-placement-drpc      28m   amagrawa-c2-29m                                     Deployed       Completed              2024-04-01T18:18:55Z   780.468742ms   True




Screencast- https://drive.google.com/file/d/1vsmCo7jAc5Py9B-pYI2JyInhSiWSXxnI/view?usp=sharing

--- Additional comment from Aman Agrawal on 2024-04-02 01:19:33 IST ---

Logs- http://rhsqe-repo.lab.eng.blr.redhat.com/OCS/ocs-qe-bugs/bz-aman/01april24-bz2272528/

--- Additional comment from gowtham on 2024-04-02 16:40:32 IST ---

I don't think this is a UI issue, UI finds the current deployment cluster name using DRPC annotation "drplacementcontrol.ramendr.openshift.io/last-app-deployment-cluster" or ACM Placement CR status, if both are missing then UI can't find under which cluster to display this app, Please check these two values and if it is missing move this BZ to ramen backend.

--- Additional comment from Aman Agrawal on 2024-04-02 16:52:42 IST ---

(In reply to gowtham from comment #6)
> I don't think this is a UI issue, UI finds the current deployment cluster
> name using DRPC annotation
> "drplacementcontrol.ramendr.openshift.io/last-app-deployment-cluster" or ACM
> Placement CR status, if both are missing then UI can't find under which
> cluster to display this app, Please check these two values and if it is
> missing move this BZ to ramen backend.

The setup was auto-destroyed today morning. Could we check and confirm it from the logs attached above?

--- Additional comment from gowtham on 2024-04-02 17:07:50 IST ---

Got the issue! subscription code is not using "drplacementcontrol.ramendr.openshift.io/last-app-deployment-cluster" annotation from DRPC but appset is referring

--- Additional comment from Aman Agrawal on 2024-04-03 19:57:28 IST ---

Looping Benamar as discussed in the DR meeting.

--- Additional comment from gowtham on 2024-04-03 20:02:02 IST ---

it is a UI issue, the UI is missing to read cluster name from DRPC annotation.

--- Additional comment from RHEL Program Management on 2024-04-17 13:00:19 IST ---

This BZ is being approved for ODF 4.16.0 release, upon receipt of the 3 ACKs (PM,Devel,QA) for the release flag 'odf‑4.16.0

--- Additional comment from RHEL Program Management on 2024-04-17 13:00:19 IST ---

Since this bug has been approved for ODF 4.16.0 release, through release flag 'odf-4.16.0+', the Target Release is being set to 'ODF 4.16.0