Bug 2178304

Summary: [IBM Z]-[Regional DR]-[HUB Recovery] - DR Policy remains in 'Not Validated' state after switching to secondary hub cluster.
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Abdul Kandathil (IBM) <akandath>
Component: documentationAssignee: Olive Lakra <olakra>
Status: ASSIGNED --- QA Contact: Neha Berry <nberry>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.12CC: asriram, egershko, kramdoss, kseeger, muagarwa, odf-bz-bot, olakra, srangana
Target Milestone: ---Flags: srangana: needinfo? (olakra)
Target Release: ODF 4.12.6   
Hardware: s390x   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Abdul Kandathil (IBM) 2023-03-14 18:57:00 UTC
Description of problem (please be detailed as possible and provide log
snippests):
DR Policy remains in 'Not Validated' state after switching to secondary hub cluster.
status says 'DRClustersUnavailable'. 
There is no application deployed.

```
[root@m4216001 ~]# oc get drpolicy ocsm1301015-ocsm4204001-5m -o jsonpath='{.status.conditions[].reason}{"\n"}'
DRClustersUnavailable
[root@m4216001 ~]#
```

Version of all relevant components (if applicable):
OCP : 4.12.0
ODF: 4.12.1-19
MCO: 4.12.1-19

[root@m1301015 ~]# oc -n openshift-dr-system get csv --show-labels
NAME                           DISPLAY                         VERSION   REPLACES                       PHASE       LABELS
odr-cluster-operator.v4.12.1   Openshift DR Cluster Operator   4.12.1    odr-cluster-operator.v4.12.0   Succeeded   operators.coreos.com/odr-cluster-operator.openshift-dr-system=
volsync-product.v0.6.1         VolSync                         0.6.1     volsync-product.v0.6.0         Succeeded   olm.copiedFrom=openshift-operators,operatorframework.io/arch.amd64=supported,operatorframework.io/arch.arm64=supported,operatorframework.io/arch.ppc64le=supported,operatorframework.io/arch.s390x=supported,operatorframework.io/os.linux=supported
[root@m1301015 ~]#

[root@m4216001 ~]# oc -n openshift-operators get csv --show-labels
NAME                                    DISPLAY                         VERSION   REPLACES                   PHASE       LABELS
odf-multicluster-orchestrator.v4.12.1   ODF Multicluster Orchestrator   4.12.1                               Succeeded   operators.coreos.com/odf-multicluster-orchestrator.openshift-operators=
odr-hub-operator.v4.12.1                Openshift DR Hub Operator       4.12.1    odr-hub-operator.v4.12.0   Succeeded   operators.coreos.com/odr-hub-operator.openshift-operators=
[root@m4216001 ~]#




Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Cant manage application after HUB Recovery.


Is there any workaround available to the best of your knowledge?
No


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?
yes


Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Setup Regional DR environment with 2 Hub cluster and 2 managed clusters
2. Perform hub recovery, and create auto-import-secret to see managed clusters in imported state.
3.


Actual results:
DR Policy in "Not Validated"


Expected results:
DR Policy in Validated state.

Additional info:
must gather from all the clusters : https://drive.google.com/file/d/1P0ywE194nlJ-jJ5AphLPTrw8saOIybZw/view?usp=sharing

Comment 1 Abdul Kandathil (IBM) 2023-03-15 09:25:50 UTC
Followed Instructions from MDR Team. : https://docs.google.com/document/d/1DOlkuKpbZJyzWnhll1-pj0jL3dJSzPFY8rvWieMqZU8/edit#

Also tried deleting pods as mentioned in: https://docs.google.com/document/d/1DbTvTgzwWvS3Gupyj7vl6Toa8BuZadkcIoPnnWkWqo4/edit#

Comment 4 Karolin Seeger 2023-05-02 15:36:58 UTC
@akandath please re-test.
Changing to ON_QA.

Comment 5 Abdul Kandathil (IBM) 2023-05-04 17:35:07 UTC
After creating SSL config, the DR policy turned to validated state.