Created attachment 1954358 [details] DRpolicy_creation_ss Description of problem (please be detailed as possible and provide log snippests): Metro Disaster Recovery policy creation fails unless the ocs-operator pod is restarted on the managed clusters. All the clusters (hub and managed) have the same ODF version v4.13.0-110.stable Version of all relevant components (if applicable): openshift-install: 4.13.0-rc.0 ODF: v4.13.0-110.stable odr-hub-operator: v4.13.0-110.stable odf-multicluster-orchestrator: v4.13.0-110.stable ACM: 2.7.2 Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? Is there any workaround available to the best of your knowledge? Restart the ocs-operator pod on managed clusters Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? Can this issue reproducible? Yes, reproduced every time (5 times) Can this issue reproduce from the UI? Yes If this is a regression, please provide more details to justify this: Steps to Reproduce: 1. Create Metro DR setup with hub cluster and 2 managed clusters 2. Install ODF on managed clusters and connect to external storage 3. Install ODF Multicluster Operator on Hub cluster 4. Configure SSL access across clusters 5. Create Disaster recovery policy on hub cluster Actual results: Creation of DR policy reports unsupported version of ODF on the managed clusters, although the ODF version is supported and same on all the 3 clusters. Disaster recovery policy creation fails unless the ocs-operator pod is restarted on the Managed clusters. Attaching the screenshot of the DR policy creation failure. Expected results: Disaster recovery policy creation should be possible without restarting the ocs-operator pod on Managed clusters. Additional info:
Created attachment 1954359 [details] ocs-operator-log-on-managed-cluster
Looking at the log, this does not look like a DR issue. It's a bug somewhere in ocs-operator. We should try to recreate it without DR to isolate the issue. Logs shows StorageCluster has this namespace/name: `"Request.Namespace":"openshift-storage","Request.Name":"ocs-external-storagecluster"` But, ocs-operator is looking for `"msg":"No StorageCluster resource.","Request.Namespace":"openshift-storage","Request.Name":"ocsinit"` Moving it to ocs-operator for RCA.
This might have occurred because of another bug: https://bugzilla.redhat.com/show_bug.cgi?id=2185188. Once that is fixed, we need to check if this issue still exists.
Please verify the issue with "4.13.0-166" build.
Sravika, please reopen if this still exists.
@uchapaga @muagarwa : Currently I don't have 4.13 environment, will verify the BZ once I move to 4.13 verification later
Reopening the issue as we were able to reproduce it on latest builds.
We need to fix this in 4.13 as it blocks DR workflows on clusters with existing ODF deployments.
After applying the below CatalogSource I am able to install the MCO operator from the operator hub on the IBM Z platform. apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: labels: ocs-operator-internal: 'true' name: redhat-operators namespace: openshift-marketplace spec: displayName: Openshift Container Storage icon: base64data: '' mediatype: '' image: quay.io/rhceph-dev/ocs-registry:latest-stable-4.13 priority: 100 publisher: Red Hat sourceType: grpc updateStrategy: registryPoll: interval: 15m
With the latest ODF build 4.13.0-203.stable, MDR policy creation is successful without restarting the ocs-operator pods on the managed clusters
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenShift Data Foundation 4.13.0 enhancement and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:3742