Bug 2255998
| Summary: | [RDR] [Hub recovery] Failover for most cephfs and a few rbd workloads remain stuck at WaitingForResourceRestore | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat OpenShift Data Foundation | Reporter: | Aman Agrawal <amagrawa> |
| Component: | odf-dr | Assignee: | Benamar Mekhissi <bmekhiss> |
| odf-dr sub component: | ramen | QA Contact: | Aman Agrawal <amagrawa> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | urgent | ||
| Priority: | unspecified | CC: | akandath, bmekhiss, ebenahar, kseeger, muagarwa |
| Version: | 4.14 | ||
| Target Milestone: | --- | ||
| Target Release: | ODF 4.16.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | 4.15.0-146 | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2024-07-17 13:11:49 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Comment 17
Benamar Mekhissi
2024-02-13 17:23:49 UTC
Tested with following versions: ceph version 18.2.1-188.el9cp (b1ae9c989e2f41dcfec0e680c11d1d9465b1db0e) reef (stable) OCP 4.16.0-0.nightly-2024-05-23-173505 ACM 2.11.0-DOWNSTREAM-2024-05-23-15-16-26 MCE 2.6.0-104 ODF 4.16.0-108.stable Gitops v1.12.3 Submariner 0.18.0 (image: brew.registry.redhat.io/rh-osbs/iib:722673) Platform- VMware When the steps to reproduce is repeated, multiple RBD and CephFS workloads with distinct states like Deployed, FailedOver and Relocated of both appset (pull) and subscription types were failedover in the final step from cluster amagrawa-c2-my28 to amagrawa-c1-28my and the failover was successful for all of them. Workload also included 2 CNV(RBD) workloads of both appset(pull) and subscription types (cnv-sub-busybox18-placement-1-drpc and cnv-appset-busybox17-placement-drpc). date -u; oc get drpc -o wide -A; echo "/////////////////////////////////////////////////////////////////////" Fri May 31 19:11:38 UTC 2024 NAMESPACE NAME AGE PREFERREDCLUSTER FAILOVERCLUSTER DESIREDSTATE CURRENTSTATE PROGRESSION START TIME DURATION PEER READY busybox-workloads-101 rbd-sub-busybox101-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:03Z False busybox-workloads-103 cephfs-sub-busybox103-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:02:41Z False busybox-workloads-13 cephfs-sub-busybox13-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:02:48Z False busybox-workloads-14 cephfs-sub-busybox14-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:02:29Z False busybox-workloads-15 cephfs-sub-busybox15-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:02:16Z False busybox-workloads-16 cephfs-sub-busybox16-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:02:09Z False busybox-workloads-18 cnv-sub-busybox18-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:06:24Z False busybox-workloads-20 rbd-sub-busybox20-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:10Z False busybox-workloads-22 cephfs-sub-busybox22-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:02:23Z False busybox-workloads-5 rbd-sub-busybox5-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:03:50Z False busybox-workloads-6 rbd-sub-busybox6-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:22Z False busybox-workloads-7 rbd-sub-busybox7-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:16Z False busybox-workloads-8 rbd-sub-busybox8-placement-1-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:03:57Z False openshift-gitops cephfs-appset-busybox10-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:02:53Z False openshift-gitops cephfs-appset-busybox102-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:02:58Z False openshift-gitops cephfs-appset-busybox11-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:03:03Z False openshift-gitops cephfs-appset-busybox12-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:03:07Z False openshift-gitops cephfs-appset-busybox21-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:03:14Z False openshift-gitops cephfs-appset-busybox9-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:03:18Z False openshift-gitops cnv-appset-busybox17-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:06:30Z False openshift-gitops rbd-appset-busybox1-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:28Z False openshift-gitops rbd-appset-busybox100-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:34Z False openshift-gitops rbd-appset-busybox19-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:39Z False openshift-gitops rbd-appset-busybox2-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:46Z False openshift-gitops rbd-appset-busybox3-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:50Z False openshift-gitops rbd-appset-busybox4-placement-drpc 31m amagrawa-c2-my28 amagrawa-c1-28my Failover FailedOver Cleaning Up 2024-05-31T19:04:54Z False ///////////////////////////////////////////////////////////////////// Since the cluster amagrawa-c2-my28 is still down, drpc progression is reporting Cleaning Up which is expected. The issue reported in this BZ was not hit, therefore I am marking this bug as verified. Fix LGTM Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.16.0 security, enhancement & bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2024:4591 |