Bug 2136457

Summary: [RDR] VRG reports lastGroupSyncTime as epoch instead of actual sync time
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Sidhant Agrawal <sagrawal>
Component: csi-addonsAssignee: Madhu Rajanna <mrajanna>
Status: CLOSED CURRENTRELEASE QA Contact: Sidhant Agrawal <sagrawal>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.12CC: mrajanna, muagarwa, ocs-bugs, odf-bz-bot, srangana
Target Milestone: ---   
Target Release: ODF 4.12.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.12.0-100 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 2139323 (view as bug list) Environment:
Last Closed: 2023-02-08 14:06:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2139323    

Description Sidhant Agrawal 2022-10-20 10:32:33 UTC 
Description of problem (please be detailed as possible and provide log
snippests):
In RDR setup, with a workload consisting of ~20 PVC and Sync schedule of DRPolicy 5m,
observed that DRPC and VRG were reporting lastGroupSyncTime: "1970-01-01T00:00:00Z" even after 60+ minutes of application deployment.
This appears to be because of few VR having epoch as lastSyncTime.

For eg. See the following outputs showing lastGroupSyncTime and lastSyncTime from VRG's yaml output
------------
DRPC status:
...
Thu Oct 20 09:56:50 UTC 2022
NAMESPACE             NAME           AGE    PREFERREDCLUSTER   FAILOVERCLUSTER   DESIREDSTATE   CURRENTSTATE   PROGRESSION   START TIME             DURATION         PEER READY
busybox-workloads-1   busybox-drpc   158m   sagrawal-c1                                         Deployed       Completed     2022-10-20T07:19:18Z   1m0.039841994s   True
...

VRG status:
-------------------
Thu Oct 20 09:57:28 UTC 2022
NAMESPACE             NAME           DESIREDSTATE   CURRENTSTATE
busybox-workloads-1   busybox-drpc   primary        Primary
    lastGroupSyncTime: "2022-10-20T09:50:01Z"
      lastSyncTime: "2022-10-20T09:55:01Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:55:01Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:50:01Z"
      lastSyncTime: "2022-10-20T09:55:01Z"
      lastSyncTime: "2022-10-20T09:50:01Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:55:01Z"
      lastSyncTime: "2022-10-20T09:50:02Z"
      lastSyncTime: "2022-10-20T09:50:01Z"
      lastSyncTime: "2022-10-20T09:50:01Z"
      lastSyncTime: "2022-10-20T09:55:01Z"
      lastSyncTime: "2022-10-20T09:55:01Z"
-------------------
Thu Oct 20 10:00:39 UTC 2022
NAMESPACE             NAME           DESIREDSTATE   CURRENTSTATE
busybox-workloads-1   busybox-drpc   primary        Primary
    lastGroupSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T10:00:00Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T10:00:00Z"
      lastSyncTime: "2022-10-20T10:00:00Z"
      lastSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T10:00:00Z"
      lastSyncTime: "2022-10-20T09:55:02Z"
      lastSyncTime: "2022-10-20T10:00:00Z"
      lastSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T10:00:00Z"
      lastSyncTime: "2022-10-20T10:00:00Z"
------------
Thu Oct 20 10:05:59 UTC 2022
NAMESPACE             NAME           DESIREDSTATE   CURRENTSTATE
busybox-workloads-1   busybox-drpc   primary        Primary
    lastGroupSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:00:01Z"
      lastSyncTime: "2022-10-20T10:00:01Z"
      lastSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T10:00:01Z"
      lastSyncTime: "2022-10-20T10:00:01Z"
      lastSyncTime: "2022-10-20T10:00:01Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T10:00:01Z"
      lastSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T10:00:01Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:00:01Z"
      lastSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
------------
Thu Oct 20 10:10:14 UTC 2022
NAMESPACE             NAME           DESIREDSTATE   CURRENTSTATE
busybox-workloads-1   busybox-drpc   primary        Primary
    lastGroupSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:01Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
      lastSyncTime: "2022-10-20T10:05:00Z"
------------
Thu Oct 20 10:15:32 UTC 2022
NAMESPACE             NAME           DESIREDSTATE   CURRENTSTATE
busybox-workloads-1   busybox-drpc   primary        Primary
    lastGroupSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T10:10:00Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:10:00Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:15:01Z"
      lastSyncTime: "2022-10-20T10:15:01Z"
      lastSyncTime: "2022-10-20T10:10:00Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:15:01Z"
      lastSyncTime: "2022-10-20T10:10:01Z"
      lastSyncTime: "2022-10-20T10:15:01Z"
      lastSyncTime: "2022-10-20T10:15:01Z"
      lastSyncTime: "1970-01-01T00:00:00Z"
      lastSyncTime: "2022-10-20T10:15:01Z"
-------------

Version of all relevant components (if applicable):
OCP: 4.12.0-0.nightly-2022-10-18-192348
ODF: 4.12.0-77
ACM: 2.6.1
Submariner: 0.13.0

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?

Is there any workaround available to the best of your knowledge?

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
2

Can this issue reproducible?

Can this issue reproduce from the UI?

If this is a regression, please provide more details to justify this:

Steps to Reproduce:
1. Configure RDR setup
2. Deploy an application containing multiple PVCs/Pods 
3. Check VRG status at different intervals

Actual results:
VRG reports lastGroupSyncTime as epoch instead of actual time.

Expected results:
VRG should not randomly report lastGroupSyncTime as epoch.

Additional info: