Bug 2183466

Summary: User cannot determine cluster time zone
Product: Red Hat Enterprise Linux 9 Reporter: Simon Foucek <sfoucek>
Component: pacemakerAssignee: Ken Gaillot <kgaillot>
Status: NEW --- QA Contact: cluster-qe <cluster-qe>
Severity: low Docs Contact:
Priority: medium    
Version: 9.2CC: cluster-maint, nwahl
Target Milestone: rcKeywords: FutureFeature, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Feature Request
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Simon Foucek 2023-03-31 09:18:55 UTC 
Description of problem:
A user has a 2-node cluster with nodes X and Y. Node X is in time zone +0, node Y is in time zone +2. On node X it is currently 12:30, on node Y it is 14:30. The user creates a resource and for it a Location rule constraint with a certain time range(12-13).  The user has no way to know which time zone is considered the main time zone for the whole cluster and whether the rule will apply to the whole cluster or not.


Version-Release number of selected component (if applicable):

Pacemaker 2.1.5-7.el9



Steps to Reproduce:
>[root@virt-491 ~]# date
Fri Mar 31 09:11:03 AM UTC 2023
>[root@virt-486 ~]# date
Fri Mar 31 11:08:32 AM CEST 2023
>[root@virt-486 ~]# pcs resource create dummy ocf:heartbeat:Dummy
>[root@virt-486 ~]# pcs status
Cluster name: STSRHTS29864
Status of pacemakerd: 'Pacemaker is running' (last updated 2023-03-31 11:11:47 +02:00)
Cluster Summary:
  * Stack: corosync
  * Current DC: virt-486 (version 2.1.5-7.el9-a3f44794f94) - partition with quorum
  * Last updated: Fri Mar 31 11:11:48 2023
  * Last change:  Fri Mar 31 11:11:44 2023 by root via cibadmin on virt-486
  * 2 nodes configured
  * 3 resource instances configured

Node List:
  * Online: [ virt-486 virt-491 ]

Full List of Resources:
  * fence-virt-486	(stonith:fence_xvm):	 Started virt-486
  * fence-virt-491	(stonith:fence_xvm):	 Started virt-491
  * dummy	(ocf:heartbeat:Dummy):	 Started virt-486

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

>[root@virt-486 ~]# pcs constraint location dummy rule score=-INFINITY date-spec hours='9-10' 
>[root@virt-486 ~]# pcs status
Cluster name: STSRHTS29864
Status of pacemakerd: 'Pacemaker is running' (last updated 2023-03-31 11:13:50 +02:00)
Cluster Summary:
  * Stack: corosync
  * Current DC: virt-486 (version 2.1.5-7.el9-a3f44794f94) - partition with quorum
  * Last updated: Fri Mar 31 11:13:50 2023
  * Last change:  Fri Mar 31 11:13:48 2023 by root via cibadmin on virt-486
  * 2 nodes configured
  * 3 resource instances configured

Node List:
  * Online: [ virt-486 virt-491 ]

Full List of Resources:
  * fence-virt-486	(stonith:fence_xvm):	 Started virt-486
  * fence-virt-491	(stonith:fence_xvm):	 Started virt-491
  * dummy	(ocf:heartbeat:Dummy):	 Started virt-486

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

>[root@virt-486 ~]# pcs constraint location remove location-dummy
>[root@virt-486 ~]# pcs constraint location dummy rule score=-INFINITY date-spec hours='11-12' 
>[root@virt-486 ~]# pcs status
Cluster name: STSRHTS29864
Status of pacemakerd: 'Pacemaker is running' (last updated 2023-03-31 11:14:49 +02:00)
Cluster Summary:
  * Stack: corosync
  * Current DC: virt-486 (version 2.1.5-7.el9-a3f44794f94) - partition with quorum
  * Last updated: Fri Mar 31 11:14:49 2023
  * Last change:  Fri Mar 31 11:14:47 2023 by root via cibadmin on virt-486
  * 2 nodes configured
  * 3 resource instances configured

Node List:
  * Online: [ virt-486 virt-491 ]

Full List of Resources:
  * fence-virt-486	(stonith:fence_xvm):	 Started virt-486
  * fence-virt-491	(stonith:fence_xvm):	 Started virt-491
  * dummy	(ocf:heartbeat:Dummy):	 Stopped

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled


Actual results:
It is not possible to find out which node is used to take the time for the whole cluster and this may imply strange behaviour in edge cases.

Expected results:

In the cluster it will be possible to find out what time zone/time is valid for the whole cluster. If this cannot be found out, it should at least be sufficiently specified and recommended to set a single time zone in the whole cluster and mention the possibility of the problem.
Comment 1 Reid Wahl 2023-03-31 10:00:52 UTC 
The DC (designated controller) node runs the scheduler and evaluates the rules. The evaluation of date-based rules is based on the time on the DC. It looks as if the time used for evaluation is relative to the system's configured time zone, rather than based on UTC. I suspect this was chosen for convenience: many users will find it less error-prone to configure rules based on their local time zone.

So the result of the evaluation depends on which node is DC. There is no persistent "cluster-wide" time -- to the extent that there is such a thing, it's the time of the current DC.

One possible solution might be to add a new "time_zone" field to rules. Another might be to have a cluster-wide time zone property that overrides the system's time zone (via pe_working_set_t:now).
Comment 2 Simon Foucek 2023-03-31 10:56:00 UTC 
If I understand the logic you describe, it would be best to add the possibility to set the TZ for DC, so it would not change when moving between nodes. The user could also see which TZ is set up for the cluster.
Comment 3 Reid Wahl 2023-04-03 08:10:25 UTC 
(In reply to Simon Foucek from comment #2)
> If I understand the logic you describe, it would be best to add the
> possibility to set the TZ for DC, so it would not change when moving between
> nodes. The user could also see which TZ is set up for the cluster.

The way I'm thinking about it, the property would set the time zone for the whole cluster, not just for the DC. It wouldn't have to affect the system time -- only the time that Pacemaker uses.

That's probably also a lot easier to implement than another approach.
Comment 4 Simon Foucek 2023-04-03 08:21:22 UTC 
That seems like a fine solution for me. I would also add the possibility of checking the property's current value.