Bug 1699533

Summary:	CronJobs failing due to timezone mismatch
Product:	OpenShift Container Platform	Reporter:	Borja Aranda <farandac>
Component:	Monitoring	Assignee:	Frederic Branczyk <fbranczy>
Status:	CLOSED ERRATA	QA Contact:	Junqi Zhao <juzhao>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	3.11.0	CC:	anpicker, dcaldwel, erooth, mloibl, ocasalsa, pkrupa, rushil, sparpate, ssadhale, surbania
Target Milestone:	---	Keywords:	Reopened
Target Release:	3.11.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-06-26 09:08:06 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Borja Aranda 2019-04-13 14:42:56 UTC

Description of problem:

If there's a timezone mismatch between the controller and kube-state-metrics pods, the cronjobs doesn't work as expected.

There's an upstream issue [1] with a lot more information about this.

[1] https://github.com/openshift/cluster-monitoring-operator/issues/279

Version-Release number of selected component (if applicable):
3.11.*

How reproducible:
Always

Steps to Reproduce:
1. Configure different TZ's for controllers and kube-static-metrics pods.

Actual results:
CronJobs unstable

Expected results:
CronJobs working as intended

Comment 1 Frederic Branczyk 2019-05-10 15:50:18 UTC

I opened https://github.com/openshift/cluster-monitoring-operator/pull/353 with the strategy outlined in https://github.com/openshift/cluster-monitoring-operator/issues/279#issuecomment-484505623

Comment 3 Junqi Zhao 2019-06-14 12:18:14 UTC

Tested with 
ose-cluster-monitoring-operator-v3.11.117-2
ose-kube-state-metrics-v3.11.117-2

create one CronJob
***************************************
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
  namespace: openshift-monitoring
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: busybox
            args:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
          restartPolicy: OnFailure

***************************************
time() and kube_cronjob_next_schedule_time{job="kube-state-metrics",namespace=~"(openshift-.*|kube-.*|default|logging)"} don't have large difference, there is not mismatch in timezone.

eg
time() - kube_cronjob_next_schedule_time{job="kube-state-metrics",namespace=~"(openshift-.*|kube-.*|default|logging)"}, value is
Element	Value
{cronjob="hello",endpoint="https-main",instance="10.130.0.18:8443",job="kube-state-metrics",namespace="openshift-monitoring",pod="kube-state-metrics-69d9644dcc-49xl6",service="kube-state-metrics"}	4.6549999713897705

Comment 5 errata-xmlrpc 2019-06-26 09:08:06 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1605

Comment 6 Simon Pasquier 2019-11-28 16:38:07 UTC

*** Bug 1751542 has been marked as a duplicate of this bug. ***