Bug 1702544 - Elasticsearch failed to start due to invalid cert date if hosts have non-UTC timezone
Summary: Elasticsearch failed to start due to invalid cert date if hosts have non-UTC ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 3.11.z
Assignee: Noriko Hosoi
QA Contact: Anping Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-24 05:38 UTC by Kenjiro Nakayama
Modified: 2019-06-26 09:08 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: When OpenShift Nodes's timezone is not set to UTC, the current non-UTC timestamp is used for the NotBefore checking. Consequence: If the timezone is ahead of UTC, the NotBefore checking fails. Fix: Regardless of the nodes' timezone, getting the UTC timestamp and set it to "startdate" in the certificates. Result: No more failure due to the non-UTC current timestamp.
Clone Of:
Environment:
Last Closed: 2019-06-26 09:08:06 UTC
Target Upstream Version:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:1605 0 None None None 2019-06-26 09:08:14 UTC

Description Kenjiro Nakayama 2019-04-24 05:38:17 UTC 
Description of problem:

- When OpenShift Nodes set timezone as non-UTC, elasticsearch failed to start due to invalid cert date.
- Error in logging-es:

  Caused by: java.security.cert.CertificateNotYetValidException: NotBefore: Tue Apr 23 10:42:11 UTC 2019
          at sun.security.x509.CertificateValidity.valid(CertificateValidity.java:270) ~[?:?]
          at sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:629) ~[?:?]
   	 ... snip ...
          at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1301) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
          at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1214) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
          ... 18 more

Version-Release number of the following components:
- openshift-ansbible-playbook-3.11.98-1.git.0.3cfa7c3.el7.noarch

How reproducible: 100%

Steps to Reproduce:
1. Set host's timezone to non-UTC(e.g GMT+9) like "# timedatectl set-timezone Asia/Shanghai"
2. Run installer (ansible playbook completes but es pod does not start well.)

Actual results:
- "Caused by: java.security.cert.CertificateNotYetValidException: NotBefore: Tue Apr 23 10:42:11 UTC 2019" error happens.
- The installation completes around 10:42:11 GMT+9, hence the cert seems to handle wrong timezone "NotBefore: Tue Apr 23 10:42:11 UTC 2019".

Expected results:
- certs should be valid as correct valid date.

Additional info:
- After 9 hours later, es pod starts fine.
Comment 3 Noriko Hosoi 2019-05-03 22:49:50 UTC 
https://github.com/openshift/openshift-ansible/pull/11584
Comment 5 Anping Li 2019-06-13 07:59:41 UTC 
The es pod can be started in openshift3/ose-logging-elasticsearch5:v3.11.117
 timedatectl status
      Local time: Thu 2019-06-13 15:59:19 CST
  Universal time: Thu 2019-06-13 07:59:19 UTC
        RTC time: Thu 2019-06-13 07:59:19
       Time zone: Asia/Shanghai (CST, +0800)
Comment 7 errata-xmlrpc 2019-06-26 09:08:06 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1605
Note You need to log in before you can comment on or make changes to this bug.