Bug 1702544

Summary: Elasticsearch failed to start due to invalid cert date if hosts have non-UTC timezone
Product: OpenShift Container Platform Reporter: Kenjiro Nakayama <knakayam>
Component: LoggingAssignee: Noriko Hosoi <nhosoi>
Status: CLOSED ERRATA QA Contact: Anping Li <anli>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 3.11.0CC: aos-bugs, jcantril, nhosoi, rmeggins
Target Milestone: ---   
Target Release: 3.11.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: When OpenShift Nodes's timezone is not set to UTC, the current non-UTC timestamp is used for the NotBefore checking. Consequence: If the timezone is ahead of UTC, the NotBefore checking fails. Fix: Regardless of the nodes' timezone, getting the UTC timestamp and set it to "startdate" in the certificates. Result: No more failure due to the non-UTC current timestamp.
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-26 09:08:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kenjiro Nakayama 2019-04-24 05:38:17 UTC
Description of problem:

- When OpenShift Nodes set timezone as non-UTC, elasticsearch failed to start due to invalid cert date.
- Error in logging-es:

  Caused by: java.security.cert.CertificateNotYetValidException: NotBefore: Tue Apr 23 10:42:11 UTC 2019
          at sun.security.x509.CertificateValidity.valid(CertificateValidity.java:270) ~[?:?]
          at sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:629) ~[?:?]
   	 ... snip ...
          at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1301) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
          at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1214) ~[netty-handler-4.1.13.Final.jar:4.1.13.Final]
          ... 18 more

Version-Release number of the following components:
- openshift-ansbible-playbook-3.11.98-1.git.0.3cfa7c3.el7.noarch

How reproducible: 100%

Steps to Reproduce:
1. Set host's timezone to non-UTC(e.g GMT+9) like "# timedatectl set-timezone Asia/Shanghai"
2. Run installer (ansible playbook completes but es pod does not start well.)

Actual results:
- "Caused by: java.security.cert.CertificateNotYetValidException: NotBefore: Tue Apr 23 10:42:11 UTC 2019" error happens.
- The installation completes around 10:42:11 GMT+9, hence the cert seems to handle wrong timezone "NotBefore: Tue Apr 23 10:42:11 UTC 2019".

Expected results:
- certs should be valid as correct valid date.

Additional info:
- After 9 hours later, es pod starts fine.

Comment 5 Anping Li 2019-06-13 07:59:41 UTC
The es pod can be started in openshift3/ose-logging-elasticsearch5:v3.11.117
 timedatectl status
      Local time: Thu 2019-06-13 15:59:19 CST
  Universal time: Thu 2019-06-13 07:59:19 UTC
        RTC time: Thu 2019-06-13 07:59:19
       Time zone: Asia/Shanghai (CST, +0800)

Comment 7 errata-xmlrpc 2019-06-26 09:08:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1605