Bug 1813883

Summary: [GSS] emphasize the importance of time synchronization
Product: [Red Hat Storage] Red Hat OpenShift Container Storage Reporter: Arne Gogala <agogala>
Component: documentationAssignee: Kusuma <kbg>
Status: CLOSED CURRENTRELEASE QA Contact: Neha Berry <nberry>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.2CC: asriram, assingh, ebenahar, hnallurv, kbg, nberry, ocs-bugs
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-04-16 14:36:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1797537    
Attachments:
Description Flags
NOTE none

Description Arne Gogala 2020-03-16 12:03:38 UTC 
* Description of problem (please be detailed as possible and provide log
snippests):

Documentation should emphasize the importance of time synchronization for OCS, as Ceph heavily depends on that.
This is especially important for customers going to use OCS in a restricted/disconnected network environment.

This happened to my customer:

- successfully installed OCP 4.3 on VMWare UPI in a restricted network
- after some some time (hours, days) they have deployed OCS 4.2.2 on OCP, time drift has happened on their VMs
- we've noticed that OCS stays in "degraded" state with CEPH status reporting "HEALTH_WARN, clock skew detected on mon.x"
- we've found Red Hat solution [ A newly deployed OCS 4 cluster status shows as "Degraded", Why? ] [1]
- with the information in [2], they were able to configure time-synchronization without full internet connectivity
- OCS 4.2.2 started to work

[1] https://access.redhat.com/solutions/4828941
[2] https://docs.openshift.com/container-platform/4.3/installing/install_config/installing-customizing.html#installation-special-config-crony_installing-customizing


* Is there any workaround available to the best of your knowledge?

Yes, KCS https://access.redhat.com/solutions/4828941
There are already many support cases linked to this one, so it will be very useful for our customers to have a better documentation here.

* Proposed documentation enhancement:

- Point to the importance of time synchronization in part [4] and/or [5] in the OCS 4.2 documentation
- Outline the need for configuring chrony to use an internal time source [2] when installing in a restricted network environment

[4] https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.2/html-single/planning_your_deployment/index
[5] https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.2/html-single/deploying_openshift_container_storage/index
Comment 4 Oded 2020-04-07 11:36:15 UTC 
Need Add Note [1] to this doc:
https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.2/html-single/deploying_openshift_container_storage/index



Note [1]:
NOTE
When you install OpenShift Container Storage in a restricted network environment, you need to apply a custom Network Time Protocol (NTP) configuration to the nodes, because by default, internet connectivity is assumed in OpenShift Container Platform and chronyd is configured to use *.rhel.pool.ntp.org servers. See https://access.redhat.com/solutions/4828941 and Configuring chrony time service for more details.

*Attached picture
Comment 5 Oded 2020-04-07 11:42:15 UTC 
Created attachment 1676889 [details]
NOTE