Bug 1404509

Summary: [RFE] Log collector should collect time diff for all hosts
Product: Red Hat Enterprise Virtualization Manager Reporter: Marcus West <mwest>
Component: ovirt-log-collectorAssignee: Sandro Bonazzola <sbonazzo>
Status: CLOSED ERRATA QA Contact: David Necpal <dnecpal>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.0.6CC: lsurette, lsvaty, mkalinin, pbrilla, rbalakri, srevivo, trichard, ykaul, ylavi
Target Milestone: ovirt-4.2.0Keywords: FutureFeature, ZStream
Target Release: 4.2.0Flags: dnecpal: testing_plan_complete-
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: ovirt-log-collector-4.2.0-1.el7ev Doc Type: Enhancement
Doc Text:
When collecting SOS reports from hosts, chrony and systemd SOS plugins can now collect information about time synchronization. In addition, a new --time-only option has been added to ovirt-log-collector, allowing information about time differences to be gathered from the hosts without gathering full SOS reports, saving a considerable amount of time for the operation.
Story Points: ---
Clone Of:
: 1455771 (view as bug list) Environment:
Last Closed: 2018-05-15 17:31:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1455771    

Description Marcus West 2016-12-14 01:14:32 UTC
## Description of problem:

It would be great if Log Collector checked the time diff for all hosts, regardless of how many hosts (sosreport) are selected.

## Version-Release number of selected component (if applicable):

ovirt-log-collector-4.0.1-1.el7ev.noarch

## How reproducible:

always

## Steps to Reproduce:

1. generate Log Collector
2. don't specify any hosts (or only specify one, etc)

## Actual results:

time_diff.txt only collects info for hosts that are specified

## Expected results:

collect time diff info for all hosts

## Additional info:

Sometimes when troubleshooting issues, we get LC ouput without any host info, then get sosreports separately.  More commmon, if we are troublehsooting a migration issue, we only get the destination host but not the source (so we can't check that both are in sync)

It would be very useful if we could include this info (time diff for all hosts) so that we can easily 'tick off' out-of-sync hosts as a cause for various problems.

Alternative - provide some easy way for the customer (or Support) to verify all hosts are in sync.  I've been getting customers to run: (one line)

for i in `cat hosts.txt`; do echo $i; ssh -i /etc/pki/ovirt-engine/keys/engine_id_rsa root@${i} 'date --iso-8601=seconds'; date --iso-8601=seconds; done


Where 'hosts.txt' is a list of hosts (from 'ovirt-log-collector list')  Is there a better way to see at a glance if all hosts are in sync?

Comment 2 Sandro Bonazzola 2017-05-16 08:23:53 UTC
A possible way to do

Comment 3 Sandro Bonazzola 2017-05-16 08:25:31 UTC
A possible way to do a check against time drift is to leverage ntpq or chronyc commands. Need to check if sos already collect this kind of data.

Comment 4 Sandro Bonazzola 2017-05-24 11:48:37 UTC
The systemd sos plugin provides the output from timedatectl:

$ timedatectl 
      Local time: mer 2017-05-24 13:35:46 CEST
  Universal time: mer 2017-05-24 11:35:46 UTC
        RTC time: mer 2017-05-24 11:35:46
       Time zone: Europe/Rome (CEST, +0200)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no


and the chrony plugin provides time drift:

# cat chronyc_sources
210 Number of sources = 4
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^? blue.1e400.net                0   9     0     -     +0ns[   +0ns] +/-    0ns
^* ntp-2.arkena.net              2   7   355   813  -2808us[-4156us] +/-   41ms
^? server1.websters-compute>     2   9     4   21m    +13ms[  +12ms] +/-   50ms
^+ iris.linocomm.net             2   7   377    39  -9300us[-9300us] +/-   78ms

# cat chronyc_tracking
Reference ID    : 5F51AD4A (ntp-2.arkena.net)
Stratum         : 3
Ref time (UTC)  : Wed May 24 11:18:51 2017
System time     : 0.000000059 seconds fast of NTP time
Last offset     : -0.001348147 seconds
RMS offset      : 0.000891597 seconds
Frequency       : 9.894 ppm fast
Residual freq   : -0.298 ppm
Skew            : 2.514 ppm
Root delay      : 0.050097819 seconds
Root dispersion : 0.018574700 seconds
Update interval : 518.1 seconds
Leap status     : Normal

Comment 5 Sandro Bonazzola 2017-05-24 17:27:18 UTC
Also added --time-only option to skip sos collection from specified hosts, in particular, from all known hosts

Comment 10 David Necpal 2017-09-04 19:37:23 UTC
Bug/problem isn't solved

Tested on version:
ovirt-log-collector-4.2.0-0.0.master.20170903141131.gitbd2607f.el7.centos.noarch

After run command 'ovirt-log-collector', created sosreport -specifically time_diff.txt- contains info only for hosts that are specified, not for all hosts.

Tested for zero, one and three(all) hosts

Comment 11 Sandro Bonazzola 2017-09-05 05:29:11 UTC
Please check for systemd sos plugin data collected from command: timedatectl
Please also check for chrony sos plugin data collected from chornyc sources and chronyc tracking commands.

In order to collectd time dirft from all host you need to supply --time-only option to skip sos collection and just collect time drift data.

Comment 12 David Necpal 2017-09-07 11:14:25 UTC
After retest verified on version: 
ovirt-log-collector-4.2.0-0.0.master.20170903141131.gitbd2607f.el7.centos.noarch

Comment 17 errata-xmlrpc 2018-05-15 17:31:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1465

Comment 18 Franta Kust 2019-05-16 13:07:27 UTC
BZ<2>Jira Resync