Bug 512376 - Guest clock is running aprox. 3 seconds before host clock.
Guest clock is running aprox. 3 seconds before host clock.
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: qemu (Show other bugs)
11
All Linux
low Severity medium
: ---
: ---
Assigned To: Glauber Costa
Fedora Extras Quality Assurance
:
Depends On:
Blocks: F11VirtTarget
  Show dependency treegraph
 
Reported: 2009-07-17 11:10 EDT by Jan ONDREJ
Modified: 2013-01-09 16:47 EST (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-09-04 07:55:50 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
Full guest dmesg (20.61 KB, text/plain)
2009-08-12 02:42 EDT, Jan ONDREJ
no flags Details
Guest configuration (1.49 KB, text/plain)
2009-08-12 02:45 EDT, Jan ONDREJ
no flags Details

  None (edit)
Description Jan ONDREJ 2009-07-17 11:10:18 EDT
Description of problem:
My host clock is running 3 seconds before host clock. I can't fix this by syncing my date on host or writing current date to bios clock.

Version-Release number of selected component (if applicable):
qemu-0.10.50-11.kvm88.fc11.i586
but same with fedora 11 stable

How reproducible:
always

Steps to Reproduce:
1. assure, guest has no ntpd or ntpdate started on boot
2. on host: ntpdate -u ntp.upjs.sk # or any other ntp server
3. on host: clock -w
4. boot guest
5. run "date" on guest and or host in one time or run date on both in cycle
6. run: ntpdate -u ntp.upjs.sk # against same ntp server

Actual results:
17 Jul 17:05:29 ntpdate[1332]: step time server 158.197.16.1 offset 2.975406 sec

Expected results:
... offset 0.XXXXX sec

Additional info:
You can see, my clock is aprox. 3 sec different, like real time. You also can see this by running date in same time on host/guest, just it's more complicated to capture this output. :)
It's always 2-3 seconds on all servers. Tested with Fedora 11 stable and also with virt-preview repo updates, with different guests (debian, fedora 10, fedora 11).

After an "reboot", clock is running well, but after next poweroff/start of guest, clock is bad again.

I can start ntpdate, ntpd on boot of guest, but I think precise time is required at boot time too.
Comment 1 Mark McLoughlin 2009-08-07 10:21:38 EDT
No idea what's going on here. Dor? Marcelo?
Comment 2 Dor Laor 2009-08-10 05:08:09 EDT
It is strange, I booted F10 with latest kvm and found no issue there.
What's your host's cpu (/proc/cpuinfo)?
Can you turn off cpuspeed service on the host and re-test?
Anything special on guest's dmesg?
Comment 3 Jan ONDREJ 2009-08-10 11:58:52 EDT
(In reply to comment #2)
> It is strange, I booted F10 with latest kvm and found no issue there.
> What's your host's cpu (/proc/cpuinfo)?

processor       : 0  
vendor_id       : GenuineIntel
cpu family      : 6  
model           : 23 
model name      : Intel(R) Core(TM)2 Duo CPU     E8400  @ 3.00GHz
stepping        : 10 
cpu MHz         : 3000.000
cache size      : 6144 KB
physical id     : 0  
siblings        : 2  
core id         : 0  
cpu cores       : 2  
apicid          : 0  
initial apicid  : 0  
fdiv_bug        : no 
hlt_bug         : no 
f00f_bug        : no 
coma_bug        : no 
fpu             : yes
fpu_exception   : yes
cpuid level     : 13 
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority
bogomips        : 6000.14
clflush size    : 64 
power management:

processor       : 1  
...

I have similar problems on all machines.

> Can you turn off cpuspeed service on the host and re-test?

Yes, similar problem, 2 seconds difference now.
#connecting from host to guest using ssh
[ondrejj@work ~]$ date; ssh root@guest date; date
Po aug 10 17:54:45 CEST 2009
Po aug 10 17:54:43 CEST 2009
Po aug 10 17:54:45 CEST 2009
[ondrejj@work ~]$ 

> Anything special on guest's dmesg?  

I can attach it, if you need. I don't see anything special.

My packages are up-to-date packages for view-preview repo on F11.
Comment 4 Dor Laor 2009-08-11 11:01:19 EDT
Can you re-try when you cancel cpufreq on the host and also make sure it does not go into c2 state - processor.max_cstate=1
Comment 5 Jan ONDREJ 2009-08-11 12:32:49 EDT
No change. Still same problem.

I done this:

chkconfig cpuspeed off
reboot

then:

[ondrejj@work ~]$ cat /proc/cmdline 
ro root=/dev/sda1 vga=0x314 rhgb panic=30 processor.max_cstate=1
[ondrejj@work ~]$ date; ssh 158.197.242.44 -l root date; date
Ut aug 11 18:32:12 CEST 2009
Ut aug 11 18:32:10 CEST 2009
Ut aug 11 18:32:12 CEST 2009
[ondrejj@work ~]$
Comment 6 Dor Laor 2009-08-11 17:54:00 EDT
If your both host/guest are f11 you probably use a kvm pv clock. Can you output 'cat /sys/devices/system/clocksource/clocksource0/current_clocksource'? 
Full guest dmesg might be also helpful
Comment 7 Glauber Costa 2009-08-12 00:12:41 EDT
Please check your rc.sysinit file, and search for any invocation of the hwclock command. Please try removing it, and see if it helps.

I have a hunch that we should not be issuing it in VMs at all...
Comment 8 Jan ONDREJ 2009-08-12 02:41:30 EDT
(In reply to comment #7)
> Please check your rc.sysinit file, and search for any invocation of the hwclock
> command. Please try removing it, and see if it helps.
> 
> I have a hunch that we should not be issuing it in VMs at all...  

I removed hwclock command from my system (mv /sbin/hwclock /sbin/hwclock.old), then it's much better, but still not 0s. Delay is between 0-1 second, as you can see here:

[ondrejj@work ~]$ date; ssh guest -l root date; date
St aug 12 08:35:13 CEST 2009
St aug 12 08:35:12 CEST 2009
St aug 12 08:35:13 CEST 2009
[ondrejj@work ~]$ date; ssh guest root date; date
St aug 12 08:35:08 CEST 2009
St aug 12 08:35:08 CEST 2009
St aug 12 08:35:08 CEST 2009
[ondrejj@work ~]$ date; ssh guest -l root date; date
St aug 12 08:35:09 CEST 2009
St aug 12 08:35:09 CEST 2009
St aug 12 08:35:10 CEST 2009

(In reply to comment #6)
> If your both host/guest are f11 you probably use a kvm pv clock. Can you output
> 'cat /sys/devices/system/clocksource/clocksource0/current_clocksource'? 

[root@localhost ~]# cat /sys/devices/system/clocksource/clocksource0/current_clocksource
kvm-clock

and on my host:
[ondrejj@work ~]$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource
tsc

Is this OK?

> Full guest dmesg might be also helpful  

Will be attached in next comment.
Comment 9 Jan ONDREJ 2009-08-12 02:42:12 EDT
Created attachment 357123 [details]
Full guest dmesg
Comment 10 Jan ONDREJ 2009-08-12 02:45:13 EDT
Created attachment 357124 [details]
Guest configuration

May be you consider guest configuration also useful. Attached too.

Btw, by guest is an fully updated Fedora 11 (stable updates only), but host is Fedora 11 (stable) + virt-preview repository. But I think this happens with F11 stable host too.
Comment 11 Mark McLoughlin 2009-08-14 13:12:24 EDT
(In reply to comment #8)
> (In reply to comment #7)
> > Please check your rc.sysinit file, and search for any invocation of the hwclock
> > command. Please try removing it, and see if it helps.
> > 
> > I have a hunch that we should not be issuing it in VMs at all...  
> 
> I removed hwclock command from my system (mv /sbin/hwclock /sbin/hwclock.old),
> then it's much better, but still not 0s. Delay is between 0-1 second

That certainly sounds a lot more acceptable

Glauber, do you know how hwclock is causing this and whether we can do anything about it?
Comment 12 Glauber Costa 2009-08-17 08:13:45 EDT
I don't know, would have to investigate. I'll take a look at it today, and report whatever I find.
Comment 13 Glauber Costa 2009-08-17 09:49:46 EDT
Can you tell me the exact invocation line of hwclock ?
it is probably in rc.sysinit, but you can probably find it by grepping inside /etc
Comment 14 Jan ONDREJ 2009-08-17 09:57:11 EDT
Me? I don't use it. It's only used by some Fedora scripts, but not by me.
Comment 15 Glauber Costa 2009-08-17 10:49:03 EDT
I can't ask fedora scripts, so I am asking you to provide me info about what they do on your behalf =p

What I want to find out, is what is the invocation of hwclock, and which files do it.
Comment 16 Glauber Costa 2009-08-17 11:28:38 EDT
nevermind

I have a RHEL5 system that suffers from the same problem. Can reproduce, and I'm taking a look
Comment 17 Jan ONDREJ 2009-08-17 11:49:11 EDT
It's called from udev:

[ondrejj@work ~]$ cat /lib/udev/rules.d/88-clock.rules 
ACTION=="add", SUBSYSTEM=="rtc", RUN+="/sbin/hwclock --hctosys --rtc=/dev/%k"
ACTION=="add", ENV{MAJOR}=="10", ENV{MINOR}=="135", RUN+="/sbin/hwclock --hctosys --rtc=/dev/%k"
Comment 18 Mark McLoughlin 2009-08-17 13:20:01 EDT
Glauber proposes removing this rule in #517886
Comment 19 Mark McLoughlin 2009-09-04 07:55:50 EDT
Okay, this should be fixed in rawhide now

See bug #489494 - we're now running hwclock --systz rather than --hctosys

Note You need to log in before you can comment on or make changes to this bug.