Bug 512376

No idea what's going on here. Dor? Marcelo?

It is strange, I booted F10 with latest kvm and found no issue there.
What's your host's cpu (/proc/cpuinfo)?
Can you turn off cpuspeed service on the host and re-test?
Anything special on guest's dmesg?

(In reply to comment #2)
> It is strange, I booted F10 with latest kvm and found no issue there.
> What's your host's cpu (/proc/cpuinfo)?

processor       : 0  
vendor_id       : GenuineIntel
cpu family      : 6  
model           : 23 
model name      : Intel(R) Core(TM)2 Duo CPU     E8400  @ 3.00GHz
stepping        : 10 
cpu MHz         : 3000.000
cache size      : 6144 KB
physical id     : 0  
siblings        : 2  
core id         : 0  
cpu cores       : 2  
apicid          : 0  
initial apicid  : 0  
fdiv_bug        : no 
hlt_bug         : no 
f00f_bug        : no 
coma_bug        : no 
fpu             : yes
fpu_exception   : yes
cpuid level     : 13 
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority
bogomips        : 6000.14
clflush size    : 64 
power management:

processor       : 1  
...

I have similar problems on all machines.

> Can you turn off cpuspeed service on the host and re-test?

Yes, similar problem, 2 seconds difference now.
#connecting from host to guest using ssh
[ondrejj@work ~]$ date; ssh root@guest date; date
Po aug 10 17:54:45 CEST 2009
Po aug 10 17:54:43 CEST 2009
Po aug 10 17:54:45 CEST 2009
[ondrejj@work ~]$ 

> Anything special on guest's dmesg?  

I can attach it, if you need. I don't see anything special.

My packages are up-to-date packages for view-preview repo on F11.

Can you re-try when you cancel cpufreq on the host and also make sure it does not go into c2 state - processor.max_cstate=1

No change. Still same problem.

I done this:

chkconfig cpuspeed off
reboot

then:

[ondrejj@work ~]$ cat /proc/cmdline 
ro root=/dev/sda1 vga=0x314 rhgb panic=30 processor.max_cstate=1
[ondrejj@work ~]$ date; ssh 158.197.242.44 -l root date; date
Ut aug 11 18:32:12 CEST 2009
Ut aug 11 18:32:10 CEST 2009
Ut aug 11 18:32:12 CEST 2009
[ondrejj@work ~]$

If your both host/guest are f11 you probably use a kvm pv clock. Can you output 'cat /sys/devices/system/clocksource/clocksource0/current_clocksource'? 
Full guest dmesg might be also helpful

Please check your rc.sysinit file, and search for any invocation of the hwclock command. Please try removing it, and see if it helps.

I have a hunch that we should not be issuing it in VMs at all...

(In reply to comment #7)
> Please check your rc.sysinit file, and search for any invocation of the hwclock
> command. Please try removing it, and see if it helps.
> 
> I have a hunch that we should not be issuing it in VMs at all...  

I removed hwclock command from my system (mv /sbin/hwclock /sbin/hwclock.old), then it's much better, but still not 0s. Delay is between 0-1 second, as you can see here:

[ondrejj@work ~]$ date; ssh guest -l root date; date
St aug 12 08:35:13 CEST 2009
St aug 12 08:35:12 CEST 2009
St aug 12 08:35:13 CEST 2009
[ondrejj@work ~]$ date; ssh guest root date; date
St aug 12 08:35:08 CEST 2009
St aug 12 08:35:08 CEST 2009
St aug 12 08:35:08 CEST 2009
[ondrejj@work ~]$ date; ssh guest -l root date; date
St aug 12 08:35:09 CEST 2009
St aug 12 08:35:09 CEST 2009
St aug 12 08:35:10 CEST 2009

(In reply to comment #6)
> If your both host/guest are f11 you probably use a kvm pv clock. Can you output
> 'cat /sys/devices/system/clocksource/clocksource0/current_clocksource'? 

[root@localhost ~]# cat /sys/devices/system/clocksource/clocksource0/current_clocksource
kvm-clock

and on my host:
[ondrejj@work ~]$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource
tsc

Is this OK?

> Full guest dmesg might be also helpful  

Will be attached in next comment.

Created attachment 357123 [details]
Full guest dmesg

Created attachment 357124 [details]
Guest configuration

May be you consider guest configuration also useful. Attached too.

Btw, by guest is an fully updated Fedora 11 (stable updates only), but host is Fedora 11 (stable) + virt-preview repository. But I think this happens with F11 stable host too.

(In reply to comment #8)
> (In reply to comment #7)
> > Please check your rc.sysinit file, and search for any invocation of the hwclock
> > command. Please try removing it, and see if it helps.
> > 
> > I have a hunch that we should not be issuing it in VMs at all...  
> 
> I removed hwclock command from my system (mv /sbin/hwclock /sbin/hwclock.old),
> then it's much better, but still not 0s. Delay is between 0-1 second

That certainly sounds a lot more acceptable

Glauber, do you know how hwclock is causing this and whether we can do anything about it?

I don't know, would have to investigate. I'll take a look at it today, and report whatever I find.

Can you tell me the exact invocation line of hwclock ?
it is probably in rc.sysinit, but you can probably find it by grepping inside /etc

Me? I don't use it. It's only used by some Fedora scripts, but not by me.

I can't ask fedora scripts, so I am asking you to provide me info about what they do on your behalf =p

What I want to find out, is what is the invocation of hwclock, and which files do it.

nevermind

I have a RHEL5 system that suffers from the same problem. Can reproduce, and I'm taking a look

It's called from udev:

[ondrejj@work ~]$ cat /lib/udev/rules.d/88-clock.rules 
ACTION=="add", SUBSYSTEM=="rtc", RUN+="/sbin/hwclock --hctosys --rtc=/dev/%k"
ACTION=="add", ENV{MAJOR}=="10", ENV{MINOR}=="135", RUN+="/sbin/hwclock --hctosys --rtc=/dev/%k"

Glauber proposes removing this rule in #517886

Okay, this should be fixed in rawhide now

See bug #489494 - we're now running hwclock --systz rather than --hctosys