Bug 1215610

Summary: Windows 7 experiencing time drift under high CPU load
Product: Red Hat Enterprise Virtualization Manager Reporter: Thorsten Scherf <tscherf>
Component: vdsmAssignee: Francesco Romani <fromani>
Status: CLOSED ERRATA QA Contact: sefi litmanovich <slitmano>
Severity: medium Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: ailan, bazulay, chayang, danken, hhuang, inetkach, juzhang, knoel, lpeer, lsurette, mavital, mazhang, mgoldboi, michal.skrivanek, michen, mkenneth, mtessun, mtosatti, pbonzini, qzhang, rbalakri, rhodain, rpacheco, scui, tscherf, virt-bugs, vrozenfe, yeylon, ykaul, yvugenfi
Target Milestone: ovirt-3.6.0-rc   
Target Release: 3.6.0   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: v4.17.2 Doc Type: Bug Fix
Doc Text:
Previously, Red Hat Enterprise Virtualization incorrectly configured the hypervisor for certain Windows versions, resulting in significant time drift on Windows virtual machines running high CPU loads. Code has been added to VDSM to inject periodic RTC interrupts, to prevent lost interrupts which caused time drift in Windows guests. The recommended hypervisor settings are now configured for Windows versions and there is no longer time drift in Windows virtual machines.
Story Points: ---
Clone Of: 1081394 Environment:
Last Closed: 2016-03-09 19:37:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
coreinfo.txt
none
process.txt
none
qemu-monitor.txt none

Comment 2 Paolo Bonzini 2015-04-27 11:33:03 UTC
Please follow the instructions from bug 1081394 comment 61 on RHEL 7.

Comment 6 Paolo Bonzini 2015-04-30 15:40:52 UTC
# /usr/libexec/qemu-kvm  --enable-kvm -cpu SandyBridge,hv_time -kernel
cpuid_dump_kernel.bin  -serial stdio

   0x40000003 0x00: eax=0x00000222 ebx=0x00000000 ecx=0x00000000 edx=0x00000000

AccessPartitionReferenceCounter, AccessHypercallMsrs, AccessPartitionReferenceTsc.

The spec says bit 6 (AccessVpIndex) is mandatory.  Do we need a patch like this?

diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index a26d25a..f62e83c 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -501,17 +501,13 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
         c = &cpuid_data.entries[cpuid_i++];
         c->function = HYPERV_CPUID_FEATURES;
-        if (cpu->hyperv_relaxed_timing) {
-            c->eax |= HV_X64_MSR_HYPERCALL_AVAILABLE;
-        }
+        c->eax = HV_X64_MSR_HYPERCALL_AVAILABLE | HV_X64_MSR_VP_INDEX_AVAILABLE;
         if (cpu->hyperv_vapic) {
-            c->eax |= HV_X64_MSR_HYPERCALL_AVAILABLE;
             c->eax |= HV_X64_MSR_APIC_ACCESS_AVAILABLE;
             has_msr_hv_vapic = true;
         }
         if (cpu->hyperv_time &&
             kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_TIME) > 0) {
-            c->eax |= HV_X64_MSR_HYPERCALL_AVAILABLE;
             c->eax |= HV_X64_MSR_TIME_REF_COUNT_AVAILABLE;
             c->eax |= 0x200;
             has_msr_hv_tsc = true;

Comment 8 Paolo Bonzini 2015-05-08 16:18:31 UTC
I'll look into it early next week.

Comment 23 Paolo Bonzini 2015-06-18 16:56:12 UTC
Thorsten, can you check if the customer can use hv_time after doing the following:

 bcdedit /set {current} useplatformclock no

Comment 27 Thorsten Scherf 2015-06-23 10:52:32 UTC
Created attachment 1042238 [details]
coreinfo.txt

Comment 28 Thorsten Scherf 2015-06-23 10:53:00 UTC
Created attachment 1042240 [details]
process.txt

Comment 29 Thorsten Scherf 2015-06-23 10:53:22 UTC
Created attachment 1042242 [details]
qemu-monitor.txt

Comment 30 Paolo Bonzini 2015-06-23 11:27:34 UTC
Ouch, the VM does not have time drift compensation active!  The -rtc-td-hack option (newer QEMUs have a more reassuring version "-global mc146818rtc.lost_tick_policy=slew") is not present.  This is inserted by libvirt when it sees this in the XML:

  <clock ...>
    <timer name='rtc' tickpolicy='catchup'/>
  </clock>

Either vdsm or RHEV-M should do that automatically for Windows guests.

Comment 35 Michal Skrivanek 2015-06-29 09:55:19 UTC
(In reply to Paolo Bonzini from comment #30)
> Ouch, the VM does not have time drift compensation active!  The -rtc-td-hack
> option (newer QEMUs have a more reassuring version "-global
> mc146818rtc.lost_tick_policy=slew") is not present.  This is inserted by
> libvirt when it sees this in the XML:
> 
>   <clock ...>
>     <timer name='rtc' tickpolicy='catchup'/>
>   </clock>
> 
> Either vdsm or RHEV-M should do that automatically for Windows guests.

Paolo, how does hypervclock and rtc co-exist? Should we use both rtc and hypervclock? Currently indeed when hyperv enlightments are enabled we do not send rtc, but only: hypervclock,pit, and explicitly disable hpet.

Comment 39 Francesco Romani 2015-07-28 13:11:02 UTC
patch merged (https://gerrit.ovirt.org/#/c/43195/)

Comment 41 Francesco Romani 2015-12-10 14:14:26 UTC
To verify, start a windows vm and make sure that the domain XML contains a snippet like

            <clock ... offset="variable">
                ...
                <timer name="hypervclock"/>
                <timer name="rtc" tickpolicy="catchup"/>
                ...

Comment 42 Francesco Romani 2015-12-10 14:19:21 UTC
(In reply to Francesco Romani from comment #41)
> To verify, start a windows vm and make sure that the domain XML contains a
> snippet like
> 
>             <clock ... offset="variable">
>                 ...
>                 <timer name="hypervclock"/>
>                 <timer name="rtc" tickpolicy="catchup"/>
>                 ...

Comment 44 sefi litmanovich 2015-12-21 16:34:37 UTC
Verified with rhevm-3.6.1.3-0.1.el6.noarch.

created a windows 7 X64 vm and started it.

in vm's xml dump the clock entity is as follows:

  <clock offset='variable' adjustment='0' basis='utc'>
    <timer name='hypervclock' present='yes'/>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>

Comment 46 errata-xmlrpc 2016-03-09 19:37:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0362.html