Bug 679207

Summary:	implement TSC based vread
Product:	Red Hat Enterprise Linux 7	Reporter:	Zachary Amsden <zamsden>
Component:	kernel	Assignee:	Marcelo Tosatti <mtosatti>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	7.0	CC:	fyang, juzhang, knoel, michen, mtosatti, pbonzini, tburke, virt-maint, xfu
Target Milestone:	rc	Keywords:	FutureFeature, Reopened
Target Release:	---
Hardware:	x86_64
OS:	Unspecified
Whiteboard:
Fixed In Version:	kernel 3.8	Doc Type:	Release Note
Doc Text:	KVM Clock Get Time Performance In Red Hat Enterprise Linux 7.0 Beta the vsyscall mechanism was enhanced to support fast reads of the clock from the user space for KVM guests. A guest virtual machine running Red Hat Enterprise Linux 7.0 Beta on a Red Hat Enterprise Linux 7.0 Beta host will see improved performance for applications that read the time of day frequently.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-06-13 10:23:31 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	461036, 580954, 767187, 850946, 1034029

Description Zachary Amsden 2011-02-21 21:57:25 UTC

Description of problem:

Should have a vread implementation using kvmclock when it is stable and available.  This needs to be implemented and tested upstream.  May also require libc changes to be effective.

This should be doable on machines with stable TSC now that we have fancy TSC offset-matching upstream (and KVM clock offset-matching soon to be there).

Comment 2 Zachary Amsden 2011-02-21 22:21:11 UTC

Not proposed for 6.1, this would be for 6.2.  Too late to add untested kernel changes this late in the game.

Comment 3 Zachary Amsden 2011-06-07 00:51:32 UTC

TSC scaling has derailed this bug quite a bit by making it substantially more complex to implement; still it may be possible for 6.2 if I can clear the other, more serious issues first.  Requesting pm ack

Comment 4 Zachary Amsden 2011-06-22 01:14:29 UTC

This turned out to be a lot more work than anticipated... it's going to require RDTSCP to do this properly

Comment 5 Marcelo Tosatti 2012-01-26 19:52:44 UTC

For the record, comparison of native vs guest gettimeofday
performance:

model name	: Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz

native, gettimeofday (vsyscall): 45ns
guest, kvmclock (syscall): 198ns

Comment 6 Dor Laor 2012-05-22 07:12:16 UTC

> Andrew Theurer wrote:
>> Wondering if a user-space gettimofday() for kvm-clock has been
>> considered before.  I am seeing a pretty large difference in
>> performance between tsc and kvm-clock.  I have to assume at least
>> some of this is due to the mode switch for kvm-clock.  Here are the
>> results:
>>
>> (this is a 16 vCPU VM on a 16 thread 2S Nehalem-EP host, looped
>> gettimeofday() calls on all vCPUs)
>>
>> tsc:        .0645 usec per call
>> kvm-clock:     .4222 usec per call (6.54x)
>>
>>
>> -Andrew Theurer
>
> https://bugzilla.redhat.com/show_bug.cgi?id=679207
>
> "model name : Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz
>
> native, gettimeofday (vsyscall): 45ns
> guest, kvmclock (syscall): 198ns"
>
> But this was before
>
> commit 489fb490dbf8dab0249ad82b56688ae3842a79e8
> Author: Glauber Costa<glommer>
> Date:   Tue May 11 12:17:40 2010 -0400
>
>      x86, paravirt: Add a global synchronization point for pvclock
>
> (see the full changelog for details).
>
> Can you try disabling the global variable, to see if that makes
> a difference (should not be enabled in production)? Untested patch
> (against guest kernel) below

The following was re-done on a 3.4 guest kernel (previously RHEL kernel):

1-way:
  tsc:        .0315
  kvm-clock:    .2112 (6.7x)

16-way:
  tsc:        .0432
  kvm-clock:     .4825 (11.1x)

Now with global var disabled:

16-way:
  kvm-clock:    .4628

Comment 7 Marcelo Tosatti 2012-05-23 03:56:44 UTC

It is not possible to implement vsyscall support for kvmclock because it would be necessary to write to the "global point" variable in kernel space introduced by commit "x86, paravirt: Add a global synchronization point for pvclock" (from userspace).

It is not possible to remove that global variable due to a fundamental characteristic of kvmclock tsc-delta based updates.

There is no guarantee that the TSC is synchronized across CPUs on large SMP systems, no matter how reliable and stable individual TSCs are. This means that
kvmclock system_time+tsc_timestamp updates must be performed eventually, which in
turn requires the global synchronization point to exist.

Comment 11 Marcelo Tosatti 2013-03-15 01:50:54 UTC

*** Bug 919899 has been marked as a duplicate of this bug. ***

Comment 12 FuXiangChun 2014-03-03 05:23:51 UTC

Marcelo,
QE duble-checked this bug, but still have not idea how to verify this bug. would please tell QE how to verify it ?  Thanks!

Comment 14 juzhang 2014-03-19 03:31:51 UTC

According to comment13, I plan to set this issue as verified.

Please add your comment if we missed some testing.

Best Regards,
Junyi

Comment 16 Ludek Smid 2014-06-13 10:23:31 UTC

This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.