Bug 238015 - vgetcpu vsyscall not working
Summary: vgetcpu vsyscall not working
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel-xen
Version: rawhide
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Eduardo Habkost
QA Contact: Brian Brock
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2007-04-26 15:23 UTC by Ulrich Drepper
Modified: 2007-11-30 22:12 UTC (History)
2 users (show)

Fixed In Version: 2.6-2.6.20-2925.8.fc7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2007-06-11 19:24:06 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Fix SMP for x86_64 XenLinux (582 bytes, text/plain)
2007-04-26 22:31 UTC, Chris Wright
no flags Details
x86_64 vgetcpu vsyscall fix (1.67 KB, patch)
2007-04-26 22:34 UTC, Chris Wright
no flags Details | Diff

Description Ulrich Drepper 2007-04-26 15:23:36 UTC
Description of problem:
See

http://post-office.corp.redhat.com/archives/virtualist/2007-April/msg00289.html

and

http://post-office.corp.redhat.com/archives/virtualist/2007-April/msg00290.html


Version-Release number of selected component (if applicable):
any current xen kernel apparently.  Seems to stretch back as far as
2.6.19-1.2895 which is what I've been using.

How reproducible:
always

Steps to Reproduce:
1.get the test program from the first mail.  I don't want to attach it here,
it's too big.
2.run with zero, one, two, ... parameters, depending on how many cores the
machine has
3.
  
Actual results:
always 0

Expected results:
0, 1, 2, ... depending on parameters

Additional info:
Use strace to see what it's doing.  It's pretty simple.  The code uses
sched_setaffinity to bind to process to one processor, depending on the
parameter count.  Then it calls the vgetcpu vsyscall.  That's the code in the
vdso.  x86-64 does not have a real syscall.

Comment 1 Chris Wright 2007-04-26 21:06:51 UTC
I suspect this has never worked.  The initialization code is all marked as
#ifndef CONFIG_XEN.  So, while vgetcpu does run successfully, it's effectively
returning unitialized data.  I'm looking at what's needed to make it useful. 
Due to lack of decent NUMA info we won't get any good node data.

Comment 2 Ulrich Drepper 2007-04-26 21:17:34 UTC
Node data is not important _right now_ (will be soon, though).  The
sched_getcpu() interface in libc only exposes the CPU index.

If you cannot fix the code you should at least make sure that the vsyscall fails
and return ENOSYS.  Currently it returns zero which is very wrong.

Comment 3 Chris Wright 2007-04-26 22:29:13 UTC
Yeah, it's all wrong, I've fixed it.

Comment 4 Chris Wright 2007-04-26 22:29:36 UTC
To fix this requires two patches.  The first fixes a problem where we simply
never launch APs, so we end up with just a UP machine with all APs paused
waiting to be initialized and launched. The second fixes an issue with the
vgetcpu vsyscall.  Turns out this vsyscall was there, but none of the
initialization code for it would run because it was commented out with #ifndef
CONFIG_XEN.  When restoring
that code, we have to make sure the GDT entry is updated via hypercall because
directly writing to the GDT causes a write fault since the GDT page is write
protected in Xen.

Comment 5 Chris Wright 2007-04-26 22:31:19 UTC
Created attachment 153558 [details]
Fix SMP for x86_64 XenLinux

First patch fixes SMP support for x86_64 XenLinux.

Comment 6 Chris Wright 2007-04-26 22:34:16 UTC
Created attachment 153559 [details]
x86_64 vgetcpu vsyscall fix

Second patch fixes x86_64 vgetcpu vsyscall for XenLinux.

Comment 7 Sam Bingner 2007-05-07 19:32:39 UTC
The patches work well for me.  Corrected the issues relating to "xm vcpu-set 0"
also, I can switch between anywhere from 1 to 4 (the number of processors I
have) without any problems/crashes.

Comment 8 Eduardo Habkost 2007-05-14 19:33:58 UTC
The patches were included on Fedora Core 7, on build
kernel-xen-2.6-2.6.20-2925.8.fc7.


Note You need to log in before you can comment on or make changes to this bug.