Bug 721205

Summary: Expose RDWRGSFS new instructions to guest
Product: Red Hat Enterprise Linux 6 Reporter: Don Dugger <ddugger>
Component: kernelAssignee: Don Dugger <ddugger>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.2CC: dbayly, juzhang, knoel, rdoty, shuang, shu, yongjie.ren
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: kernel-2.6.32-182.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 13:49:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 674218, 703053    
Bug Blocks:    

Description Don Dugger 2011-07-14 03:41:02 UTC
1. Feature Overview:
 a) Name of feature: 
RDWRGSFS new instructions

 b) Feature Description: 
Exposes news instructions to read/write the FS/GS registers to the guest.
The FS/GS registers are frequently used to implement thread local storage but
current access methods limit the number of threads to 8K.  These new
instructions increase the scalability of systems by allowing >8K threads.

 2. Feature Details:
 a) Architectures:
32-bit x86
64-bit Intel EM64T/AMD64

 b) Bugzilla Dependencies:

 c) Drivers or hardware dependencies:
Intel Ivy Bridge processor based SDVs.

 d) Upstream acceptance information: 
Accepted: CS 8820392d131af4b01c90b8a09f26c939066aba4e
          CS 71fe0a8b9960559670661ad7ad23d7adfa368523
          CS 71fe0a8b9960559670661ad7ad23d7adfa368523

 e) External links:

 f) Severity (H,M,L):
High (required for Hardware Enablement)

 g) Feature Needed by:
Q3 of 2011.

 3. Business Justification:
 a) Why is this feature needed?
Significan scalability enhancement.

 b) What hardware does this enable?
General Ivy Bridge platforms.

 c) Business impact?

 d) Other business drivers:

 4. Primary contact at Red Hat, email, phone (chat)

    Don Dugger
    donald.d.dugger

 5. Primary contact at Partner, email, phone (chat)

     Keve Gabbert
     keve.a.gabbert
     Phone Number: +1 503 264 7597

Comment 2 Suqin Huang 2011-07-28 03:22:51 UTC
Hi Don,
Can you give some suggestion for the application which really use RDWRGSFS, and the cpuid of RDWRGSFS

Thanks
Suqin

Comment 3 Don Dugger 2011-07-28 03:44:05 UTC
As pointed out in the feature request this is a new feature that enhances the scalability of threading and, as such, will require changes to the threading code in the C library.  This isn't providing such library support, just enabling the feature so that when the library wants to utilize the feature it will see it.

As far as applications the would utilize it that would be any application that is highly threaded.  I'm not up on what those applications would be right now.

Also, this is not a CPUID issue, this is a bit in the CR4 register.

Comment 4 Suqin Huang 2011-07-28 09:22:38 UTC
Hi Don,
Can you provide the steps how to verify this bug

Thanks
Suqin

Comment 5 Don Dugger 2011-07-28 15:27:35 UTC
Note that this is just exposing new CPUID bits to the guest.  Verification is just to boot a kernel in the guest that knows about this capability and checking the contents of `/proc/cpuinfo' to see if the CPUID bit is set.

Comment 6 Suqin Huang 2011-07-29 02:20:16 UTC
(In reply to comment #5)
> Note that this is just exposing new CPUID bits to the guest.  Verification is
> just to boot a kernel in the guest that knows about this capability and
> checking the contents of `/proc/cpuinfo' to see if the CPUID bit is set.

Don,
Confused, according to comment3, it's not a CPUID issue.
I want to know the ECX bit for RDWRGSFS (e.g SSE4.2: 20bit, x2apic: 21bit), so I can check it with x86info if it can not be distinguished by old kernel.

Comment 7 Aristeu Rozanski 2011-08-07 16:38:50 UTC
Patch(es) available on kernel-2.6.32-182.el6

Comment 9 Suqin Huang 2011-08-09 09:13:02 UTC
Hi Don, 
this new feature is not exposed to guest

1). guest
2.6.32-183.el6.x86_64

guest flags:

flags		: fpu de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 ht syscall lm unfair_spinlock pni cx16 hypervisor lahf_lm

2). host
2.6.32-183.el6.x86_64
qemu-kvm-0.12.1.2-2.177.el6.x86_64

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 58
model name	: Genuine Intel(R) CPU  @ 1.80GHz
stepping	: 2
cpu MHz		: 1300.000
cache size	: 4096 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 2
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid

4. cmd
/usr/libexec/qemu-kvm -drive file=/root/RHEL-Server-6.1-64-virtio.qcow2,index=0,if=none,id=drive-virtio-disk1,media=disk,cache=none,format=qcow2,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk1,id=virtio-disk1 -device virtio-net-pci,netdev=idoq0tcT,mac=9a:5e:b5:68:17:d8,id=ndev00idoq0tcT,bus=pci.0,addr=0x3 -netdev tap,id=idoq0tcT,vhost=on,ifname=t0-205512-G596,script=/home/Auto/autotest/client/tests/kvm/scripts/qemu-ifup-switch,downscript=no -m 2048 -smp 2,cores=2,threads=1,sockets=1 -cpu cpu64-rhel6 -vnc :0 -rtc base=utc,clock=host,driftfix=none -M rhel6.2.0 -boot order=cdn,once=c,menu=off  -no-kvm-pit-reinjection -enable-kvm -monitor stdio -chardev socket,id=serial-rhel6,path=/tmp/serial-rhel6,server,nowait -device isa-serial,chardev=serial-rhel6

Comment 10 Don Dugger 2011-08-10 21:17:11 UTC
Suqin-

This feature is not part of the standard CPU definitions, you will only see the feature in the guest if you use the KVM option `-cpu host'.

Comment 13 Suqin Huang 2011-08-12 02:16:53 UTC
Hi Don,

The feature is not exposed to guest with -cpu host

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 58
model name	: Genuine Intel(R) CPU  @ 1.80GHz
stepping	: 2
cpu MHz		: 1795.838
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall lm constant_tsc unfair_spinlock pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt aes hypervisor lahf_lm xsaveopt
bogomips	: 3591.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual

eax in: 0x00000000, eax = 0000000d ebx = 756e6547 ecx = 6c65746e edx = 49656e69
eax in: 0x00000001, eax = 000306a2 ebx = 01020800 ecx = 82b82201 edx = 1f8bf3ff
eax in: 0x00000002, eax = 00000001 ebx = 00000000 ecx = 00000000 edx = 002c307d
eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000004, eax = 04000121 ebx = 01c0003f ecx = 0000003f edx = 00000001
eax in: 0x00000005, eax = 00000000 ebx = 00000000 ecx = 00000003 edx = 00000000
eax in: 0x00000006, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000008, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x0000000a, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x0000000b, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x0000000c, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x0000000d, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000

eax in: 0x80000000, eax = 80000008 ebx = 756e6547 ecx = 6c65746e edx = 49656e69
eax in: 0x80000001, eax = 000306a2 ebx = 00000000 ecx = 00000001 edx = 20000800
eax in: 0x80000002, eax = 20202020 ebx = 20202020 ecx = 20202020 edx = 20202020
eax in: 0x80000003, eax = 756e6547 ebx = 20656e69 ecx = 65746e49 edx = 2952286c
eax in: 0x80000004, eax = 55504320 ebx = 20402020 ecx = 30382e31 edx = 007a4847
eax in: 0x80000005, eax = 01ff01ff ebx = 01ff01ff ecx = 40020140 edx = 40020140
eax in: 0x80000006, eax = 00000000 ebx = 42004200 ecx = 02008140 edx = 00000000
eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000008, eax = 00003028 ebx = 00000000 ecx = 00000001 edx = 00000000

Comment 14 Shaolong Hu 2011-10-13 03:31:05 UTC
Guest hangs when booting on ivy host with "-cpu host":

host:2.6.32-206.el6.x86_64
guest:2.6.32-207.el6.x86_64
qemu-kvm-0.12.1.2-2.195.el6.x86_64

Since currently there is Bug 674218 for using "-cpu host" on the latest intel host, set this bug depending on 674218, will try to verify this bug when 674218 fixed, if i am wrong, please correct me.

Comment 15 Suqin Huang 2011-10-21 05:57:26 UTC
Hi Don,
1. fsgsbase is enabled in host cpu(7).ebx(0)
eax in: 0x00000007, eax = 00000000 ebx = 00000281 ecx = 00000000 edx = 00000000

2. fsgsbase is not shown in host /proc/cpuinfo

fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vp

3. fsgsbase is not exposed to guest with -cpu host
I think the flag should also be added in arch/x86/kvm/x86.c, otherwise it can not be exposed.

eax in: 0x00000007, eax = 00000000 ebx = 00000001 ecx = 00000000 edx = 00000000


2.6.32-206.el6.x86_64
qemu-kvm-0.12.1.2-2.198.el6.x86_64

Comment 16 Shaolong Hu 2011-10-21 06:02:33 UTC
Verified on:
---------------------
host:
qemu-kvm-0.12.1.2-2.198.el6.x86_64
2.6.32-206.el6.x86_64
guest:
2.6.32-207.el6.x86_64

in host:

#x86info -a
eax in: 0x00000007, eax = 00000000 ebx = 00000281 ecx = 00000000 edx = 00000000
                                                ^
                                           0 bit of ebx  

in guest:

#x86info -a
eax in: 0x00000007, eax = 00000000 ebx = 00000001 ecx = 00000000 edx = 00000000
                                                ^
                                           0 bit of ebx

Comment 17 Suqin Huang 2011-10-21 06:06:42 UTC
(In reply to comment #15)
> Hi Don,
> 1. fsgsbase is enabled in host cpu(7).ebx(0)
> eax in: 0x00000007, eax = 00000000 ebx = 00000281 ecx = 00000000 edx = 00000000
> 
> 2. fsgsbase is not shown in host /proc/cpuinfo
> 
> fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts
> acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc
> arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq
> dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt aes
> xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi
> flexpriority ept vp
> 
> 3. fsgsbase is not exposed to guest with -cpu host
> I think the flag should also be added in arch/x86/kvm/x86.c, otherwise it can
> not be exposed.
> 
> eax in: 0x00000007, eax = 00000000 ebx = 00000001 ecx = 00000000 edx = 00000000

sorry, make a mistake, fsgsbase is already exposed to guest cpu(7).ebx(0) --> ebx = 00000001

Comment 18 Suqin Huang 2011-10-21 06:19:33 UTC
Hi Don,
Can I verify this bug according to comment 16,

fsgsbase is not shown on host /proc/cpuinfo, but the cpuid cpu(7).ebx(0) is 1

eax in: 0x00000007, eax = 00000000 ebx = 00000281 ecx = 00000000 edx = 00000000

and it is exposed to guest

eax in: 0x00000007, eax = 00000000 ebx = 00000001 ecx = 00000000 edx = 00000000


Thanks
Suqin

Comment 19 Don Dugger 2011-10-21 17:06:12 UTC
Not sure what you're asking for here.  If the ebx bit one is correctly being exposed to the guest then this BZ is solved, that's what needs to happen.  The fact that `/proc/cpuinfo' is not showing this is just a missing string in the kernel, not a virtualization problem.

Comment 20 juzhang 2011-10-25 02:26:11 UTC
According to comment16,comment18 and comment19,set this issue as verified

Comment 21 Suqin Huang 2011-10-27 10:39:49 UTC
Hi yongjie,
this bug is verified, but I'd like you to test it as well. can you test it?

Thanks
Suqin

Comment 22 Yongjie Ren 2011-10-27 11:12:24 UTC
Sure. I verified this on rhel6.2 snapshot 3 . I got the same result as the comment #18.  This bug has got fixed.
related rpm packages on rhel6.2 snapshot3: 
  kernel-2.6.32-211.el6.x86_64.rpm 
  qemu-kvm-0.12.1.2-2.199.el6.x86_64.rpm  
  qemu-kvm-tools-0.12.1.2-2.199.el6.x86_64.rpm 
  libvirt-0.9.4-19.el6.x86_64.rp

Comment 23 errata-xmlrpc 2011-12-06 13:49:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2011-1530.html