Bug 1371619

Summary: Flags xsaveopt xsavec xgetbv1 are missing on qemu-kvm
Product: Red Hat Enterprise Linux 7 Reporter: Guo, Zhiyi <zhguo>
Component: qemu-kvmAssignee: Eduardo Habkost <ehabkost>
Status: CLOSED ERRATA QA Contact: Guo, Zhiyi <zhguo>
Severity: high Docs Contact:
Priority: high    
Version: 7.3CC: chayang, ehabkost, juzhang, knoel, lmiksik, michen, rbalakri, virt-maint, xfu, zhguo
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-1.5.3-123.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-03 20:02:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Guo, Zhiyi 2016-08-30 15:58:23 UTC
Description of problem:
Flags ht arat xsaveopt xsavec xgetbv1 are missing on qemu-kvm

Version-Release number of selected component (if applicable):
qemu-kvm:qemu-kvm-1.5.3-121.el7.x86_64
rhel 7.3 host & guest kernel:3.10.0-495.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Boot guest with cli:
/usr/libexec/qemu-kvm -name rhel7.3 -m 2048 \
        -cpu Skylake-Client,check\
        -smp 1,threads=2,cores=1,sockets=3,maxcpus=6 \
         -vga qxl\
        -spice port=3003,disable-ticketing \
        -device virtio-serial -chardev spicevmc,id=vdagent,debug=0,name=vdagent \
        -serial unix:/tmp/m,server,nowait \
        -device virtserialport,chardev=vdagent,name=com.redhat.spice.0 \
        -drive file=ss1rhel73.qcow2,if=none,id=drive-scsi-disk0,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-scsi-pci,id=scsi0 -device scsi-hd,drive=drive-scsi-disk0,bus=scsi0.0,scsi-id=0,lun=0,id=scsi-disk0,bootindex=1 \
        -monitor stdio \
        -usb -device usb-kbd,id=input0 \
        -netdev tap,id=idinWyYp,vhost=on -device virtio-net-pci,mac=42:ce:a9:d2:4d:d7,id=idlbq7eA,netdev=idinWyYp \
        -qmp tcp:localhost:4444,server,nowait \
2. get flags by cat /proc/cpuinfo inside guest
3.

Actual results:
ht arat xsaveopt xsavec xgetbv1 are missing
#cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 94
model name	: Intel Core Processor (Skylake)
stepping	: 3
microcode	: 0x1
cpu MHz		: 3600.006
cache size	: 4096 KB
physical id	: 0
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap

Expected results:
These flags exist.


Additional info:
These flags exist on qemu-kvm-rhev-2.6.0-22.el7.x86_64:
#cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 94
model name	: Intel Core Processor (Skylake)
stepping	: 3
microcode	: 0x1
cpu MHz		: 3600.006
cache size	: 4096 KB
physical id	: 0
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch arat fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap xsaveopt xsavec xgetbv1

Host flags:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 94
model name	: Intel(R) Xeon(R) CPU E3-1270 v5 @ 3.60GHz
stepping	: 3
microcode	: 0x9e
cpu MHz		: 3999.937
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 22
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch ida arat epb pln pts dtherm hwp hwp_noitfy hwp_act_window hwp_epp intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1
bogomips	: 7200.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:
....

Comment 2 Eduardo Habkost 2016-09-06 20:38:12 UTC
ARAT:

ARAT is not supported by qemu-kvm-1.5.3, and it is never included in any machine-type + CPU combination in machine-types <= rhel7.2.0 (PC_RHEL7_2_COMPAT in qemu-kvm-rhev has arat=off).


HT:

HT is missing because the following command-line option:
 -smp 1,threads=2,cores=1,sockets=3,maxcpus=6 
conflicts with the semantics of "sockets" in qemu-kvm-1.5.3. "sockets" is only useful is to help calculate the nr_threads and nr_cores value, and it is calculated based on the number of online CPUs (1), not maxcpus (6), and nr_cores and nr_threads end up set to 1.

Recommendations:
* I recommend opening a separate BZ for that bug.
* While the bug is not fixed, I recommend using just "threads=2,cores=1" and _not_ add a "sockets" argument.


XSAVEOPT, XSAVEC, XGETBV1:

This is unexpected. I can reproduce it (I see CPUID[EAX=0xD,ECX=1].EAX = 0 in my testing). I am investigating the cause.

Comment 3 Eduardo Habkost 2016-09-06 20:46:02 UTC
Found the bug, it is a regression. All XSAVE flags are being cleared because of a bug in the backport of patch "target-i386: add feature flags for CPUID[EAX=0xd, ECX=1]".

Comment 6 Miroslav Rezanina 2016-09-09 09:25:41 UTC
Fix included in qemu-kvm-1.5.3-123.el7

Comment 8 Guo, Zhiyi 2016-09-13 08:55:31 UTC
Verify against package qemu-kvm-tools-1.5.3-123.el7.x86_64:
Query flags inside guest, xsaveopt xsavec xgetbv1 find now
# cat /proc/cpuinfo 
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 94
model name	: Intel Core Processor (Skylake)
stepping	: 3
microcode	: 0x1
cpu MHz		: 3695.996
cache size	: 4096 KB
physical id	: 0
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap xsaveopt xsavec xgetbv1

Comment 9 Guo, Zhiyi 2016-09-13 09:34:45 UTC
Move to verified per comment 8, ht issue is tracked by Bug 1375507 - ht flag is missing when there is sockets option in smp

Comment 11 errata-xmlrpc 2016-11-03 20:02:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2585.html