Bug 1739987 - -cpu host failed to expose topoext to guest on EPYC host
Summary: -cpu host failed to expose topoext to guest on EPYC host
Keywords:
Status: NEW
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Eduardo Habkost
QA Contact: Yumei Huang
URL:
Whiteboard:
Depends On:
Blocks: 1740002
TreeView+ depends on / blocked
 
Reported: 2019-08-12 03:32 UTC by Yumei Huang
Modified: 2020-04-21 06:02 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Cause: The TOPOEXT CPU feature flag is available only when using the EPYC CPU model. Consequence: The TOPOEXT feature isn't exposed to guests when using CPU host passthrough mode. Workaround (if any): Use the EPYC CPU model when SMT support needs to be exposed to the guest. Result: The TOPOEXT feature flag will be exposed to the guest.
Clone Of:
: 1740002 (view as bug list)
Environment:
Last Closed:
Type: Feature Request
Target Upstream Version:


Attachments (Terms of Use)

Description Yumei Huang 2019-08-12 03:32:50 UTC
Description of problem:
On EPYC host, when boot guest with "-cpu host", flag topoext is not exposed to guest, and strangely flag "arch_capabilities" is in guest.

Version-Release number of selected component (if applicable):
qemu-kvm-4.0.0-6.module+el8.1.0+3736+a2aefea3
kernel-4.18.0-128.el8.x86_64

How reproducible:
always

Steps to Reproduce:
1. Boot guest with "-cpu host"
# /usr/libexec/qemu-kvm -cpu host -smp 4,sockets=1,cores=2,threads=2 

2. Check cpu flags inside guest
# lscpu


Actual results:
QEMU print warning, 
"(qemu) qemu-kvm: warning: This family of AMD CPU doesn't support hyperthreading(2)
Please configure -smp options properly or try enabling topoext feature."

No topoext but arch_capabilities is present in guest.

# lscpu
lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               1
Model name:          AMD EPYC 7251 8-Core Processor
Stepping:            2
CPU MHz:             2096.056
BogoMIPS:            4192.11
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           64K
L1i cache:           64K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-3
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibpb vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 virt_ssbd arat arch_capabilities


Expected results:
No warning when set threads=2, and flag topoext is exposed to guest.

Additional info:

Comment 1 Yumei Huang 2019-08-12 05:19:13 UTC
Adding host cpu info.

# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  8
Socket(s):           2
NUMA node(s):        8
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               1
Model name:          AMD EPYC 7251 8-Core Processor
Stepping:            2
CPU MHz:             2324.271
CPU max MHz:         2100.0000
CPU min MHz:         1200.0000
BogoMIPS:            4191.53
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           64K
L2 cache:            512K
L3 cache:            4096K
NUMA node0 CPU(s):   0,1,16,17
NUMA node1 CPU(s):   2,3,18,19
NUMA node2 CPU(s):   4,5,20,21
NUMA node3 CPU(s):   6,7,22,23
NUMA node4 CPU(s):   8,9,24,25
NUMA node5 CPU(s):   10,11,26,27
NUMA node6 CPU(s):   12,13,28,29
NUMA node7 CPU(s):   14,15,30,31
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme ssbd sev ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca

Comment 2 Yumei Huang 2019-08-12 06:41:33 UTC
I found the comment in cpu.c.

        /*                                                                      
         * TOPOEXT is always allowed but can't be enabled blindly by                                  
         * "-cpu host", as it requires consistent cache topology info           
         * to be provided so it doesn't confuse guests.                         
         */                                                                     
        .no_autoenable_flags = CPUID_EXT3_TOPOEXT, 

So it's expected that no topoext flag in guest in this case. 

The left question is, is arch_capabilities expected to be in guest? If so, I will close this bug. Thanks.

Comment 5 Yumei Huang 2019-08-28 06:07:10 UTC
Hi Eduardo,

Could you please tell if it is expected that arch_capabilities can be exposed to guest though host doesn't have the flag? E.g. 1) Boot guest with '-cpu host' on EPYC host or Haswell host, 2)Boot guest with '-cpu ModelName,+arch-capabilities' on any host, flag arch_capabilities is in guest.

I only see the flag in Coffeelake and Cascadelake host. Does the flag 'arch-capabilities' in QEMU have nothing to do with the flag 'arch_capabilities' in host, and we can turn it on with any CPU model on any host? If so, it's not necessary to verify bugs like 1709993, only on Cascadelake host, other hosts should fit too, right ?

Comment 6 Eduardo Habkost 2019-08-28 17:54:56 UTC
(In reply to Yumei Huang from comment #5)

It is possible to emulate the arch-capabilities MSR on any Intel host (it shouldn't be possible on AMD hosts).  But in order to validate the bug you need a host that has the MSR available, so you can compare vulnerability info provided by the host with vulnerability info seen by the guest.

Comment 7 Yumei Huang 2019-09-03 05:36:36 UTC
(In reply to Eduardo Habkost from comment #6)
> (In reply to Yumei Huang from comment #5)
> 
> It is possible to emulate the arch-capabilities MSR on any Intel host (it
> shouldn't be possible on AMD hosts).  

So is it something to be fixed that '-cpu host' on amd host and '-cpu EPYC,+arch-capabilities' would expose arch_capabilities to guest ? If so, I will file a new bug, thanks!

Comment 8 Eduardo Habkost 2019-09-03 14:22:12 UTC
(In reply to Yumei Huang from comment #7)
> (In reply to Eduardo Habkost from comment #6)
> > (In reply to Yumei Huang from comment #5)
> > 
> > It is possible to emulate the arch-capabilities MSR on any Intel host (it
> > shouldn't be possible on AMD hosts).  
> 
> So is it something to be fixed that '-cpu host' on amd host and '-cpu
> EPYC,+arch-capabilities' would expose arch_capabilities to guest ? If so, I
> will file a new bug, thanks!

Being able to emulate it on AMD is not a bug, it was just a limitation of KVM, until the following upstream commit (backport included in kernel-4.18.0-97.el8).  After the commit below, emulation of MSR_IA32_ARCH_CAPABILITIES is possible on AMD CPUs.

It's important to check if guests won't crash if arch-capabilities is enabled on AMD hosts, though.  We had some bugs in the arch-capabilities CPUID and MSR code in the past.


commit 0cf9135b773bf32fba9dd8e6699c1b331ee4b749
Author: Sean Christopherson <sean.j.christopherson@intel.com>
Date:   Thu Mar 7 15:43:02 2019 -0800

    KVM: x86: Emulate MSR_IA32_ARCH_CAPABILITIES on AMD hosts
    
    The CPUID flag ARCH_CAPABILITIES is unconditioinally exposed to host
    userspace for all x86 hosts, i.e. KVM advertises ARCH_CAPABILITIES
    regardless of hardware support under the pretense that KVM fully
    emulates MSR_IA32_ARCH_CAPABILITIES.  Unfortunately, only VMX hosts
    handle accesses to MSR_IA32_ARCH_CAPABILITIES (despite KVM_GET_MSRS
    also reporting MSR_IA32_ARCH_CAPABILITIES for all hosts).
    
    Move the MSR_IA32_ARCH_CAPABILITIES handling to common x86 code so
    that it's emulated on AMD hosts.
    
    Fixes: 1eaafe91a0df4 ("kvm: x86: IA32_ARCH_CAPABILITIES is always supported")
    Cc: stable@vger.kernel.org
    Reported-by: Xiaoyao Li <xiaoyao.li@linux.intel.com>
    Cc: Jim Mattson <jmattson@google.com>
    Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Comment 9 Eduardo Habkost 2019-09-16 20:12:14 UTC
Setting FutureFeature, as "-cpu host" support for TOPOEXT isn't implemented upstream.

Comment 11 Ademar Reis 2020-02-05 23:02:36 UTC
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks


Note You need to log in before you can comment on or make changes to this bug.