Setting ITR=8.2.0 and adding Doc Text.

Tested adding the TOPOEXT feature in the command like "-cpu host,topoext". I see that CPU topology feature gets enabled in the guest.

We are trying to understand the requirements here. Look at the code and comments below. topoext is not automatically enabled for "-cpu host"

/*
  * TOPOEXT is always allowed but can't be enabled blindly by
  * "-cpu host", as it requires consistent cache topology info
  * to be provided so it doesn't confuse guests.
  */
   .no_autoenable_flags = CPUID_EXT3_TOPOEXT,

(In reply to Babu Moger from comment #6)
> We are trying to understand the requirements here. Look at the code and
> comments below. topoext is not automatically enabled for "-cpu host"
> 
> /*
>   * TOPOEXT is always allowed but can't be enabled blindly by
>   * "-cpu host", as it requires consistent cache topology info
>   * to be provided so it doesn't confuse guests.
>   */
>    .no_autoenable_flags = CPUID_EXT3_TOPOEXT,

The goal of this feature request is to implement what's necessary to remove that line safely.  Including (but not limited to) the bug that was reported at:
https://bugzilla.redhat.com/show_bug.cgi?id=1613277

(In reply to Katerina Nemcova from comment #8)
> Hi Eduardo,
> 
> I have edited the description of this issue in the Doc Text field above.
> 
> Could you please review it to confirm if it's okay?

Hi Katerina,

Some comments below:

> .SMT does not work on virtual machines

This gives the impression that SMT does not work on virtual machines at all.  
The problem is more specific: it is just about detection of SMT VCPU topology
bu the guest, only on EPYC hosts, and only if using host passthrough.

>·
> When a virtual machine (VM) boots with CPU host passthrough mode on an AMD
> EPYC host, the `TOPOEXT` CPU feature flag is not present.

This is accurate.


> Consequently, Simultaneous MultiThreading (SMT) does not work on the VM.

This is ambiguous: VCPUs will still be able to run on host CPU threads, so SMT 
is still working.  I would reword it to "The guest won't be able to detect a
virtual CPU topology with multiple threads per core".


> To work around this problem, set up the `TOPOEXT` feature flag on EPYC CPU
> manually.

This won't work, and may cause guest crashes (see bug 1619798).  The workaround
I've tried to describe at comment #2 is to use a named CPU model (EPYC) instead
of host passthrough.

(In reply to Eduardo Habkost from comment #7)
> (In reply to Babu Moger from comment #6)
> > We are trying to understand the requirements here. Look at the code and
> > comments below. topoext is not automatically enabled for "-cpu host"
> > 
> > /*
> >   * TOPOEXT is always allowed but can't be enabled blindly by
> >   * "-cpu host", as it requires consistent cache topology info
> >   * to be provided so it doesn't confuse guests.
> >   */
> >    .no_autoenable_flags = CPUID_EXT3_TOPOEXT,
> 
> The goal of this feature request is to implement what's necessary to remove
> that line safely.  Including (but not limited to) the bug that was reported
> at:
> https://bugzilla.redhat.com/show_bug.cgi?id=1613277

Spent sometime recreating the bug #1613277.
1. Checkout the stable-3.0 version from https://github.com/qemu/qemu.git. This is the time EPYC feature TOPOEXT was introduced, 

1. Commented the code below.
  //.no_autoenable_flags = CPUID_EXT3_TOPOEXT,

2. Started the guest with "-cpu host". Was expecting Kernel panic as reported in the bug #1613277.  But guest started without any issues.
   Went back as far as the kernel version 3.10.0-862.el7.x86_64 for host as well the guest. I am not able to recreate the kernel panic issue.
   Probably I am missing something here.

QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks

Hello,

Any info on this? currently hitting this bug in RHEL 8.2 running  4.18.0-193.el8. If I boot into  4.18.0-147.8.1.el8_1 , machines will run fine with the --cpu host flag. When using 4.18.0-193.el8 machines will not boot and host cpu spikes at 100%.


$ lscpu 
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              24
On-line CPU(s) list: 0-23
Thread(s) per core:  2
Core(s) per socket:  12
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               113
Model name:          AMD Ryzen 9 3900 12-Core Processor
Stepping:            0
CPU MHz:             3572.224
CPU max MHz:         3100.0000
CPU min MHz:         2200.0000
BogoMIPS:            6188.28
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-23
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd mba sev ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca

(In reply to Ivan Garcia from comment #17)
> Hello,
> 
> Any info on this? currently hitting this bug in RHEL 8.2 running 
> 4.18.0-193.el8. If I boot into  4.18.0-147.8.1.el8_1 , machines will run
> fine with the --cpu host flag. When using 4.18.0-193.el8 machines will not
> boot and host cpu spikes at 100%.

I have had problems recreating this locally. I will try RHEL 8.2 and recreate. Will debug. Thanks

Installed the redhat 8.2 from beta version of iso(RHEL-8.2.0-20191219.0-x86_64-dvd1.iso). This came with kernel version 4.18.0-167.el8.x86_64 and qemu-kvm-2.12.0-94. I did not see any issues. Here is my command line.

/usr/libexec/qemu-kvm -name rhe8 -m 16384 -smp cores=8 -hda vdisk.qcow2 -enable-kvm -net nic -net bridge,br=virbr0,helper=/usr/libexec/qemu-bridge-helper -cpu host,+topoext -nographic

/usr/libexec/qemu-kvm -name rhe8 -m 16384 -smp cores=8 -hda vdisk.qcow2 -enable-kvm -net nic -net bridge,br=virbr0,helper=/usr/libexec/qemu-bridge-helper -cpu host -nographic


Looks like Ivan saw the problem with kernel 4.18.0-193.el8.  Can someone post the link to get kernel 4.18.0-193.el8?

Thanks
Babu

My recommendation is to get  the latest 8.2 iso and update your system. It should install  4.18.0-193.el8 kernel (which is the latest) Boot into that kernel and see if you can replicate. tried myself a few mins ago and same outcome, machines will not boot and host cpu spikes at 100% (using  4.18.0-193.el8 kernel)  . When booting 4.18.0-147.8.1.el8_1 everything went back to normal.

I tried on my EPYC host, guest can boot well with kernel-4.18.0-193.el8. 

Host: (kernel-4.18.0-194.el8.x86_64)

# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  8
Socket(s):           2
NUMA node(s):        8
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               1
Model name:          AMD EPYC 7251 8-Core Processor
Stepping:            2
CPU MHz:             2869.988
CPU max MHz:         2100.0000
CPU min MHz:         1200.0000
BogoMIPS:            4191.50
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           64K
L2 cache:            512K
L3 cache:            4096K
NUMA node0 CPU(s):   0,1,16,17
NUMA node1 CPU(s):   2,3,18,19
NUMA node2 CPU(s):   4,5,20,21
NUMA node3 CPU(s):   6,7,22,23
NUMA node4 CPU(s):   8,9,24,25
NUMA node5 CPU(s):   10,11,26,27
NUMA node6 CPU(s):   12,13,28,29
NUMA node7 CPU(s):   14,15,30,31
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme ssbd sev ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca

Currently running the following kernel (works fine):

[root@test ~]# uname -r
4.18.0-147.8.1.el8_1.x86_64

set the index to 0 so we boot into the latest kernel

[root@test ~]# grubby --set-default-index=0
The default is /boot/loader/entries/220d24f78cff44a1b9d199b745f05d08-4.18.0-193.el8.x86_64.conf with index 0 and kernel /boot/vmlinuz-4.18.0-193.el8.x86_64

reboot


OHHHH actually this is very interesting....it is only happening with freebsd machines..(at least for me)  I have a pfsense vm and it does not boot and it eats 100% of the cpu (this does not happen with 4.18.0-147.8.1.el8_1.x86_64. Only happens with 4.18.0-193.el8.x86_64). The rest of the vms are fine. I only checked this vm and assumed all of them were bad :(

this is the sanitized  dump of the pfsense box




[root@test ~]# virsh dumpxml 8
<domain type='kvm' id='8'>
  <name>pfsense</name>
  <uuid>935c1c28-80a8-4b24-b5b8-c08e62159046</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://freebsd.org/freebsd/9.0"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>
    <vendor>AMD</vendor>
    <feature policy='require' name='x2apic'/>
    <feature policy='require' name='tsc-deadline'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='clwb'/>
    <feature policy='require' name='umip'/>
    <feature policy='require' name='stibp'/>
    <feature policy='require' name='arch-capabilities'/>
    <feature policy='require' name='ssbd'/>
    <feature policy='require' name='cmp_legacy'/>
    <feature policy='require' name='perfctr_core'/>
    <feature policy='require' name='wbnoinvd'/>
    <feature policy='require' name='virt-ssbd'/>
    <feature policy='require' name='rdctl-no'/>
    <feature policy='require' name='skip-l1dfl-vmentry'/>
    <feature policy='require' name='mds-no'/>
    <feature policy='disable' name='monitor'/>
    <feature policy='disable' name='svm'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>


top output


  5251 qemu      20   0 5123684 157784  20380 S 100.0   0.1   9:58.71 qemu-kvm           



lscpu

[root@test ~]# lscpu 
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              24
On-line CPU(s) list: 0-23
Thread(s) per core:  2
Core(s) per socket:  12
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               113
Model name:          AMD Ryzen 9 3900 12-Core Processor
Stepping:            0
CPU MHz:             4064.418
CPU max MHz:         3100.0000
CPU min MHz:         2200.0000
BogoMIPS:            6188.39
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-23
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd mba sev ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca

Tried to recreate again on couple of EPYC servers with latest redhat 8.2 iso(RHEL-8.2.0-20200404.0-x86_64-dvd1.iso). I am not able to recreate.

# uname -r
4.18.0-193.el8.x86_64

]# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              96
On-line CPU(s) list: 0-95
Thread(s) per core:  2
Core(s) per socket:  24
Socket(s):           2
NUMA node(s):        8
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               1
Model name:          AMD Eng Sample: 2S1308A3VCCF4_28/13_N
Stepping:            1
CPU MHz:             2239.370
CPU max MHz:         1300.0000
CPU min MHz:         800.0000
BogoMIPS:            2595.06
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           64K
L2 cache:            512K
L3 cache:            8192K
NUMA node0 CPU(s):   0-5,48-53
NUMA node1 CPU(s):   6-11,54-59
NUMA node2 CPU(s):   12-17,60-65
NUMA node3 CPU(s):   18-23,66-71
NUMA node4 CPU(s):   24-29,72-77
NUMA node5 CPU(s):   30-35,78-83
NUMA node6 CPU(s):   36-41,84-89
NUMA node7 CPU(s):   42-47,90-95
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme ssbd sev vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca

Tried following combinations. Everything seemed to work fine.

#/usr/libexec/qemu-kvm -name rhel8 -m 16384 -smp cores=8 -hda vdisk.qcow2 -enable-kvm -net nic -net bridge,br=virbr0,helper=/usr/libexec/qemu-bridge-helper -cpu EPYC -nographic

#/usr/libexec/qemu-kvm -name rhel8 -m 16384 -smp cores=8 -hda vdisk.qcow2 -enable-kvm -net nic -net bridge,br=virbr0,helper=/usr/libexec/qemu-bridge-helper -cpu host -nographic

#/usr/libexec/qemu-kvm -name rhel8 -m 16384 -smp cores=8 -hda vdisk.qcow2 -enable-kvm -net nic -net bridge,br=virbr0,helper=/usr/libexec/qemu-bridge-helper -cpu host,+topoext -nographic


Output from VM
=================
# uname -r
4.18.0-193.el8.x86_64

# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  1
Core(s) per socket:  8
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               1
Model name:          AMD Eng Sample: 2S1308A3VCCF4_28/13_N
Stepping:            1
CPU MHz:             1297.443
BogoMIPS:            2594.88
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           64K
L1i cache:           64K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid es
[root@localhost ~]#

Can you try and create a pfsense vm and see if you can replicate? iso url: https://www.pfsense.org/download/

Thanks!

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Still can reproduce this issue on EPYC host.
re-open this bug to continue track.

Test environments:
    amd-milan-02.ml3.eng.bos.redhat.com
    kernel-4.18.0-287.el8.dt3.x86_64
    qemu-kvm-4.2.0-45.module+el8.4.0+9900+483536ed.x86_64


Host info:
# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              128
On-line CPU(s) list: 0-127
Thread(s) per core:  1
Core(s) per socket:  64
Socket(s):           2
NUMA node(s):        8
Vendor ID:           AuthenticAMD
BIOS Vendor ID:      Advanced Micro Devices, Inc.
CPU family:          25
Model:               1
Model name:          AMD EPYC 7713 64-Core Processor
BIOS Model name:     AMD EPYC 7713 64-Core Processor                
Stepping:            1
CPU MHz:             2181.451
CPU max MHz:         2000.0000
CPU min MHz:         1500.0000
BogoMIPS:            3992.71
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            32768K
NUMA node0 CPU(s):   0-15
NUMA node1 CPU(s):   16-31
NUMA node2 CPU(s):   32-47
NUMA node3 CPU(s):   48-63
NUMA node4 CPU(s):   64-79
NUMA node5 CPU(s):   80-95
NUMA node6 CPU(s):   96-111
NUMA node7 CPU(s):   112-127
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate sme ssbd mba sev ibrs ibpb stibp vmmcall sev_es fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm


Best regards
Liu Nana

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Hi，


Do we have a plan to support this recently?
If so please help to re-open this, thanks.



Best regards
Liu Nana

There's no plan from Red Hat side to fix this soon.  It's up to AMD to implement this upstream.