Description of problem: 5 more -noTSX x86 CPU models are newly introduced since libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64, including Skylake-Server-noTSX-IBRS, Skylake-Client-noTSX-IBRS, Icelake-Server-noTSX, Icelake-Client-noTSX, Cascadelake-Server-noTSX Version-Release number of selected component (if applicable): kernel-4.18.0-202.el8.x86_64 qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64 libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64 How reproducible: 100% Steps to Reproduce: 1. Tested the following steps with libvirt-6.0.0-17.module+el8.3.0+6423+e4cb6418.x86_64. # ls /usr/share/libvirt/cpu_map/ > 60017-libvirt # virsh domcapabilities > 60017-libvirt-domcap 2. Tested the following steps with libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64 # ls /usr/share/libvirt/cpu_map/ > 6301-libvirt # virsh domcapabilities > 6301-libvirt-domcap 3. Diff the files in step-1 and step-2 # diff 60017-libvirt 6301-libvirt 15a16 > x86_Cascadelake-Server-noTSX.xml 29a31 > x86_Icelake-Client-noTSX.xml 30a33 > x86_Icelake-Server-noTSX.xml 54a58 > x86_Skylake-Client-noTSX-IBRS.xml 56a61 > x86_Skylake-Server-noTSX-IBRS.xml # diff 60017-libvirt-domcap 6301-libvirt-domcap 60a62 > <model usable='yes'>Skylake-Server-noTSX-IBRS</model> 62a65 > <model usable='yes'>Skylake-Client-noTSX-IBRS</model> 76a80 > <model usable='no'>Icelake-Server-noTSX</model> 77a82 > <model usable='no'>Icelake-Client-noTSX</model> 86a92 > <model usable='no'>Cascadelake-Server-noTSX</model> Actual results: Showed in the steps above, Skylake-Server-noTSX-IBRS, Skylake-Client-noTSX-IBRS, Icelake-Server-noTSX, Icelake-Client-noTSX, Cascadelake-Server-noTSX, 5 -noTSX cpu models are newly introduced. Expected results: Additional info: 1. There is a commit info from upstream QEMU. commit dd17a4eb Author: Christian Ehrhardt <christian.ehrhardt> Date: Tue Mar 10 11:48:06 2020 +0100 cpu_map: Add more -noTSX x86 CPU models One of the mitigation methods for TAA[1] is to disable TSX support on the host system. Linux added a mechanism to disable TSX globally through the kernel command line, and many Linux distributions now default to tsx=off. This makes existing CPU models that have HLE and RTM enabled not usable anymore. Add new versions of all CPU models that have the HLE and RTM features enabled, that can be used when TSX is disabled in the host system. On systems disabling the features without those types defined in cpu-maps users end up without modern CPU types in the list of usable CPUs to use in the likes of virsh domcapabilities or tools higher in the stack like virt-manager. This adds: -Cascadelake-Server-noTSX -Icelake-Client-noTSX -Icelake-Server-noTSX -Skylake-Server-noTSX-IBRS -Skylake-Client-noTSX-IBRS Introduced in QEMU by commit v4.2.0-rc2-3-g9ab2237f19 (function) and commit v4.2.0-rc2-4-g02fa60d101 (names) 2. I filed this bugs cause this is a new function introduced in libvirt, which needs some patch info and testing.
Also found the new changes about decode element in cpu_map.xml as follows on libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64. # cat /usr/share/libvirt/cpu_map/* | grep decode <decode host='on' guest='on'/> ... <decode host='on' guest='off'/> ... Which showed in the upstream patch: cpu_map: Add <decode> element to x86 CPU model definitions commit 17cdefe5 Author: Jiri Denemark <jdenemar> Date: Tue Mar 17 22:32:16 2020 +0100 cpu_map: Don't use new noTSX models for host-model CPUs Host-model CPU definitions (and domain capabilities) will use the original CPU models (without noTSX in their name) and explicitly disable hle and rtm features. This way domains with host-model CPUs will be migratable even to older versions of libvirt which do not support the noTSX model variants. The new models will be advertised in host capabilities and they may be used explicitly with custom CPUs. Signed-off-by: Jiri Denemark <jdenemar> Reviewed-by: Christian Ehrhardt <christian.ehrhardt> Tested-by: Christian Ehrhardt <christian.ehrhardt> commit 7cd896ef Author: Jiri Denemark <jdenemar> Date: Tue Mar 17 22:27:49 2020 +0100 cpu_x86: Honor CPU models' <decode> element Signed-off-by: Jiri Denemark <jdenemar> Reviewed-by: Christian Ehrhardt <christian.ehrhardt> Tested-by: Christian Ehrhardt <christian.ehrhardt>
Hi Jiri, I have tested on a Cascadelake-Server physical host with hle but not rtm, the result is as below. And I have a question about step 3 result below: why it shows "<feature policy='require' name='amd-stibp'/> <feature policy='require' name='amd-ssbd'/>" on such Intel host? Please help to confirm, Thank you! 1. Check the hardware info of the host: # rpm -q libvirt-libs qemu-kvm kernel libvirt-libs-6.6.0-4.module+el8.3.0+7883+3d717aa8.x86_64 qemu-kvm-5.1.0-4.module+el8.3.0+7846+ae9b566f.x86_64 Kernel-4.18.0-234.el8.x86_64 # lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 40 On-line CPU(s) list: 0-39 Thread(s) per core: 2 Core(s) per socket: 10 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz Stepping: 7 CPU MHz: 1000.832 BogoMIPS: 4400.00 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 1024K L3 cache: 14080K NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke avx512_vnni md_clear flush_l1d arch_capabilities # lscpu | grep rtm # (no outputs) # lscpu | grep hle ... tsc_adjust bmi1 hle avx2… 2. Check virsh capabilities recognized it as "Cascadelake-Server-noTSX"; # virsh capabilities <capabilities> <host> <uuid>4c4c4544-004a-5410-8035-b4c04f593233</uuid> <cpu> <arch>x86_64</arch> <model>Cascadelake-Server-noTSX</model> ***** <vendor>Intel</vendor> <microcode version='83898113'/> <counter name='tsc' frequency='2194843000' scaling='yes'/> <topology sockets='1' dies='1' cores='10' threads='2'/> <feature name='ds'/> <feature name='acpi'/> <feature name='ss'/> <feature name='ht'/> <feature name='tm'/> <feature name='pbe'/> <feature name='dtes64'/> <feature name='monitor'/> <feature name='ds_cpl'/> <feature name='vmx'/> <feature name='smx'/> <feature name='est'/> <feature name='tm2'/> <feature name='xtpr'/> <feature name='pdcm'/> <feature name='dca'/> <feature name='osxsave'/> <feature name='tsc_adjust'/> <feature name='cmt'/> <feature name='intel-pt'/> <feature name='pku'/> <feature name='ospke'/> <feature name='md-clear'/> <feature name='stibp'/> <feature name='arch-capabilities'/> <feature name='xsaves'/> <feature name='mbm_total'/> <feature name='mbm_local'/> <feature name='invtsc'/> <feature name='rdctl-no'/> <feature name='ibrs-all'/> <feature name='skip-l1dfl-vmentry'/> <feature name='mds-no'/> <feature name='tsx-ctrl'/> <pages unit='KiB' size='4'/> <pages unit='KiB' size='2048'/> <pages unit='KiB' size='1048576'/> </cpu> 3. Check virsh domcapabilities shows it as "Cascadelake-Server" with hle and rtm disabled which is as expected; and model usable for "Cascadelake-Server-noTSX" is yes, model usable for "Cascadelake-Server" is no. The result is as expected. # virsh domcapabilities ... <mode name='host-model' supported='yes'> <model fallback='forbid'>Cascadelake-Server</model> <vendor>Intel</vendor> <feature policy='require' name='ss'/> <feature policy='require' name='vmx'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='umip'/> <feature policy='require' name='pku'/> <feature policy='require' name='md-clear'/> <feature policy='require' name='stibp'/> <feature policy='require' name='arch-capabilities'/> <feature policy='require' name='xsaves'/> <feature policy='require' name='invtsc'/> <feature policy='require' name='ibpb'/> *** <feature policy='require' name='amd-stibp'/> *** *** <feature policy='require' name='amd-ssbd'/> *** <feature policy='require' name='rdctl-no'/> <feature policy='require' name='ibrs-all'/> <feature policy='require' name='skip-l1dfl-vmentry'/> <feature policy='require' name='mds-no'/> <feature policy='require' name='pschange-mc-no'/> <feature policy='disable' name='hle'/> *** <feature policy='disable' name='rtm'/> *** </mode> … <mode name='custom' supported='yes'> …... <model usable='yes'>Conroe</model> <model usable='yes'>Cascadelake-Server-noTSX</model> *** <model usable='no'>Cascadelake-Server</model> *** <model usable='yes'>Broadwell-noTSX-IBRS</model> <model usable='yes'>Broadwell-noTSX</model> <model usable='no'>Broadwell-IBRS</model> <model usable='no'>Broadwell</model> <model usable='yes'>486</model> </mode>
Test on Skylake-Client-IBRS host with hle and rtm flags, the result is as expected. Start a vm with host-passthrough cpu model with hle and rtm disabled, and login the vm: # lscpu | grep hle # (no outputs) # lscpu | grep rtm # (no outputs) # rpm -q libvirt-libs qemu-kvm libvirt-libs-6.6.0-4.module+el8.3.0+7883+3d717aa8.x86_64 qemu-kvm-5.1.0-4.module+el8.3.0+7846+ae9b566f.x86_64 # virsh capabilities <capabilities> <host> <uuid>f1da8f96-56d7-4d28-a3ef-703bfe97fa54</uuid> <cpu> <arch>x86_64</arch> <model>Skylake-Client-noTSX-IBRS</model> … </cpu> # virsh domcapabilities … <mode name='host-model' supported='yes'> <model fallback='forbid'>Skylake-Client-IBRS</model> <vendor>Intel</vendor> <feature policy='require' name='ss'/> ... <feature policy='disable' name='hle'/> <feature policy='disable' name='rtm'/> <feature policy='disable' name='mpx'/> </mode> <mode name='custom' supported='yes'> <model usable='no'>Skylake-Server-noTSX-IBRS</model> <model usable='no'>Skylake-Server-IBRS</model> <model usable='no'>Skylake-Server</model> <model usable='yes'>Skylake-Client-noTSX-IBRS</model> <model usable='no'>Skylake-Client-IBRS</model> ....
(In reply to yalzhang from comment #4) > # virsh domcapabilities > ... > <mode name='host-model' supported='yes'> > <model fallback='forbid'>Cascadelake-Server</model> > <vendor>Intel</vendor> > <feature policy='require' name='ss'/> > <feature policy='require' name='vmx'/> > <feature policy='require' name='hypervisor'/> > <feature policy='require' name='tsc_adjust'/> > <feature policy='require' name='umip'/> > <feature policy='require' name='pku'/> > <feature policy='require' name='md-clear'/> > <feature policy='require' name='stibp'/> > <feature policy='require' name='arch-capabilities'/> > <feature policy='require' name='xsaves'/> > <feature policy='require' name='invtsc'/> > <feature policy='require' name='ibpb'/> > *** <feature policy='require' name='amd-stibp'/> *** > *** <feature policy='require' name='amd-ssbd'/> *** > <feature policy='require' name='rdctl-no'/> > <feature policy='require' name='ibrs-all'/> > <feature policy='require' name='skip-l1dfl-vmentry'/> > <feature policy='require' name='mds-no'/> > <feature policy='require' name='pschange-mc-no'/> > <feature policy='disable' name='hle'/> *** > <feature policy='disable' name='rtm'/> *** > </mode> Features that show as enabled in domain capabilities even though they are not available in the host capabilities are enabled and/or emulated by QEMU/KVM regardless of what the host supports.
Test on Skylake-Server-IBRS host: 1. Start L1 guest with cpu setting as below: # virsh dumpxml rhel --inactive | grep /cpu -B3 <cpu mode='host-passthrough' check='none' migratable='on'> <feature policy='disable' name='hle'/> <feature policy='disable' name='rtm'/> </cpu> 2.login L1 guest and install the latest virt module, then check the cpu related info: # lscpu | grep hle # (no outputs) # lscpu | grep rtm # (no outputs) # virsh capabilities <capabilities> <host> <uuid>75b068b8-2482-43f9-88e8-17b0f33e9b83</uuid> <cpu> <arch>x86_64</arch> <model>Skylake-Server-noTSX-IBRS</model> ... # virsh domcapabilities <domainCapabilities> <path>/usr/libexec/qemu-kvm</path> <domain>kvm</domain> ... <mode name='host-model' supported='yes'> <model fallback='forbid'>Skylake-Server-IBRS</model> <vendor>Intel</vendor> ... <feature policy='disable' name='hle'/> <feature policy='disable' name='rtm'/> <feature policy='disable' name='mpx'/> </mode> <mode name='custom' supported='yes'> <model usable='yes'>Skylake-Server-noTSX-IBRS</model> <model usable='no'>Skylake-Server-IBRS</model> <model usable='no'>Skylake-Server</model> <model usable='yes'>Skylake-Client-noTSX-IBRS</model> <model usable='no'>Skylake-Client-IBRS</model> <model usable='no'>Skylake-Client</model> ... 3. start a L2 guest with host-model, guest can start successfully with " <cpu mode='host-model' check='partial'/>"; 4. start L2 guest with custom 'Skylake-Server-noTSX-IBRS': # virsh dumpxml rh | grep /cpu -B3 <cpu mode='custom' match='exact' check='partial'> <model fallback='forbid'>Skylake-Server-noTSX-IBRS</model> <vendor>Intel</vendor> </cpu> # virsh start rh Domain rh started The result is as expected.
Hi Jiri, I have loan a icelake host, but the output of "virsh capabilities" show it is 'Westmere-IBRS' with several features. And 'virsh domcapabilities' show it is Icelake-Server with several features disabled. I have checked the cpu flags, and found many flags like 'avx' does not exists. I have below questions, could you please help to have a check? Thank you! 1) How can libvirt recognize the cpu module in "virsh capabilities" and "virsh domcapabilities"? 2) Some flags do not exits, is it disabled by kernel? 3) Do you think this is expected? # rpm -q libvirt-libs qemu-kvm libvirt-libs-6.6.0-6.module+el8.3.0+8125+aefcf088.x86_64 qemu-kvm-5.1.0-10.module+el8.3.0+8254+568ca30d.x86_64 1. # virsh capabilities <capabilities> <host> <uuid>a5a5a5a5-a5a5-0d14-080b-0708a5a5a5a5</uuid> <cpu> <arch>x86_64</arch> ****** <model>Westmere-IBRS</model> ******** <vendor>Intel</vendor> <microcode version='2600534023'/> <counter name='tsc' frequency='2200000000' scaling='yes'/> <topology sockets='1' dies='1' cores='24' threads='1'/> <feature name='vme'/> <feature name='ds'/> <feature name='acpi'/> <feature name='ss'/> <feature name='ht'/> <feature name='tm'/> <feature name='pbe'/> <feature name='pclmuldq'/> <feature name='dtes64'/> <feature name='monitor'/> <feature name='ds_cpl'/> <feature name='vmx'/> <feature name='smx'/> <feature name='est'/> <feature name='tm2'/> <feature name='xtpr'/> <feature name='pdcm'/> <feature name='x2apic'/> <feature name='movbe'/> <feature name='tsc-deadline'/> <feature name='xsave'/> <feature name='osxsave'/> <feature name='rdrand'/> <feature name='arat'/> <feature name='fsgsbase'/> <feature name='tsc_adjust'/> <feature name='smep'/> <feature name='erms'/> <feature name='cmt'/> <feature name='rdseed'/> <feature name='smap'/> <feature name='clflushopt'/> <feature name='clwb'/> <feature name='intel-pt'/> <feature name='sha-ni'/> <feature name='umip'/> <feature name='waitpkg'/> <feature name='gfni'/> <feature name='rdpid'/> <feature name='cldemote'/> <feature name='movdiri'/> <feature name='movdir64b'/> <feature name='md-clear'/> <feature name='stibp'/> <feature name='arch-capabilities'/> <feature name='ssbd'/> <feature name='xsaveopt'/> <feature name='xsavec'/> <feature name='xgetbv1'/> <feature name='xsaves'/> <feature name='mbm_total'/> <feature name='mbm_local'/> <feature name='rdtscp'/> <feature name='3dnowprefetch'/> <feature name='invtsc'/> <feature name='rdctl-no'/> <feature name='ibrs-all'/> <feature name='skip-l1dfl-vmentry'/> <feature name='mds-no'/> <feature name='pschange-mc-no'/> <feature name='taa-no'/> <pages unit='KiB' size='4'/> <pages unit='KiB' size='2048'/> </cpu> ... 2. # virsh domcapabilities <domainCapabilities> <path>/usr/libexec/qemu-kvm</path> <domain>kvm</domain> <machine>pc-i440fx-rhel7.6.0</machine> <arch>x86_64</arch> <vcpu max='240'/> <iothreads supported='yes'/> <os supported='yes'> <enum name='firmware'/> <loader supported='yes'> <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value> <enum name='type'> <value>rom</value> <value>pflash</value> </enum> <enum name='readonly'> <value>yes</value> <value>no</value> </enum> <enum name='secure'> <value>no</value> </enum> </loader> </os> <cpu> <mode name='host-passthrough' supported='yes'> <enum name='hostPassthroughMigratable'> <value>on</value> <value>off</value> </enum> </mode> <mode name='host-model' supported='yes'> ****** <model fallback='forbid'>Icelake-Server</model> ****** <vendor>Intel</vendor> <feature policy='require' name='ss'/> <feature policy='require' name='vmx'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='sha-ni'/> <feature policy='require' name='waitpkg'/> <feature policy='require' name='rdpid'/> <feature policy='require' name='cldemote'/> <feature policy='require' name='movdiri'/> <feature policy='require' name='movdir64b'/> <feature policy='require' name='md-clear'/> <feature policy='require' name='stibp'/> <feature policy='require' name='arch-capabilities'/> <feature policy='require' name='xsaves'/> <feature policy='require' name='invtsc'/> <feature policy='require' name='ibpb'/> <feature policy='require' name='amd-stibp'/> <feature policy='require' name='amd-ssbd'/> <feature policy='require' name='rdctl-no'/> <feature policy='require' name='ibrs-all'/> <feature policy='require' name='skip-l1dfl-vmentry'/> <feature policy='require' name='mds-no'/> <feature policy='require' name='pschange-mc-no'/> <feature policy='disable' name='fma'/> <feature policy='disable' name='pcid'/> <feature policy='disable' name='avx'/> <feature policy='disable' name='f16c'/> <feature policy='disable' name='bmi1'/> <feature policy='disable' name='hle'/> <feature policy='disable' name='avx2'/> <feature policy='disable' name='bmi2'/> <feature policy='disable' name='invpcid'/> <feature policy='disable' name='rtm'/> <feature policy='disable' name='mpx'/> <feature policy='disable' name='avx512f'/> <feature policy='disable' name='avx512dq'/> <feature policy='disable' name='adx'/> <feature policy='disable' name='intel-pt'/> <feature policy='disable' name='avx512cd'/> <feature policy='disable' name='avx512bw'/> <feature policy='disable' name='avx512vl'/> <feature policy='disable' name='avx512vbmi'/> <feature policy='disable' name='pku'/> <feature policy='disable' name='avx512vbmi2'/> <feature policy='disable' name='vaes'/> <feature policy='disable' name='vpclmulqdq'/> <feature policy='disable' name='avx512vnni'/> <feature policy='disable' name='avx512bitalg'/> <feature policy='disable' name='avx512-vpopcntdq'/> <feature policy='disable' name='la57'/> <feature policy='disable' name='pdpe1gb'/> <feature policy='disable' name='abm'/> <feature policy='disable' name='wbnoinvd'/> </mode> <mode name='custom' supported='yes'> <model usable='yes'>qemu64</model> <model usable='yes'>qemu32</model> <model usable='no'>phenom</model> <model usable='yes'>pentium3</model> <model usable='yes'>pentium2</model> <model usable='yes'>pentium</model> <model usable='yes'>n270</model> <model usable='yes'>kvm64</model> <model usable='yes'>kvm32</model> <model usable='yes'>coreduo</model> <model usable='yes'>core2duo</model> <model usable='no'>athlon</model> <model usable='yes'>Westmere-IBRS</model> <model usable='yes'>Westmere</model> <model usable='no'>Skylake-Server-noTSX-IBRS</model> <model usable='no'>Skylake-Server-IBRS</model> <model usable='no'>Skylake-Server</model> <model usable='no'>Skylake-Client-noTSX-IBRS</model> <model usable='no'>Skylake-Client-IBRS</model> <model usable='no'>Skylake-Client</model> <model usable='no'>SandyBridge-IBRS</model> <model usable='no'>SandyBridge</model> <model usable='yes'>Penryn</model> <model usable='no'>Opteron_G5</model> <model usable='no'>Opteron_G4</model> <model usable='no'>Opteron_G3</model> <model usable='yes'>Opteron_G2</model> <model usable='yes'>Opteron_G1</model> <model usable='yes'>Nehalem-IBRS</model> <model usable='yes'>Nehalem</model> <model usable='no'>IvyBridge-IBRS</model> <model usable='no'>IvyBridge</model> **** <model usable='no'>Icelake-Server-noTSX</model> ***** should be yes?? <model usable='no'>Icelake-Server</model> **** <model usable='no'>Icelake-Client-noTSX</model> **** should be yes?? <model usable='no'>Icelake-Client</model> <model usable='no'>Haswell-noTSX-IBRS</model> <model usable='no'>Haswell-noTSX</model> <model usable='no'>Haswell-IBRS</model> <model usable='no'>Haswell</model> <model usable='no'>EPYC-IBPB</model> <model usable='no'>EPYC</model> <model usable='no'>Dhyana</model> <model usable='no'>Cooperlake</model> <model usable='yes'>Conroe</model> <model usable='no'>Cascadelake-Server-noTSX</model> <model usable='no'>Cascadelake-Server</model> <model usable='no'>Broadwell-noTSX-IBRS</model> <model usable='no'>Broadwell-noTSX</model> <model usable='no'>Broadwell-IBRS</model> <model usable='no'>Broadwell</model> <model usable='yes'>486</model> </mode> </cpu> <devices> <disk supported='yes'> <enum name='diskDevice'> <value>disk</value> <value>cdrom</value> <value>floppy</value> <value>lun</value> </enum> <enum name='bus'> <value>ide</value> <value>fdc</value> <value>scsi</value> <value>virtio</value> <value>usb</value> <value>sata</value> </enum> <enum name='model'> <value>virtio</value> <value>virtio-transitional</value> <value>virtio-non-transitional</value> </enum> </disk> <graphics supported='yes'> <enum name='type'> <value>sdl</value> <value>vnc</value> <value>spice</value> </enum> </graphics> <video supported='yes'> <enum name='modelType'> <value>vga</value> <value>cirrus</value> <value>qxl</value> <value>virtio</value> <value>none</value> <value>bochs</value> <value>ramfb</value> </enum> </video> <hostdev supported='yes'> <enum name='mode'> <value>subsystem</value> </enum> <enum name='startupPolicy'> <value>default</value> <value>mandatory</value> <value>requisite</value> <value>optional</value> </enum> <enum name='subsysType'> <value>usb</value> <value>pci</value> <value>scsi</value> </enum> <enum name='capsType'/> <enum name='pciBackend'/> </hostdev> <rng supported='yes'> <enum name='model'> <value>virtio</value> <value>virtio-transitional</value> <value>virtio-non-transitional</value> </enum> <enum name='backendModel'> <value>random</value> <value>egd</value> <value>builtin</value> </enum> </rng> </devices> <features> <gic supported='no'/> <vmcoreinfo supported='yes'/> <genid supported='yes'/> <backingStoreInput supported='yes'/> <backup supported='no'/> <sev supported='no'/> </features> </domainCapabilities> 3. Check host cpu: # lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 24 On-line CPU(s) list: 0-23 Thread(s) per core: 1 Core(s) per socket: 24 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 134 Model name: Intel Atom(R) P5362 processor Stepping: 5 CPU MHz: 1039.772 CPU max MHz: 2200.0000 CPU min MHz: 800.0000 BogoMIPS: 4400.00 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 4608K L3 cache: 15360K NUMA node0 CPU(s): 0-23 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg cx16 xtpr pdcm sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave rdrand lahf_lm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 cdp_l3 cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust smep erms cqm rdt_a rdseed smap clflushopt clwb intel_pt sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect dtherm arat pln pts umip waitpkg gfni rdpid cldemote movdiri movdir64b md_clear flush_l1d arch_capabilities # lscpu | grep avx -->no outputs 4. Start vm with cpu type is "host-model", start the vm, and check the live xml, the cpu type is Icelake-Server with hle and rtm disabled, which is expected. # virsh dumpxml rhel ... <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Icelake-Server</model> <vendor>Intel</vendor> <feature policy='require' name='ss'/> <feature policy='require' name='vmx'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='sha-ni'/> <feature policy='require' name='waitpkg'/> <feature policy='require' name='rdpid'/> <feature policy='require' name='cldemote'/> <feature policy='require' name='movdiri'/> <feature policy='require' name='movdir64b'/> <feature policy='require' name='md-clear'/> <feature policy='require' name='stibp'/> <feature policy='require' name='arch-capabilities'/> <feature policy='require' name='xsaves'/> <feature policy='require' name='ibpb'/> <feature policy='require' name='amd-stibp'/> <feature policy='require' name='amd-ssbd'/> <feature policy='require' name='rdctl-no'/> <feature policy='require' name='ibrs-all'/> <feature policy='require' name='skip-l1dfl-vmentry'/> <feature policy='require' name='mds-no'/> <feature policy='require' name='pschange-mc-no'/> <feature policy='disable' name='fma'/> <feature policy='disable' name='pcid'/> <feature policy='disable' name='avx'/> <feature policy='disable' name='f16c'/> <feature policy='disable' name='bmi1'/> <feature policy='disable' name='hle'/> <feature policy='disable' name='avx2'/> <feature policy='disable' name='bmi2'/> <feature policy='disable' name='invpcid'/> <feature policy='disable' name='rtm'/> <feature policy='disable' name='mpx'/> <feature policy='disable' name='avx512f'/> <feature policy='disable' name='avx512dq'/> <feature policy='disable' name='adx'/> <feature policy='disable' name='intel-pt'/> <feature policy='disable' name='avx512cd'/> <feature policy='disable' name='avx512bw'/> <feature policy='disable' name='avx512vl'/> <feature policy='disable' name='avx512vbmi'/> <feature policy='disable' name='pku'/> <feature policy='disable' name='avx512vbmi2'/> <feature policy='disable' name='vaes'/> <feature policy='disable' name='vpclmulqdq'/> <feature policy='disable' name='avx512vnni'/> <feature policy='disable' name='avx512bitalg'/> <feature policy='disable' name='avx512-vpopcntdq'/> <feature policy='disable' name='la57'/> <feature policy='disable' name='pdpe1gb'/> <feature policy='disable' name='abm'/> <feature policy='disable' name='wbnoinvd'/> </cpu> ...
(In reply to yalzhang from comment #8) > 1) How can libvirt recognize the cpu module in "virsh capabilities" and > "virsh domcapabilities"? Libvirt tries to find a model with matching signature (family, model, stepping numbers) if possible and uses a "shortest additional features list" heuristics if a model with matching signature is not present in our CPU map. But we cannot disable features present in the selected model for virsh capabilities, which means libvirt has to show a CPU model that is fully supported by the host (no features have to be disabled) plus a list of additional features on top of it. > 2) Some flags do not exits, is it disabled by kernel? Unlikely, the CPU just does not support them. > 3) Do you think this is expected? > 2. > # virsh domcapabilities > <domainCapabilities> > <cpu> > <mode name='host-passthrough' supported='yes'> > <enum name='hostPassthroughMigratable'> > <value>on</value> > <value>off</value> > </enum> > </mode> > <mode name='host-model' supported='yes'> > ****** <model fallback='forbid'>Icelake-Server</model> ****** > <vendor>Intel</vendor> ... > <feature policy='disable' name='fma'/> > <feature policy='disable' name='pcid'/> > <feature policy='disable' name='avx'/> > <feature policy='disable' name='f16c'/> > <feature policy='disable' name='bmi1'/> > <feature policy='disable' name='hle'/> > <feature policy='disable' name='avx2'/> > <feature policy='disable' name='bmi2'/> > <feature policy='disable' name='invpcid'/> > <feature policy='disable' name='rtm'/> > <feature policy='disable' name='mpx'/> > <feature policy='disable' name='avx512f'/> > <feature policy='disable' name='avx512dq'/> > <feature policy='disable' name='adx'/> > <feature policy='disable' name='intel-pt'/> > <feature policy='disable' name='avx512cd'/> > <feature policy='disable' name='avx512bw'/> > <feature policy='disable' name='avx512vl'/> > <feature policy='disable' name='avx512vbmi'/> > <feature policy='disable' name='pku'/> > <feature policy='disable' name='avx512vbmi2'/> > <feature policy='disable' name='vaes'/> > <feature policy='disable' name='vpclmulqdq'/> > <feature policy='disable' name='avx512vnni'/> > <feature policy='disable' name='avx512bitalg'/> > <feature policy='disable' name='avx512-vpopcntdq'/> > <feature policy='disable' name='la57'/> > <feature policy='disable' name='pdpe1gb'/> > <feature policy='disable' name='abm'/> > <feature policy='disable' name='wbnoinvd'/> Wow, that's a lot of disabled features. I think the CPU is not in fact Icelake-Server. > **** <model usable='no'>Icelake-Server-noTSX</model> ***** should be > yes?? > <model usable='no'>Icelake-Server</model> > **** <model usable='no'>Icelake-Client-noTSX</model> **** should be > yes?? No, usable='yes' would mean you can just create a domain with Icelake-Server guest CPU and expect all its features to be enabled in the guest. Which is definitely not going to happen on this host. > 3. Check host cpu: > # lscpu > Architecture: x86_64 > CPU op-mode(s): 32-bit, 64-bit > Byte Order: Little Endian > CPU(s): 24 > On-line CPU(s) list: 0-23 > Thread(s) per core: 1 > Core(s) per socket: 24 > Socket(s): 1 > NUMA node(s): 1 > Vendor ID: GenuineIntel > CPU family: 6 > Model: 134 > Model name: Intel Atom(R) P5362 processor > Stepping: 5 Oh, it's Atom. The CPU signature (family 6, model 134, stepping 5) actually matches the one of Icelake-Server CPUs in our CPU map, but it doesn't really look like Icelake-Server. I wonder what it is, ark.intel.com does not show this CPU anywhere. The closest (by name) are Intel Atom P59* which are SnowRidge CPUs. Most likely the signatures for Icelake CPU models are incorrect or at least incomplete in our CPU map. Mainly because only a few CPUs of this architecture were actually released. The second issue is we're missing SnowRidge CPU model, which would seem to be the right match for this host. That said, the results are expected with the current state of libvirt's CPU map.
on ice-lake cpu which lack of "mpx" and "intel_pt" 1. on host: # virsh capabilities <capabilities> <host> <uuid>88888888-8887-1814-080b-152ba5a5a5a5</uuid> <cpu> <arch>x86_64</arch> <model>Broadwell-IBRS</model> <vendor>Intel</vendor> <microcode version='2332033632'/> <counter name='tsc' frequency='2700000000' scaling='yes'/> <topology sockets='1' dies='1' cores='24' threads='2'/> <feature name='vme'/> <feature name='ds'/> <feature name='acpi'/> <feature name='ss'/> <feature name='ht'/> <feature name='tm'/> <feature name='pbe'/> <feature name='dtes64'/> <feature name='monitor'/> <feature name='ds_cpl'/> <feature name='vmx'/> <feature name='smx'/> <feature name='est'/> <feature name='tm2'/> <feature name='xtpr'/> <feature name='pdcm'/> <feature name='dca'/> <feature name='osxsave'/> <feature name='f16c'/> <feature name='rdrand'/> <feature name='arat'/> <feature name='tsc_adjust'/> <feature name='cmt'/> <feature name='avx512f'/> <feature name='avx512dq'/> <feature name='avx512ifma'/> <feature name='clflushopt'/> <feature name='clwb'/> <feature name='intel-pt'/> <feature name='avx512cd'/> <feature name='sha-ni'/> <feature name='avx512bw'/> <feature name='avx512vl'/> <feature name='avx512vbmi'/> <feature name='umip'/> <feature name='pku'/> <feature name='ospke'/> <feature name='avx512vbmi2'/> <feature name='gfni'/> <feature name='vaes'/> <feature name='vpclmulqdq'/> <feature name='avx512vnni'/> <feature name='avx512bitalg'/> <feature name='avx512-vpopcntdq'/> <feature name='la57'/> <feature name='rdpid'/> <feature name='md-clear'/> <feature name='pconfig'/> <feature name='stibp'/> <feature name='arch-capabilities'/> <feature name='ssbd'/> <feature name='xsaveopt'/> <feature name='xsavec'/> <feature name='xgetbv1'/> <feature name='xsaves'/> <feature name='mbm_total'/> <feature name='mbm_local'/> <feature name='pdpe1gb'/> <feature name='abm'/> <feature name='invtsc'/> <feature name='wbnoinvd'/> <feature name='rdctl-no'/> <feature name='ibrs-all'/> <feature name='skip-l1dfl-vmentry'/> <feature name='mds-no'/> <feature name='pschange-mc-no'/> <pages unit='KiB' size='4'/> <pages unit='KiB' size='2048'/> <pages unit='KiB' size='1048576'/> </cpu> ... # virsh domcapabilities ... <mode name='host-model' supported='yes'> <model fallback='forbid'>Icelake-Server</model> <vendor>Intel</vendor> <feature policy='require' name='ss'/> <feature policy='require' name='vmx'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='avx512ifma'/> <feature policy='require' name='sha-ni'/> <feature policy='require' name='rdpid'/> <feature policy='require' name='md-clear'/> <feature policy='require' name='stibp'/> <feature policy='require' name='arch-capabilities'/> <feature policy='require' name='xsaves'/> <feature policy='require' name='invtsc'/> <feature policy='require' name='ibpb'/> <feature policy='require' name='amd-stibp'/> <feature policy='require' name='amd-ssbd'/> <feature policy='require' name='rdctl-no'/> <feature policy='require' name='ibrs-all'/> <feature policy='require' name='skip-l1dfl-vmentry'/> <feature policy='require' name='mds-no'/> <feature policy='require' name='pschange-mc-no'/> <feature policy='disable' name='mpx'/> <feature policy='disable' name='intel-pt'/> </mode> ... 2. start a vm with cpu <cpu mode='host-passthrough' check='partial' migratable='on'> <feature policy='disable' name='hle'/> <feature policy='disable' name='rtm'/> </cpu> Login the guest, and check # lscpu | grep hle # lscpu | grep rtm # (no outputs) # virsh domcapabilities … <mode name='host-model' supported='yes'> <model fallback='forbid'>Icelake-Server</model> <vendor>Intel</vendor> <feature policy='require' name='ss'/> <feature policy='require' name='vmx'/> … <feature policy='disable' name='hle'/> <feature policy='disable' name='rtm'/> <feature policy='disable' name='mpx'/> <feature policy='disable' name='intel-pt'/> </mode> … <mode name='custom' supported='yes'> <model usable='yes'>Icelake-Server-noTSX</model> <model usable='no'>Icelake-Server</model> <model usable='yes'>Icelake-Client-noTSX</model> <model usable='no'>Icelake-Client</model> … </mode> </cpu> the result is as expected.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5137