Bug 1761678 - Wrongly support "Cascadelake-Server" on physical host without avx512_vnni cpu flag
Summary: Wrongly support "Cascadelake-Server" on physical host without avx512_vnni cpu...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.2
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: 8.3
Assignee: Jiri Denemark
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 1840010 2030006
TreeView+ depends on / blocked
 
Reported: 2019-10-15 05:08 UTC by jiyan
Modified: 2021-12-07 19:29 UTC (History)
11 users (show)

Fixed In Version: libvirt-6.3.0-1.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1840010 2030006 (view as bug list)
Environment:
Last Closed: 2020-11-17 17:45:34 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
81_fast_capabilities (10.26 KB, text/plain)
2019-10-15 05:11 UTC, jiyan
no flags Details
81fast_domcapabilities (5.13 KB, text/plain)
2019-10-15 05:11 UTC, jiyan
no flags Details
81slow_capabilities (10.10 KB, text/plain)
2019-10-15 05:11 UTC, jiyan
no flags Details
81slow_domcapabilities (4.84 KB, text/plain)
2019-10-15 05:12 UTC, jiyan
no flags Details
RHEL-AV-CPU-GATHER-RESULT (26.44 KB, text/plain)
2019-10-16 07:56 UTC, jiyan
no flags Details
RHEL-CPU-GATHER-RESULT (16.82 KB, text/plain)
2019-10-16 07:57 UTC, jiyan
no flags Details
rhel_slow_cpu_gather_result (15.80 KB, text/plain)
2019-10-17 07:44 UTC, jiyan
no flags Details
rhel_fast_cpu_gather_result (24.31 KB, text/plain)
2019-10-17 07:50 UTC, jiyan
no flags Details

Description jiyan 2019-10-15 05:08:28 UTC
Description of problem:
Wrongly support "Cascadelake-Server" on physical host without avx512_vnni cpu flag

Version-Release number of selected component (if applicable):
libvirt-4.5.0-35.module+el8.1.0+4227+b2722cb3.x86_64
qemu-kvm-2.12.0-88.module+el8.1.0+4233+bc44be3f.x86_64
kernel-4.18.0-147.el8.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Check physical host cpu info
# lscpu
...
Model name:          Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
...
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke md_clear flush_l1d

# lscpu |grep avx512_vnni
No output

2. Check "virsh capabilities" and "virsh domcapabilities"
# virsh capabilities
<capabilities>

  <host>
    <uuid>4c4c4544-0044-3210-8034-cac04f4e5232</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Skylake-Server-IBRS</model>
      <vendor>Intel</vendor>
      <microcode version='33554526'/>
      <topology sockets='1' cores='16' threads='2'/>
...

# virsh domcapabilities
...
  <cpu>
    <mode name='host-passthrough' supported='yes'/>
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
      <feature policy='require' name='skip-l1dfl-vmentry'/>
      <feature policy='disable' name='avx512vnni'/>
    </mode>
    <mode name='custom' supported='yes'>
...
      <model usable='yes'>Skylake-Server</model>
      <model usable='yes'>Skylake-Server-IBRS</model>
      <model usable='yes'>Skylake-Client</model>
      <model usable='yes'>Skylake-Client-IBRS</model>
...
      <model usable='no'>Cascadelake-Server</model>
...
    </mode>
  </cpu>
...

3. Start a shutdown VM with the following conf
# virsh domstate avocado-vt-vm1 
shut off

# virsh dumpxml avocado-vt-vm1 |grep "<cpu" -A2
  <cpu mode='host-model' check='partial'>
    <model fallback='allow'/>
  </cpu>

# virsh start avocado-vt-vm1
Domain avocado-vt-vm1 started

# virsh dumpxml avocado-vt-vm1 |grep "<cpu" -A20
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Cascadelake-Server</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='umip'/>
    <feature policy='require' name='pku'/>
    <feature policy='require' name='md-clear'/>
    <feature policy='require' name='stibp'/>
    <feature policy='require' name='arch-capabilities'/>
    <feature policy='require' name='xsaves'/>
    <feature policy='require' name='skip-l1dfl-vmentry'/>
    <feature policy='disable' name='avx512vnni'/>
    <feature policy='disable' name='mpx'/>
  </cpu>

# ps -ef |grep avocado-vt-vm1
-cpu Cascadelake-Server,ss=on,vmx=on,hypervisor=on,tsc-adjust=on,umip=on,pku=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on,skip-l1dfl-vmentry=on,avx512vnni=off

# virsh console avocado-vt-vm1
onnected to domain avocado-vt-vm1
Escape character is ^]

Red Hat Enterprise Linux 8.1 (Ootpa)
Kernel 4.18.0-147.el8.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Tue Oct 15 02:45:16 from 192.168.122.1
[root@localhost ~]# lscpu 
...
Model name:          Intel Xeon Processor (Cascadelake)
...
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke md_clear arch_capabilities

[root@localhost ~]# lscpu |grep avx512_vnni
No output

Actual results:
As step-2 and step-3 show.

Expected results:
Since the physical host does not support Cascadelake-Server cpu model and the "virsh domcapabilities" also shows this model is not supported. however, the default for host-mode cpu conf is Cascadelake-Server and VM can start successfully with this kind of conf although without avx512_vnni enabled.

Additional info:
1. Can not reproduce this issue with RHEL-8.1AV
Version:
qemu-kvm-4.1.0-13.module+el8.1.0+4313+ef76ec61.x86_64
libvirt-5.6.0-6.module+el8.1.0+4244+9aa4e6bb.x86_64
kernel-4.18.0-147.el8.x86_64

Steps:
# virsh capabilities 
<capabilities>

  <host>
    <uuid>4c4c4544-0044-3210-8034-cac04f4e5232</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Skylake-Server-IBRS</model>
...

# virsh domcapabilities
...
  <cpu>
    <mode name='host-passthrough' supported='yes'/>
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Skylake-Server-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='clflushopt'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='ssbd'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
      <feature policy='require' name='skip-l1dfl-vmentry'/>
    </mode>
    <mode name='custom' supported='yes'>
...
      <model usable='no'>Cascadelake-Server</model>
...
    </mode>
  </cpu>
...

2. I can reproduce this issue with the combination of the following components:
Version:
qemu-kvm-2.12.0-88.module+el8.1.0+4233+bc44be3f.x86_64
kernel-4.18.0-147.el8.x86_64
libvirt-5.6.0-6.virtcov.el8.x86_64

So I filed this bug against qemu-kvm-2.12.0-88.module+el8.1.0+4233+bc44be3f.x86_64 with RHEL-8.1 (slow train).

Comment 1 jiyan 2019-10-15 05:11:04 UTC
Created attachment 1625855 [details]
81_fast_capabilities

Comment 2 jiyan 2019-10-15 05:11:31 UTC
Created attachment 1625856 [details]
81fast_domcapabilities

Comment 3 jiyan 2019-10-15 05:11:55 UTC
Created attachment 1625857 [details]
81slow_capabilities

Comment 4 jiyan 2019-10-15 05:12:49 UTC
Created attachment 1625858 [details]
81slow_domcapabilities

Comment 7 Eduardo Habkost 2019-10-15 20:03:49 UTC
This behavior is implemented by libvirt.  Moving to the right component.

Comment 8 Eduardo Habkost 2019-10-15 20:05:53 UTC
Seems related to bug 1558558.

Comment 9 Jiri Denemark 2019-10-16 07:27:37 UTC
Right, this is related to bug 1686895 (or the one mentioned by Eduardo, but
that on is for RHEL 7.7).

The strange behavior is caused by insufficient CPU signature checks in
libvirt. We only consider family and model parts of the signuture, which is
the same for both Skylake-Server and Cascadelake-Server CPUs (family 6, model
85). The two CPUs differ only in stepping which is ignored by libvirt. I
already started working on fixing this, but fot distracted by other issues.

Comment 10 Jiri Denemark 2019-10-16 07:35:55 UTC
Could you please run the cpu-gather.sh script from libvirt's tests direcotry
(https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=tests/cputestdata/cpu-gather.sh)
on both RHEL and RHEL-AV hosts and attach the result to this bz?

Note that you need to have the cpuid tool installed first. Unfortunately it
was dropped from RHEL-8 so you may need to install it from
http://www.etallen.com/cpuid.html

Comment 11 jiyan 2019-10-16 07:56:50 UTC
Created attachment 1626325 [details]
RHEL-AV-CPU-GATHER-RESULT

Comment 12 jiyan 2019-10-16 07:57:19 UTC
Created attachment 1626326 [details]
RHEL-CPU-GATHER-RESULT

Comment 13 Jiri Denemark 2019-10-16 08:18:13 UTC
Hmm, the results show AMD EPYC 7251 8-Core Processor, which doesn't really
match the CPU in this bug description. Are you sure you used the same host for
running the script?

Comment 14 jiyan 2019-10-16 08:38:00 UTC
Oh, Sry, I made a mistake, I will borrow that machine and update the info here.

Comment 15 jiyan 2019-10-16 08:38:14 UTC
Oh, Sry, I made a mistake, I will borrow that machine and update the info here.

Comment 16 jiyan 2019-10-17 07:44:15 UTC
Created attachment 1626691 [details]
rhel_slow_cpu_gather_result

Comment 17 jiyan 2019-10-17 07:50:25 UTC
Created attachment 1626692 [details]
rhel_fast_cpu_gather_result

Comment 18 lijin 2020-03-27 01:42:57 UTC
CNV2.3 SVVP job "Verify processor virtualization support" failed due to this bug: host is Skylake but Cascadelake is assigned to vm.
For now, we do find a workaround for this job, but we still hope this issue can be fixed asap as it affect layer product.

libvirt-client-5.6.0-10.module+el8.1.1+5309+6d656f05.x86_64
qemu-kvm-4.1.0-23.module+el8.1.1+5748+5fcc84a8.1.x86_64
4.18.0-147.5.1.el8_1.x86_64

Comment 19 Jiri Denemark 2020-03-27 13:50:06 UTC
Patches sent upstream for review:

https://www.redhat.com/archives/libvir-list/2020-March/msg01197.html

Comment 20 Jiri Denemark 2020-04-14 12:23:05 UTC
This was fixed in a series of commits which ends with

commit 5d6059f8ec16d64f240dc5e6413ca55a3b46b3f7
Refs: v6.2.0-111-g5d6059f8ec
Author:     Jiri Denemark <jdenemar>
AuthorDate: Thu Mar 26 21:55:14 2020 +0100
Commit:     Jiri Denemark <jdenemar>
CommitDate: Wed Apr 8 17:52:50 2020 +0200

    cpu_map: Distinguish Cascadelake-Server from Skylake-Server

    The signatures of these two CPU model differ only in stepping as both
    report family 6 and model 85. Skylake-Server uses stepping 4 or less and
    Cascadelake-Server uses stepping 5..7.

    https://bugzilla.redhat.com/show_bug.cgi?id=1761678

    Signed-off-by: Jiri Denemark <jdenemar>
    Reviewed-by: Ján Tomko <jtomko>

Comment 23 jiyan 2020-05-26 02:35:52 UTC
Reproduced this bug with libvirt-6.0.0-17.module+el8.3.0+6423+e4cb6418.x86_64.

Version:
kernel-4.18.0-202.el8.x86_64
libvirt-6.0.0-17.module+el8.3.0+6423+e4cb6418.x86_64
qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64

Steps:
1: Check physical host cpu info
# lscpu 
...
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
...
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke md_clear flush_l1d

# lscpu |grep avx512_vnni
No output

2. Check "virsh capabilities" and "virsh domcapabilities"
# virsh capabilities
<capabilities>
  <host>
    <uuid>4c4c4544-0044-3210-8035-cac04f305332</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Skylake-Server-IBRS</model>
      <vendor>Intel</vendor>
      <microcode version='33554533'/>
      <counter name='tsc' frequency='2095077000' scaling='yes'/>
      <topology sockets='1' dies='1' cores='16' threads='2'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='dca'/>
      <feature name='osxsave'/>
      <feature name='tsc_adjust'/>
      <feature name='cmt'/>
      <feature name='clflushopt'/>
      <feature name='intel-pt'/>
      <feature name='pku'/>
      <feature name='ospke'/>
      <feature name='md-clear'/>
      <feature name='stibp'/>
      <feature name='ssbd'/>
      <feature name='xsaves'/>
      <feature name='mbm_total'/>
      <feature name='mbm_local'/>
      <feature name='invtsc'/>
...
    </cpu>

# virsh domcapabilities
<domainCapabilities>
...
  <cpu>
    <mode name='host-passthrough' supported='yes'/>
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
      <feature policy='require' name='ibpb'/>
      <feature policy='require' name='amd-ssbd'/>
      <feature policy='require' name='skip-l1dfl-vmentry'/>
      <feature policy='disable' name='avx512vnni'/>
    </mode>
...
      <model usable='yes'>Skylake-Server-IBRS</model>
      <model usable='yes'>Skylake-Server</model>
      <model usable='yes'>Skylake-Client-IBRS</model>
      <model usable='yes'>Skylake-Client</model>
...
      <model usable='no'>Cascadelake-Server</model>

3. Start a shutdown VM with the following conf
# virsh domstate avocado-vt-vm1 
shut off

# virsh dumpxml avocado-vt-vm1 | grep "<cpu" -A2
  <cpu mode='host-model' check='partial'/>

# virsh start avocado-vt-vm1
Domain avocado-vt-vm1 started

4. Check active dumpxml, qemu cmd line and guest cpu info after step-3
# virsh dumpxml avocado-vt-vm1 | grep "<cpu" -A20
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Cascadelake-Server</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='umip'/>
    <feature policy='require' name='pku'/>
    <feature policy='require' name='md-clear'/>
    <feature policy='require' name='stibp'/>
    <feature policy='require' name='arch-capabilities'/>
    <feature policy='require' name='xsaves'/>
    <feature policy='require' name='ibpb'/>
    <feature policy='require' name='amd-ssbd'/>
    <feature policy='require' name='skip-l1dfl-vmentry'/>
    <feature policy='disable' name='avx512vnni'/>
    <feature policy='disable' name='mpx'/>
  </cpu>

# ps -ef | grep avocado-vt-vm1
-cpu Cascadelake-Server,ss=on,vmx=on,hypervisor=on,tsc-adjust=on,umip=on,pku=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on,ibpb=on,amd-ssbd=on,skip-l1dfl-vmentry=on,avx512vnni=off 

# virsh console avocado-vt-vm1
Connected to domain avocado-vt-vm1
Escape character is ^]

Red Hat Enterprise Linux 8.2 Beta (Ootpa)
Kernel 4.18.0-187.el8.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Mon May 25 22:58:14 from 192.168.122.1
[root@localhost ~]# lscpu 
...
CPU family:          6
Model:               85
Model name:          Intel Xeon Processor (Cascadelake)
...
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke md_clear arch_capabilities

[root@localhost ~]# lscpu |grep avx512_vnni
No output

Comment 24 jiyan 2020-05-26 02:42:07 UTC
Verified this bug with libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64.

Version:
kernel-4.18.0-202.el8.x86_64
libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64
qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64

Steps:
1: Check physical host cpu info
# lscpu 
...
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
...
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke md_clear flush_l1d

# lscpu |grep avx512_vnni
No output

2. Check "virsh capabilities" and "virsh domcapabilities"
# virsh capabilities
<capabilities>
  <host>
    <uuid>4c4c4544-0044-3210-8035-cac04f305332</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Skylake-Server-IBRS</model>
      <vendor>Intel</vendor>
      <microcode version='33554533'/>
      <counter name='tsc' frequency='2095077000' scaling='yes'/>
      <topology sockets='1' dies='1' cores='16' threads='2'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='dca'/>
      <feature name='osxsave'/>
      <feature name='tsc_adjust'/>
      <feature name='cmt'/>
      <feature name='clflushopt'/>
      <feature name='intel-pt'/>
      <feature name='pku'/>
      <feature name='ospke'/>
      <feature name='md-clear'/>
      <feature name='stibp'/>
      <feature name='ssbd'/>
      <feature name='xsaves'/>
      <feature name='mbm_total'/>
      <feature name='mbm_local'/>
      <feature name='invtsc'/>
...
    </cpu>

# virsh domcapabilities
<domainCapabilities>
...
  <cpu>
    <mode name='host-passthrough' supported='yes'/>
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Skylake-Server-IBRS</model>   *********
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='clflushopt'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='ssbd'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
      <feature policy='require' name='ibpb'/>
      <feature policy='require' name='amd-ssbd'/>
      <feature policy='require' name='skip-l1dfl-vmentry'/>
    </mode>
...
      <model usable='yes'>Skylake-Server-noTSX-IBRS</model>
      <model usable='yes'>Skylake-Server-IBRS</model>
      <model usable='yes'>Skylake-Server</model>
      <model usable='yes'>Skylake-Client-noTSX-IBRS</model>
      <model usable='yes'>Skylake-Client-IBRS</model>
      <model usable='yes'>Skylake-Client</model>
...
      <model usable='no'>Cascadelake-Server-noTSX</model>
      <model usable='no'>Cascadelake-Server</model>

3. Start a shutdown VM with the following conf
# virsh domstate avocado-vt-vm1 
shut off

# virsh dumpxml avocado-vt-vm1 | grep "<cpu" -A2
  <cpu mode='host-model' check='partial'/>

# virsh start avocado-vt-vm1
Domain avocado-vt-vm1 started

4. Check active dumpxml, qemu cmd line and guest cpu info after step-3
# virsh dumpxml avocado-vt-vm1 | grep "<cpu" -A20
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Skylake-Server-IBRS</model>   *********
    <vendor>Intel</vendor>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='clflushopt'/>
    <feature policy='require' name='umip'/>
    <feature policy='require' name='pku'/>
    <feature policy='require' name='md-clear'/>
    <feature policy='require' name='stibp'/>
    <feature policy='require' name='arch-capabilities'/>
    <feature policy='require' name='ssbd'/>
    <feature policy='require' name='xsaves'/>
    <feature policy='require' name='ibpb'/>
    <feature policy='require' name='amd-ssbd'/>
    <feature policy='require' name='skip-l1dfl-vmentry'/>
    <feature policy='disable' name='mpx'/>
  </cpu>

# ps -ef | grep avocado-vt-vm1
-cpu Skylake-Server-IBRS,ss=on,vmx=on,hypervisor=on,tsc-adjust=on,clflushopt=on,umip=on,pku=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaves=on,ibpb=on,amd-ssbd=on,skip-l1dfl-vmentry=on    *********

# virsh console avocado-vt-vm1
Connected to domain avocado-vt-vm1
Escape character is ^] (Ctrl + ])

Red Hat Enterprise Linux 8.2 Beta (Ootpa)
Kernel 4.18.0-187.el8.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Tue May 26 10:34:35 on ttyS0
[root@localhost ~]# lscpu 
...
CPU family:          6
Model:               85
Model name:          Intel Xeon Processor (Skylake, IBRS)    *********
...
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke md_clear arch_capabilities

All the test results are as expected, move this bug to be verified.

Comment 28 errata-xmlrpc 2020-11-17 17:45:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5137


Note You need to log in before you can comment on or make changes to this bug.