Bug 1693433

The main thing to add here is MSR-based CPU features, so it's not just adding a CPU model.

Exactly. I already have the patches for MSR features functional. I just need
to polish the series, update tests, and deal with some QEMU interaction issues.

Verified this bug on libvirt-4.5.0-31.module+el8.1.0+3808+3325c1a3.x86_64.

Version:
libvirt-4.5.0-31.module+el8.1.0+3808+3325c1a3.x86_64
qemu-kvm-2.12.0-82.module+el8.1.0+3738+0d8c0249.x86_64
kernel-4.18.0-126.el8.x86_64

S1: Check cascadelake related info and start VM with host-model
Steps:
1. Prepare a physical host with the following cpu info
# lscpu
Model name:          Intel(R) Xeon(R) Platinum 8260L CPU @ 2.40GHz
Flags:               fpu ... avx512_vnni ...

2. Check cpu_map.xml; domcapabilities; capabilities
# cat /usr/share/libvirt/cpu_map.xml |grep Cas -A100
    <model name='Cascadelake-Server'>
      <signature family='6' model='85'/> <!-- 050654 -->
      <vendor name='Intel'/>
      <feature name='3dnowprefetch'/>
      <feature name='abm'/>
...
      <feature name='avx512f'/>
      <feature name='avx512vl'/>
      <feature name='avx512vnni'/>
...

# virsh capabilities 
<capabilities>

  <host>
    <uuid>88888888-8887-0813-0412-1f25a5a5a5a5</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <microcode version='67108892'/>
      <topology sockets='1' cores='24' threads='2'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      ...
    </cpu>

# virsh domcapabilities 
  <cpu>
    <mode name='host-passthrough' supported='yes'/>
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
    </mode>

3. Prepare a shutdown VM with the following conf; Start VM and check qemu cmd line info and dumpxml
# virsh domstate vm1
shut off

# virsh dumpxml vm1 --inactive |grep "<cpu" -A2
  <cpu mode='host-model' check='partial'>
    <model fallback='allow'/>
  </cpu>

# virsh start vm1
Domain vm1 started

# virsh dumpxml vm1 |grep "<cpu" -A20
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Cascadelake-Server</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='umip'/>
    <feature policy='require' name='pku'/>
    <feature policy='require' name='md-clear'/>
    <feature policy='require' name='stibp'/>
    <feature policy='require' name='arch-capabilities'/>
    <feature policy='require' name='xsaves'/>
    <feature policy='disable' name='mpx'/>
  </cpu>

# ps -ef |grep vm1
qemu      49456      1 33 11:24 ?        
...
-cpu Cascadelake-Server,ss=on,vmx=on,hypervisor=on,tsc_adjust=on,umip=on,pku=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on

4. Login to VM and check avx512_vnni cpu flag
# virsh console vm1
Connected to domain vm1
Escape character is ^]

Red Hat Enterprise Linux 8.1 Beta (Ootpa)
Kernel 4.18.0-107.el8.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Fri Aug  2 22:28:55 on ttyS0
[root@localhost ~]# lscpu
...
CPU family:          6
Model:               85
Model name:          Intel Xeon Processor (Cascadelake)
...
Flags:               fpu vme ... avx512_vnni ...

S2: Test other features with cpu and check cpu related cpu info
2.1 Create snapshot for VM and check cpu info in snapshot dumpxml 
# virsh snapshot-create-as vm1 s1 --disk-only --diskspec hda,file=/tmp/s1
Domain snapshot s1 created

# virsh snapshot-dumpxml vm1 s1 
...
    <cpu mode='custom' match='exact' check='partial'>
      <model fallback='forbid'>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='xsaves'/>
    </cpu>
...


2.2 Save VM and restore VM; check save-img-dumpxml file
# virsh save vm1 /tmp/save.1
Domain vm1 saved to /tmp/save.1

# virsh save-image-dumpxml /tmp/save.1
  <cpu mode='custom' match='exact' check='partial'>
    <model fallback='forbid'>Cascadelake-Server</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='umip'/>
    <feature policy='require' name='pku'/>
    <feature policy='require' name='md-clear'/>
    <feature policy='require' name='stibp'/>
    <feature policy='require' name='arch-capabilities'/>
    <feature policy='require' name='xsaves'/>
  </cpu>

# virsh restore /tmp/save.1 
Domain restored from /tmp/save.1


2.3 Managedsave VM and start VM again
# virsh managedsave vm1 
Domain vm1 state saved by libvirt

# virsh start vm1 
Domain vm1 started

# systemctl restart libvirtd

# ps -ef |grep vm1
qemu      57038      1  2 21:20 ?        
-cpu Cascadelake-Server,ss=on,vmx=on,hypervisor=on,tsc_adjust=on,umip=on,pku=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on,mpx=off


2.4 Test vcpupin and vcpu hot-plug
# virsh domstate vm1
running

# virsh dumpxml vm1 |grep vcpu
  <vcpu placement='static' current='3'>8</vcpu>

# virsh vcpuinfo vm1
VCPU:           0
CPU:            58
State:          running
CPU time:       12.3s
CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

VCPU:           1
CPU:            167
State:          running
CPU time:       3.6s
CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

VCPU:           2
CPU:            191
State:          running
CPU time:       3.8s
CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

# virsh setvcpus vm1 --count 6

# virsh vcpupin vm1 3 176

# virsh vcpupin vm1 4 0-99,^76

# virsh vcpupin vm1 5 100-101

# virsh vcpuinfo vm1
VCPU:           0
CPU:            1
State:          running
CPU time:       24.6s
CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

VCPU:           1
CPU:            48
State:          running
CPU time:       6.1s
CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

VCPU:           2
CPU:            155
State:          running
CPU time:       6.4s
CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

VCPU:           3
CPU:            176
State:          running
CPU time:       2.2s
CPU Affinity:   --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------y---------------

VCPU:           4
CPU:            55
State:          running
CPU time:       2.2s
CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy-yyyyyyyyyyyyyyyyyyyyyyy--------------------------------------------------------------------------------------------

VCPU:           5
CPU:            100
State:          running
CPU time:       2.6s
CPU Affinity:   ----------------------------------------------------------------------------------------------------yy------------------------------------------------------------------------------------------

# virsh vcpupin vm1
VCPU: CPU Affinity
----------------------------------
   0: 0-191
   1: 0-191
   2: 0-191
   3: 176
   4: 0-75,77-99
   5: 100-101
   6: 0-191
   7: 0-191

# virsh console vm1
Connected to domain vm1
Escape character is ^]

Red Hat Enterprise Linux 8.1 Beta (Ootpa)
Kernel 4.18.0-107.el8.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Fri Aug  2 23:27:11 on ttyS0
[root@localhost ~]# lscpu
...
CPU(s):              6
On-line CPU(s) list: 0-5
Model name:          Intel Xeon Processor (Cascadelake)


S3: Negative test cascadelake cpu model on physical host does not support this func
# virsh domstate vm1
shut off

# virsh dumpxml vm1 --inactive |grep "<cpu" -A3
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Cascadelake-Server</model>
    <vendor>Intel</vendor>
  </cpu>

# virsh start vm1
error: Failed to start domain vm1
error: operation failed: guest CPU doesn't match specification: extra features: hypervisor, missing features: hle,rtm,mpx,avx512f,avx512dq,rdseed,adx,smap,clflushopt,clwb,avx512cd,avx512bw,avx512vl,pku,avx512vnni,xsavec,xgetbv1,3dnowprefetch

All the test results are expected; move this bug to be verified.

*** Bug 1749522 has been marked as a duplicate of this bug. ***

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:3345