Bug 1851227

Summary:	When doing a cpu-baseline between skylake and cascadelake, cascadelake is selected as baseline.
Product:	Red Hat Enterprise Linux 8	Reporter:	David Hill <dhill>
Component:	libvirt	Assignee:	Jiri Denemark <jdenemar>
Status:	CLOSED ERRATA	QA Contact:	Luyao Huang <lhuang>
Severity:	high	Docs Contact:
Priority:	high
Version:	8.5	CC:	chhu, cmayapka, dhill, dyuan, dzheng, fdeutsch, fgadkano, fjin, gveitmic, jdenemar, jocran, jsuchane, kchamart, lhuang, lmen, mkalinin, vasanth.mohanraj, virt-maint, xuzhang, yalzhang, ymankad
Target Milestone:	rc	Keywords:	Triaged, Upstream, ZStream
Target Release:	---	Flags:	pm-rhel: mirror+
Hardware:	x86_64
OS:	All
Whiteboard:
Fixed In Version:	libvirt-8.0.0-7.module+el8.7.0+15262+04e62783	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	2084030 2084031 2151852 (view as bug list)		Environment:
Last Closed:	2022-11-08 09:18:32 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:	8.4.0
Embargoed:
Bug Depends On:
Bug Blocks:	1897025, 2084030, 2084031, 2151852

Description David Hill 2020-06-25 19:58:03 UTC

Description of problem:
When doing a cpu-baseline between skylake and cascadelake, cascadelake is selected as baseline:

    <cpu>
      <arch>x86_64</arch>
      <model>Skylake-Server-IBRS</model>
      <vendor>Intel</vendor>
      <microcode version='33554537'/>
      <counter name='tsc' frequency='2600005000' scaling='yes'/>
      <topology sockets='1' dies='1' cores='12' threads='2'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='dca'/>
      <feature name='osxsave'/>
      <feature name='tsc_adjust'/>
      <feature name='cmt'/>
      <feature name='clflushopt'/>
      <feature name='intel-pt'/>
      <feature name='pku'/>
      <feature name='ospke'/>
      <feature name='md-clear'/>
      <feature name='stibp'/>
      <feature name='ssbd'/>
      <feature name='xsaves'/>
      <feature name='mbm_total'/>
      <feature name='mbm_local'/>
      <feature name='invtsc'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>

    <cpu>
      <arch>x86_64</arch>
      <model>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <microcode version='83886124'/>
      <counter name='tsc' frequency='2099999000' scaling='yes'/>
      <topology sockets='1' dies='1' cores='20' threads='2'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='dca'/>
      <feature name='osxsave'/>
      <feature name='tsc_adjust'/>
      <feature name='cmt'/>
      <feature name='intel-pt'/>
      <feature name='pku'/>
      <feature name='ospke'/>
      <feature name='md-clear'/>
      <feature name='stibp'/>
      <feature name='arch-capabilities'/>
      <feature name='xsaves'/>
      <feature name='mbm_total'/>
      <feature name='mbm_local'/>
      <feature name='invtsc'/>
      <feature name='rdctl-no'/>
      <feature name='ibrs-all'/>
      <feature name='skip-l1dfl-vmentry'/>
      <feature name='mds-no'/>
      <feature name='tsx-ctrl'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>


[root@undercloud-0-rhosp16 ~]# virsh cpu-baseline allo5 
<cpu mode='custom' match='exact'>
  <model fallback='allow'>Cascadelake-Server</model>
  <vendor>Intel</vendor>
  <feature policy='require' name='ds'/>
  <feature policy='require' name='acpi'/>
  <feature policy='require' name='ss'/>
  <feature policy='require' name='ht'/>
  <feature policy='require' name='tm'/>
  <feature policy='require' name='pbe'/>
  <feature policy='require' name='dtes64'/>
  <feature policy='require' name='monitor'/>
  <feature policy='require' name='ds_cpl'/>
  <feature policy='require' name='vmx'/>
  <feature policy='require' name='smx'/>
  <feature policy='require' name='est'/>
  <feature policy='require' name='tm2'/>
  <feature policy='require' name='xtpr'/>
  <feature policy='require' name='pdcm'/>
  <feature policy='require' name='dca'/>
  <feature policy='require' name='tsc_adjust'/>
  <feature policy='require' name='intel-pt'/>
  <feature policy='require' name='pku'/>
  <feature policy='require' name='md-clear'/>
  <feature policy='require' name='stibp'/>
  <feature policy='require' name='xsaves'/>
  <feature policy='require' name='invtsc'/>
  <feature policy='disable' name='avx512vnni'/>
</cpu>


Version-Release number of selected component (if applicable):
4.5, 6.3 and 6.4

How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 jiyan 2020-06-28 03:26:18 UTC

Reproduced this bug with libvirt-6.0.0-24.module+el8.2.1+6997+c666f621.x86_64.

Version:
libvirt-6.0.0-24.module+el8.2.1+6997+c666f621.x86_64
qemu-kvm-4.2.0-27.module+el8.2.1+7092+9d345e72.x86_64
kernel-4.18.0-193.8.1.el8_2.x86_64

Steps:
1: On the Skylake-Server-IBRS host 
# lscpu
...
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz
Stepping:            4
...

# virsh domcapabilities 
...
  <cpu>
    <mode name='host-passthrough' supported='yes'/>
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Skylake-Server-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='clflushopt'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='ssbd'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
    </mode>

# virsh domcapabilities >> Skylake-Server-IBRS.xml

# scp Skylake-Server-IBRS.xml root@Cascadelake-Server-Host:/root

2: On the Cascadelake-Server host
# lscpu 
Architecture:        x86_64
...
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Platinum 8260L CPU @ 2.40GHz
Stepping:            6
...

# virsh domcapabilities 
...
  <cpu>
    <mode name='host-passthrough' supported='yes'/>
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
      <feature policy='require' name='ibpb'/>
      <feature policy='require' name='amd-ssbd'/>
      <feature policy='require' name='rdctl-no'/>
      <feature policy='require' name='ibrs-all'/>
      <feature policy='require' name='skip-l1dfl-vmentry'/>
      <feature policy='require' name='mds-no'/>
      <feature policy='require' name='pschange-mc-no'/>
      <feature policy='require' name='tsx-ctrl'/>
    </mode>

# virsh domcapabilities >> Cascadelake-Server.xml

# cat Skylake-Server-IBRS.xml Cascadelake-Server.xml >> cpu-baseline.xml

# virsh hypervisor-cpu-baseline cpu-baseline.xml 
<cpu mode='custom' match='exact'>
  <model fallback='forbid'>Cascadelake-Server</model>
  <vendor>Intel</vendor>
  <feature policy='require' name='ss'/>
  <feature policy='require' name='hypervisor'/>
  <feature policy='require' name='tsc_adjust'/>
  <feature policy='require' name='umip'/>
  <feature policy='require' name='pku'/>
  <feature policy='require' name='md-clear'/>
  <feature policy='require' name='stibp'/>
  <feature policy='require' name='xsaves'/>
  <feature policy='require' name='invtsc'/>
  <feature policy='disable' name='avx512vnni'/>
</cpu>

Comment 3 Marina Kalinin 2020-11-10 21:51:01 UTC

David,

What is the impact on customer environment? I see 3 cases attached and I see they complain that live migration fails. Is it due to this bug?
IF so - shouldn't we raise the severity from low? 
Maybe you can attach a KCS clarifying the situation?

Comment 4 David Hill 2020-11-10 23:06:39 UTC

Hey Marina,

   We do have a couple of workarounds like manually selecting an architecture that would be supported by all nodes which defeats the purpose of having this baseline tool.
There's already a KCS attached to this case ... we can modify it (or maybe you can?) to suit your needs.

Thank you very much,

David Hill

Comment 9 John Ferlan 2021-09-09 15:35:23 UTC

Bulk update: Move RHEL-AV bugs to RHEL9. If necessary to resolve in RHEL8, then clone to the current RHEL8 release.

Comment 12 David Hill 2022-03-28 18:35:10 UTC

I don't understand why this BZ was changed for RHEL9 as this issue is hit on RHEL8 and we need a fix quickly.

Comment 14 Jaroslav Suchanek 2022-03-31 10:11:19 UTC

(In reply to David Hill from comment #12)
> I don't understand why this BZ was changed for RHEL9 as this issue is hit on
> RHEL8 and we need a fix quickly.

You're right, it should be tracked in rhel-8 product. It was moved in a batch and it just probably did not trigger some condition.

Comment 22 Jiri Denemark 2022-04-19 15:11:05 UTC

Yes, the result is incorrect, but the fix will not straightforward at all as
computing a baseline CPU definition is basically a heuristics. And it was not
really clear what problem it causes as the result should be usable on both
hosts anyway. Although the guest may be surprised by a strange CPU (a new CPU
model unexpectedly lacking some features it was always supposed to have).

Especially because Openstack was only using the baseline API for getting a
list of all CPU features supported on a host. But now I found
https://access.redhat.com/solutions/2891431 which I believe is the main reason
for the failure because it is incorrect. It suggests computing the baseline
CPU definition, but using only the CPU model part rather than the whole XML,
i.e., completely ignoring all the feature elements. The baseline API was never
supposed to be used like this. Only the result XML as a whole is guaranteed to
be compatible with all input hosts. So the documentation needs to be fixed in
some way.

That said, libvirt should be fixed too as Cascadelake is newer than Skylake
and thus it should not be selected as the baseline. But unfortunately there is
no total ordering between Intel CPU models and you cannot easily tell which
one is better than another one. I have some ideas for improving the
heuristics, which I will test and see if they could be a usable solution, but
fixing libvirt may still not solve the issue caused by following incorrect
documentation.

Comment 23 David Hill 2022-04-19 15:20:49 UTC

Would https://review.opendev.org/c/openstack/nova/+/762330 make sense here ?  Is that what we should be doing ?

Comment 25 Jiri Denemark 2022-04-20 07:24:05 UTC

(In reply to David Hill from comment #23)
> Would https://review.opendev.org/c/openstack/nova/+/762330 make sense here ?
> Is that what we should be doing ?

Sure, using the newer Hypervisor versions of any of the CPU APIs is always a
good idea. However, I don't think it will make any difference in this specific
case.

Comment 29 Jiri Denemark 2022-05-04 16:56:04 UTC

Patches sent upstream for review:

https://listman.redhat.com/archives/libvir-list/2022-May/230676.html

Comment 30 Jiri Denemark 2022-05-05 07:06:21 UTC

With the patches applied, cpu-baseline on the two CPU definitions from this
bug description gives:

    <cpu mode='custom' match='exact'>
      <model fallback='allow'>Skylake-Server-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ds'/>
      <feature policy='require' name='acpi'/>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='ht'/>
      <feature policy='require' name='tm'/>
      <feature policy='require' name='pbe'/>
      <feature policy='require' name='dtes64'/>
      <feature policy='require' name='monitor'/>
      <feature policy='require' name='ds_cpl'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='smx'/>
      <feature policy='require' name='est'/>
      <feature policy='require' name='tm2'/>
      <feature policy='require' name='xtpr'/>
      <feature policy='require' name='pdcm'/>
      <feature policy='require' name='dca'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='clflushopt'/>
      <feature policy='require' name='intel-pt'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='ssbd'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
    </cpu>

and hypervisor-cpu-baseline (although it provides the best results if used on
CPU definitions from domcapabilities) returns:

    <cpu mode='custom' match='exact'>
      <model fallback='forbid'>Skylake-Client</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ds'/>
      <feature policy='require' name='acpi'/>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='ht'/>
      <feature policy='require' name='tm'/>
      <feature policy='require' name='pbe'/>
      <feature policy='require' name='dtes64'/>
      <feature policy='require' name='monitor'/>
      <feature policy='require' name='ds_cpl'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='smx'/>
      <feature policy='require' name='est'/>
      <feature policy='require' name='tm2'/>
      <feature policy='require' name='xtpr'/>
      <feature policy='require' name='pdcm'/>
      <feature policy='require' name='dca'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='avx512f'/>
      <feature policy='require' name='avx512dq'/>
      <feature policy='require' name='clflushopt'/>
      <feature policy='require' name='clwb'/>
      <feature policy='require' name='intel-pt'/>
      <feature policy='require' name='avx512cd'/>
      <feature policy='require' name='avx512bw'/>
      <feature policy='require' name='avx512vl'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='spec-ctrl'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='ssbd'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='pdpe1gb'/>
      <feature policy='require' name='invtsc'/>
    </cpu>

So in this particular case even following the knowledge base article should
provide a working configuration.

Comment 31 Jiri Denemark 2022-05-06 15:38:25 UTC

Fixed upstream by

commit 48341b025acdd04a66696a709c7b09b3bfd42acf
Refs: v8.3.0-42-g48341b025a
Author:     Jiri Denemark <jdenemar>
AuthorDate: Tue Apr 26 15:06:30 2022 +0200
Commit:     Jiri Denemark <jdenemar>
CommitDate: Fri May 6 17:33:47 2022 +0200

    cpu_x86: Penalize disabled features when computing CPU model

    For finding the best matching CPU model for a given set of features
    while we don't know the CPU signature (i.e., when computing a baseline
    CPU model) we've been using a "shortest list of features" heuristics.
    This works well if new CPU models are supersets of older models, but
    that's not always the case. As a result it may actually select a new CPU
    model as a baseline while removing some features from it to make it
    compatible with older models. This is in general worse than using an old
    CPU model with a bunch of added features as a guest OS or apps may crash
    when using features that were disabled.

    On the other hand we don't want to end up with a very old model which
    would guarantee no disabled features as it could stop a guest OS or apps
    from using some features provided by the CPU because they would not
    expect them on such an old CPU.

    This patch changes the heuristics to something in between. Enabled and
    disabled features are counted separately so that a CPU model requiring
    some features to be disabled looks worse than a model with fewer
    disabled features even if its complete list of features is longer. The
    penalty given for each additional disabled feature gets bigger to make
    longer list of disabled features look even worse.

    https://bugzilla.redhat.com/show_bug.cgi?id=1851227

    Signed-off-by: Jiri Denemark <jdenemar>
    Reviewed-by: Michal Privoznik <mprivozn>

commit bb6cedd2082599323257ee0df18c93a6e0551b0b
Refs: v8.3.0-43-gbb6cedd208
Author:     Jiri Denemark <jdenemar>
AuthorDate: Fri Apr 29 10:35:02 2022 +0200
Commit:     Jiri Denemark <jdenemar>
CommitDate: Fri May 6 17:33:47 2022 +0200

    cpu_x86: Ignore enabled features for input models in x86DecodeUseCandidate

    While we don't want to aim for the shortest list of disabled features in
    the baseline result (it would select a very old model), we want to do so
    while looking at any of the input models for which we're trying to
    compute a baseline CPU model. Given a set of input models, we always
    want to take the least capable one of them (i.e., the one with shortest
    list of disabled features) or a better model which is not one of the
    input models.

    So when considering an input model, we just check whether its list of
    disabled features is shorter than the currently best one. When looking
    at other models we check both enabled and disabled features while
    penalizing disabled features as implemented by the previous patch.

    Signed-off-by: Jiri Denemark <jdenemar>
    Reviewed-by: Michal Privoznik <mprivozn>

Comment 40 Luyao Huang 2022-05-23 07:16:07 UTC

Verify this bug with libvirt-daemon-8.0.0-7.module+el8.7.0+15262+04e62783.x86_64:

1. prepare a xml file which combined by virsh capabilities output collected from skylake and cascadelake:

# cat caps.xml

    <cpu>
      <arch>x86_64</arch>
      <model>Skylake-Server-IBRS</model>
      <vendor>Intel</vendor>
      <microcode version='33554537'/>
      <counter name='tsc' frequency='2600005000' scaling='yes'/>
      <topology sockets='1' dies='1' cores='12' threads='2'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='dca'/>
      <feature name='osxsave'/>
      <feature name='tsc_adjust'/>
      <feature name='cmt'/>
      <feature name='clflushopt'/>
      <feature name='intel-pt'/>
      <feature name='pku'/>
      <feature name='ospke'/>
      <feature name='md-clear'/>
      <feature name='stibp'/>
      <feature name='ssbd'/>
      <feature name='xsaves'/>
      <feature name='mbm_total'/>
      <feature name='mbm_local'/>
      <feature name='invtsc'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>

    <cpu>
      <arch>x86_64</arch>
      <model>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <microcode version='83886124'/>
      <counter name='tsc' frequency='2099999000' scaling='yes'/>
      <topology sockets='1' dies='1' cores='20' threads='2'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='dca'/>
      <feature name='osxsave'/>
      <feature name='tsc_adjust'/>
      <feature name='cmt'/>
      <feature name='intel-pt'/>
      <feature name='pku'/>
      <feature name='ospke'/>
      <feature name='md-clear'/>
      <feature name='stibp'/>
      <feature name='arch-capabilities'/>
      <feature name='xsaves'/>
      <feature name='mbm_total'/>
      <feature name='mbm_local'/>
      <feature name='invtsc'/>
      <feature name='rdctl-no'/>
      <feature name='ibrs-all'/>
      <feature name='skip-l1dfl-vmentry'/>
      <feature name='mds-no'/>
      <feature name='tsx-ctrl'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>

2. run cpu-baseline command with this xml file and Skylake-Server-IBRS is selected as baseline
# virsh cpu-baseline caps.xml 
<cpu mode='custom' match='exact'>
  <model fallback='allow'>Skylake-Server-IBRS</model>
  <vendor>Intel</vendor>
  <feature policy='require' name='ds'/>
  <feature policy='require' name='acpi'/>
  <feature policy='require' name='ss'/>
  <feature policy='require' name='ht'/>
  <feature policy='require' name='tm'/>
  <feature policy='require' name='pbe'/>
  <feature policy='require' name='dtes64'/>
  <feature policy='require' name='monitor'/>
  <feature policy='require' name='ds_cpl'/>
  <feature policy='require' name='vmx'/>
  <feature policy='require' name='smx'/>
  <feature policy='require' name='est'/>
  <feature policy='require' name='tm2'/>
  <feature policy='require' name='xtpr'/>
  <feature policy='require' name='pdcm'/>
  <feature policy='require' name='dca'/>
  <feature policy='require' name='tsc_adjust'/>
  <feature policy='require' name='clflushopt'/>
  <feature policy='require' name='intel-pt'/>
  <feature policy='require' name='pku'/>
  <feature policy='require' name='md-clear'/>
  <feature policy='require' name='stibp'/>
  <feature policy='require' name='ssbd'/>
  <feature policy='require' name='xsaves'/>
  <feature policy='require' name='invtsc'/>
</cpu>


3. run hypervisor-cpu-baseline command with this xml file and Skylake-Server-IBRS is selected as baseline

# virsh hypervisor-cpu-baseline caps.xml
<cpu mode='custom' match='exact'>
  <model fallback='forbid'>Skylake-Client-IBRS</model>
  <vendor>Intel</vendor>
  <feature policy='require' name='ds'/>
  <feature policy='require' name='acpi'/>
  <feature policy='require' name='ss'/>
  <feature policy='require' name='ht'/>
  <feature policy='require' name='tm'/>
  <feature policy='require' name='pbe'/>
  <feature policy='require' name='dtes64'/>
  <feature policy='require' name='monitor'/>
  <feature policy='require' name='ds_cpl'/>
  <feature policy='require' name='vmx'/>
  <feature policy='require' name='smx'/>
  <feature policy='require' name='est'/>
  <feature policy='require' name='tm2'/>
  <feature policy='require' name='xtpr'/>
  <feature policy='require' name='pdcm'/>
  <feature policy='require' name='dca'/>
  <feature policy='require' name='tsc_adjust'/>
  <feature policy='require' name='avx512f'/>
  <feature policy='require' name='avx512dq'/>
  <feature policy='require' name='clflushopt'/>
  <feature policy='require' name='clwb'/>
  <feature policy='require' name='intel-pt'/>
  <feature policy='require' name='avx512cd'/>
  <feature policy='require' name='avx512bw'/>
  <feature policy='require' name='avx512vl'/>
  <feature policy='require' name='pku'/>
  <feature policy='require' name='md-clear'/>
  <feature policy='require' name='stibp'/>
  <feature policy='require' name='ssbd'/>
  <feature policy='require' name='xsaves'/>
  <feature policy='require' name='pdpe1gb'/>
  <feature policy='require' name='invtsc'/>
</cpu>


4. similarly, prepare a xml file which combined by virsh domcapabilities output collected from skylake and cascadelake, and run hypervisor-cpu-baseline with this xml file:

# cat domcaps.xml 
...
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Skylake-Client-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='pdcm'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='clflushopt'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='ssbd'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='pdpe1gb'/>
      <feature policy='require' name='invtsc'/>
      <feature policy='require' name='ibpb'/>
      <feature policy='require' name='ibrs'/>
      <feature policy='require' name='amd-stibp'/>
      <feature policy='require' name='amd-ssbd'/>
      <feature policy='require' name='rsba'/>
      <feature policy='require' name='skip-l1dfl-vmentry'/>
      <feature policy='require' name='pschange-mc-no'/>
    </mode>

...
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
      <feature policy='require' name='ibpb'/>
      <feature policy='require' name='amd-ssbd'/>
      <feature policy='require' name='rdctl-no'/>
      <feature policy='require' name='ibrs-all'/>
      <feature policy='require' name='skip-l1dfl-vmentry'/>
      <feature policy='require' name='mds-no'/>
      <feature policy='require' name='pschange-mc-no'/>
      <feature policy='require' name='tsx-ctrl'/>
    </mode>
...

# virsh hypervisor-cpu-baseline domcaps.xml 
<cpu mode='custom' match='exact'>
  <model fallback='forbid'>Skylake-Client-IBRS</model>
  <vendor>Intel</vendor>
  <feature policy='require' name='ss'/>
  <feature policy='require' name='vmx'/>
  <feature policy='require' name='hypervisor'/>
  <feature policy='require' name='tsc_adjust'/>
  <feature policy='require' name='clflushopt'/>
  <feature policy='require' name='umip'/>
  <feature policy='require' name='md-clear'/>
  <feature policy='require' name='stibp'/>
  <feature policy='require' name='arch-capabilities'/>
  <feature policy='require' name='ssbd'/>
  <feature policy='require' name='xsaves'/>
  <feature policy='require' name='pdpe1gb'/>
  <feature policy='require' name='invtsc'/>
  <feature policy='require' name='ibpb'/>
  <feature policy='require' name='amd-ssbd'/>
  <feature policy='require' name='skip-l1dfl-vmentry'/>
  <feature policy='require' name='pschange-mc-no'/>
</cpu>

# cat domcaps2.xml
...
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Skylake-Server-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='clflushopt'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='ssbd'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
    </mode>
...


...
    <mode name='host-model' supported='yes'>
      <model fallback='forbid'>Cascadelake-Server</model>
      <vendor>Intel</vendor>
      <feature policy='require' name='ss'/>
      <feature policy='require' name='vmx'/>
      <feature policy='require' name='hypervisor'/>
      <feature policy='require' name='tsc_adjust'/>
      <feature policy='require' name='umip'/>
      <feature policy='require' name='pku'/>
      <feature policy='require' name='md-clear'/>
      <feature policy='require' name='stibp'/>
      <feature policy='require' name='arch-capabilities'/>
      <feature policy='require' name='xsaves'/>
      <feature policy='require' name='invtsc'/>
      <feature policy='require' name='ibpb'/>
      <feature policy='require' name='amd-ssbd'/>
      <feature policy='require' name='rdctl-no'/>
      <feature policy='require' name='ibrs-all'/>
      <feature policy='require' name='skip-l1dfl-vmentry'/>
      <feature policy='require' name='mds-no'/>
      <feature policy='require' name='pschange-mc-no'/>
      <feature policy='require' name='tsx-ctrl'/>
    </mode>
...

# virsh hypervisor-cpu-baseline domcaps2.xml
<cpu mode='custom' match='exact'>
  <model fallback='forbid'>Skylake-Client-IBRS</model>
  <vendor>Intel</vendor>
  <feature policy='require' name='ss'/>
  <feature policy='require' name='hypervisor'/>
  <feature policy='require' name='tsc_adjust'/>
  <feature policy='require' name='avx512f'/>
  <feature policy='require' name='avx512dq'/>
  <feature policy='require' name='clflushopt'/>
  <feature policy='require' name='clwb'/>
  <feature policy='require' name='avx512cd'/>
  <feature policy='require' name='avx512bw'/>
  <feature policy='require' name='avx512vl'/>
  <feature policy='require' name='umip'/>
  <feature policy='require' name='pku'/>
  <feature policy='require' name='md-clear'/>
  <feature policy='require' name='stibp'/>
  <feature policy='require' name='ssbd'/>
  <feature policy='require' name='xsaves'/>
  <feature policy='require' name='pdpe1gb'/>
  <feature policy='require' name='invtsc'/>
</cpu>

Comment 42 errata-xmlrpc 2022-11-08 09:18:32 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: virt:rhel and virt-devel:rhel security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7472