1582667 – RFE - Ability to add IBPB feature policy to mitigate Spectre v2 for VM guests

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1582667 - RFE - Ability to add IBPB feature policy to mitigate Spectre v2 for VM guests

Summary: RFE - Ability to add IBPB feature policy to mitigate Spectre v2 for VM guests

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	virt-manager
Sub Component:
Version:	7.5
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Pavel Hrdina
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2018-05-25 21:19 UTC by amashah
Modified:	2023-03-24 14:05 UTC (History)
CC List:	13 users (show)
Fixed In Version:	virt-manager-1.5.0-6.el7
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2019-08-06 13:08:01 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
debug log for secure=no (13.41 KB, text/plain) 2019-05-20 09:13 UTC, zhoujunqin	no flags	Details
View All

Links
System	ID	Priority	Status	Summary	Last Updated
Red Hat Bugzilla	1582582	unspecified	CLOSED	No IBPB CPU type for Opteron processors to mitigate Spectre v2 for VM	2023-03-24 14:05:50 UTC
Red Hat Bugzilla	1582613	high	CLOSED	[RHV] No IBPB CPU type for Opteron processors to mitigate Spectre v2 for VM	2023-09-18 00:13:42 UTC
Red Hat Product Errata	RHBA-2019:2232	None	None	None	2019-08-06 13:08:13 UTC

Internal Links: 1582582 1582613

Description amashah 2018-05-25 21:19:40 UTC

Description of problem:

There is no way to add feature policy to CPU type, in order for Opteron CPU's to mitigate Spectre vulnerabilities, the following feature policy must be enabled for Opteron CPU's. [1]

   <cpu mode='custom' match='exact'>
      <model fallback='forbid'>Opteron_G5</model>
      <feature policy='require' name='ibpb'/>
    </cpu>

Version-Release number of selected component (if applicable):


How reproducible:

100% 

Steps to Reproduce:
1. Create a new VM on a RHEL host using an Opteron G4/5 CPU via virt-manager
2. Ensure vendor provided patches for CVE-2017-5715 are applied to the host
3. Test guest VM which is not patched for Spectre v2

Actual results:

Guest VM is vulnerable to Spectre v2

Expected results:

VM should not be vulnerable. Add ability to specify this in virt-manager or inherit the proper mitigation if the host microcode has been updated. Also provide a method to make changes to existing VM's so they do not need to be edited with virsh.


Additional info:


[1] https://bugzilla.redhat.com/show_bug.cgi?id=1582582#c3

Comment 4 Pavel Hrdina 2018-05-31 10:50:13 UTC

So as it was stated in BZ 1582582 there is no need for -IBPB model.

Selecting specific features was removed from virt-manager GUI because it is considered as advanced usage.  In general virt-manager is supposed to be used
in workstation environment, it is not an enterprise virtualization solution.

To fix existing domains you can use virt-xml:

    virt-xml --edit --cpu require=ibpb $domainName

I'm closing this as wontfix.  Possible workaround for new guests is to use virt-install where you can configure cpu features.

Comment 5 Daniel Berrangé 2018-06-01 10:10:54 UTC

Re-opening this, because if virt-manager is going to support selection of named CPU models, it *MUST* have some mechanism to get a number of extra CPU flags enabled to fix critical hardware security flaws.  Wishing to run a guest without security flaws can't be considered "advanced usage".

Now this doesn't necessarily mean virt-manager needs to expose UI for selecting individual CPU flags, but if it is not going to expose such UI, then virt-manager needs to be able to automatically turn on security related CPU flags such as ibpb, and the new spec-ctrl / virt-spec-ctrl flags without user invovlement.

Comment 6 Pavel Hrdina 2018-06-01 13:59:39 UTC

So the whole CPU configuration is super complex and it also depends on the
combination of libvirt and QEMU versions.

Since libvirt-3.2.0 we can guarantee that requested features will be
correctly exposed to the guest by setting check='full' and requested feature
with policy='require'.

With older libvirt we can just do the best effort by setting feature
policy='force'

One possible solution would be to expose some UI which would be used to
enable/disable security features (ideally from some external database)
and would fail to start the guest if the features are enabled and not
available.  Possible with third option "enable it with 'force/optional'
policy without any guarantees".  With libvirt < 3.2.0 we would not allow
to enforce the security features since we cannot guarantee it.

This all applies to selecting specific CPU model with mode=custom.

In case of using mode=host-model the story is slightly different.  It's safe
to use it only with libvirt >= 3.2.0 and QEMU >= 2.9.0, otherwise we should
copy the CPU definition from domain capabilities.  UI to enforce security
features would be used for this mode as well.

Because of this virt-manager would automatically enable security features
only using the best effort way which would improve the situation but the
guest still might be affected by security flaws.  That was the main motivation
to close it as WANTFIX.  If we decide to add some UI to enable/disable/enforce
security features we need some way how to get all known security features.

Comment 15 zhoujunqin 2018-09-07 01:48:49 UTC

Hi Pavel,
It seems there is no new version of virt-manager on rhel7.6, for i see a lot of bugs have been moved to rhel-7.7.0, so how about this bug, please have a look, thanks.

Comment 22 Daniel Berrangé 2019-05-14 19:14:54 UTC

Putting back to ASSIGNED so we can add md-clear https://www.redhat.com/archives/virt-tools-list/2019-May/msg00020.html

Comment 23 Daniel Berrangé 2019-05-14 19:18:38 UTC

Opps, the last link should have been this one that was actually pushed https://www.redhat.com/archives/virt-tools-list/2019-May/msg00019.html

Comment 25 zhoujunqin 2019-05-20 09:10:36 UTC

Verify this bug with new build:
virt-manager-1.5.0-5.el7.noarch
virt-install-1.5.0-5.el7.noarch
libvirt-4.5.0-17.el7.x86_64
qemu-kvm-rhev-2.12.0-29.el7.x86_64

Steps:
1. Prepare a host which support ibpb(there is IBRS or IBPB keyword in cpu model):  

# virsh capabilities
<capabilities>

  <host>
    <uuid>4c4c4544-0033-3810-804c-b7c04f315332</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>EPYC-IBPB</model>
      <vendor>AMD</vendor>
      <microcode version='134222375'/>
      <topology sockets='1' cores='4' threads='2'/>
      <feature name='ht'/>
      <feature name='osxsave'/>
      <feature name='xsaves'/>
      <feature name='cmp_legacy'/>
      <feature name='extapic'/>
      <feature name='skinit'/>
      <feature name='wdt'/>
      <feature name='tce'/>
      <feature name='topoext'/>
      <feature name='perfctr_core'/>
      <feature name='perfctr_nb'/>
      <feature name='invtsc'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>
...

Scenario-1 Testing on virt-manager UI

1. Launch virt-manager
# virt-manager

2. Set 'Edit'->'Preferences'->'New VM defaults'->select 'CPU default' as 'Copy host CPU definition', then click 'Close'.

3. Install a vm.
3.1 Select the local connection: QEMU/KVM
3.2 Click NEW button at the top of Virtual Machine Manager dialogue.
3.3 Select Network Install(HTTP,FTP or NFS)", and Click "Forward" button.
3.4 Enter URL in "URL" field, such as "http://download-node-02.eng.bos.redhat.com/rhel-7/rel-eng/RHEL-7//RHEL-7.7-20190514.2/compose/Server/x86_64/os/"
3.5 Check "Automatically detect operating system based on install media", then click 'Forward'.
3.6 Keep following pages as default setting, and click 'Forward' step by step until click "Finish" button.
Result: Installation finishes with no error.

4. Check "new checkbox to control CPU security features" works. Double click a VM, and turn to "Show virtual hardware details" page.
Result: 
4.1 Cpu Model shows "EYPC-IBPB".
4.2 New checkbox "Enable available CPU security flaw mitigations" is added and it is checked by default.
# virsh dumpxml rhel7.7
...
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>
    <vendor>AMD</vendor>
    <feature policy='require' name='x2apic'/>
    <feature policy='require' name='tsc-deadline'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='cmp_legacy'/>
    <feature policy='require' name='virt-ssbd'/>
    <feature policy='disable' name='monitor'/>
    <feature policy='disable' name='svm'/>
    <feature policy='require' name='topoext'/>
  </cpu>


4.2.1 Login guest and check guest cpu feature have ibpb
# cat /proc/cpuinfo

fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm art rep_good nopl xtopology extd_apicid eagerfpu pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext retpoline_amd ssbd ibpb vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 virt_ssbd arat

Result: CPU security features(virt_ssbd, ibpb, ssbd) are added.

4.3 Uncheck "Enable available CPU security flaw mitigations" and restart vm again.
# virsh dumpxml rhel7.7
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='x2apic'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='disable' name='svm'/>
    <feature policy='require' name='topoext'/>
  </cpu>

4.3.1 Login guest and check guest cpu feature don't have security CPU security features.
# cat /proc/cpuinfo


As a summary, it works well on virt-manager UI.

Scenario-2 Testing with virt-install cml.
1. Enable CPU secure parameter
# virt-install --name cpu --memory 1024 --disk /var/lib/libvirt/images/rhel7.7.qcow2  --import --cpu EPYC-IBPB,secure=yes --noautoconsole --debug
...
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>
    <feature policy='require' name='ibpb'/>
    <feature policy='require' name='virt-ssbd'/>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='x2apic'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='disable' name='svm'/>
    <feature policy='require' name='topoext'/>
  </cpu>
Result: CPU security features are enabled.

2. Disable CPU secure parameter
# virt-install --name cpu --memory 1024 --disk /var/lib/libvirt/images/rhel7.7.qcow2  --import --cpu EPYC-IBPB,secure=no  --noautoconsole --debug
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>
    <feature policy='require' name='ibpb'/>
    <feature policy='require' name='virt-ssbd'/>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='x2apic'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='disable' name='svm'/>
    <feature policy='require' name='topoext'/>
  </cpu>
Result: CPU security features are not disabled with "secure=no" setting

As a summry, it seems virt-install does't work as expected when I specify "--cpu EPYC-IBPB,secure=no".
So Pavel, please help me have a look of virt-install cml output, thanks.

Comment 26 zhoujunqin 2019-05-20 09:13:12 UTC

Created attachment 1571247 [details]
debug log for secure=no

Comment 27 Pavel Hrdina 2019-05-20 20:05:44 UTC

Hi Junqin,

Nice catch, the secure=no doesn't actually work because it is processed after setting
CPU model and at that time it's too late.

Moving back to assigned as it needs to be fixed.

Thanks,

Pavel

Comment 28 zhoujunqin 2019-05-22 02:47:15 UTC

(In reply to Pavel Hrdina from comment #27)
> Hi Junqin,
> 
> Nice catch, the secure=no doesn't actually work because it is processed
> after setting
> CPU model and at that time it's too late.
> 
> Moving back to assigned as it needs to be fixed.
> 
> Thanks,
> 
> Pavel

Hi Pavel,
Do you mean there will be a new virt-manager version on rhel7.7.
If so, I hope you can build it early then we can do some related testing, thanks.

BR
juzhou

Comment 29 Pavel Hrdina 2019-05-22 14:48:38 UTC

Yes, there will be new build for RHEL-7.7, I'll try to build it this week.

Comment 31 zhoujunqin 2019-05-24 07:22:01 UTC

Try to verify this bug with new build:

virt-manager-1.5.0-6.el7.noarch
virt-install-1.5.0-6.el7.noarch
libvirt-4.5.0-18.el7.x86_64
qemu-kvm-rhev-2.12.0-29.el7.x86_64


Steps:
1. Prepare a host which support ibpb(there is IBRS or IBPB keyword in cpu model):  

# virsh capabilities
<capabilities>

  <host>
    <uuid>36383738-3231-4e43-3739-303130325a33</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>EPYC-IBPB</model>
      <vendor>AMD</vendor>
      <microcode version='134222375'/>
...

Scenario-1 Testing on virt-manager UI

1. Launch virt-manager
# virt-manager

2. Set 'Edit'->'Preferences'->'New VM defaults'->select 'CPU default' as 'Copy host CPU definition', then click 'Close'.

3. Install a vm.
3.1 Select the local connection: QEMU/KVM
3.2 Click NEW button at the top of Virtual Machine Manager dialogue.
3.3 Select Network Install(HTTP,FTP or NFS)", and Click "Forward" button.
3.4 Enter URL in "URL" field, such as "http://download-node-02.eng.bos.redhat.com/rhel-7/rel-eng/RHEL-7//RHEL-7.7-20190514.2/compose/Server/x86_64/os/"
3.5 Check "Automatically detect operating system based on install media", then click 'Forward'.
3.6 Keep following pages as default setting, and click 'Forward' step by step until click "Finish" button.
Result: Installation finishes with no error.

4. Check "new checkbox to control CPU security features" works. Double click a VM, and turn to "Show virtual hardware details" page.
Result: 
4.1 Cpu Model shows "EYPC-IBPB".
4.2 New checkbox "Enable available CPU security flaw mitigations" is added and it is checked by default.
# virsh dumpxml rhel7.7
...
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>
    <feature policy='require' name='ibpb'/>
    <feature policy='require' name='virt-ssbd'/>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='disable' name='arat'/>
    <feature policy='disable' name='svm'/>
  </cpu>


4.2.1 Login guest and check guest cpu feature have ibpb
# cat /proc/cpuinfo

fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm art rep_good nopl extd_apicid eagerfpu pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw retpoline_amd ssbd ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 virt_ssbd arat


Result: CPU security features(virt_ssbd, ssbd, ibpb) are added.

4.3 Uncheck "Enable available CPU security flaw mitigations" and restart vm again.
# virsh dumpxml rhel7.7
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='disable' name='arat'/>
    <feature policy='disable' name='svm'/>
  </cpu>


4.3.1 Login guest and check guest cpu feature don't have security CPU security features: virt_ssbd and ssbd.
# cat /proc/cpuinfo

fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm art rep_good nopl extd_apicid eagerfpu pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw retpoline_amd ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat


As a summary, it works well on virt-manager UI.

Scenario-2 Testing with virt-install cml.
1. Enable CPU secure parameter
# virt-install --name cpu --memory 1024 --disk /var/lib/libvirt/images/rhel7.7.qcow2  --import --cpu EPYC-IBPB,secure=yes --noautoconsole --debug
...
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>
    <feature policy='require' name='ibpb'/>
    <feature policy='require' name='virt-ssbd'/>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='x2apic'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='disable' name='svm'/>
    <feature policy='require' name='topoext'/>
  </cpu>

Result: CPU security features are enabled with "secure=yes" setting

2. Disable CPU secure parameter
# virt-install --name cpu --memory 1024 --disk /var/lib/libvirt/images/rhel7.7.qcow2  --import --cpu EPYC-IBPB,secure=no  --noautoconsole --debug
...
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='x2apic'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='disable' name='svm'/>
    <feature policy='require' name='topoext'/>
  </cpu>

Result: CPU security features are disabled with "secure=no" setting

As a summary, adding CPU security features work well, so I move this bug from ON_QA to VERIFIED, thanks.

Comment 33 errata-xmlrpc 2019-08-06 13:08:01 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2232

Note You need to log in before you can comment on or make changes to this bug.