RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1377913 - Libvirt should refuse to start a Skylake-Client VM on a non-Skylake CPU.
Summary: Libvirt should refuse to start a Skylake-Client VM on a non-Skylake CPU.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.3
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: rc
: 7.4
Assignee: Jiri Denemark
QA Contact: Jing Qi
URL:
Whiteboard:
Depends On:
Blocks: libvirtCPUconfig
TreeView+ depends on / blocked
 
Reported: 2016-09-21 02:00 UTC by chhu
Modified: 2017-08-01 23:57 UTC (History)
7 users (show)

Fixed In Version: libvirt-2.5.0-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-01 17:16:43 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:1846 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2017-08-01 18:02:50 UTC

Description chhu 2016-09-21 02:00:25 UTC 
Description: libvirt should refuse to start a Skylake-Client VM on a non-Skylake CPU.

How producible:
100%

Tested with packages:
libvirt-2.0.0-9.el7.x86_64
qemu-kvm-rhev-2.6.0-25.el7.x86_64
3.10.0-506.el7.x86_64

Test steps:
1. Started a VM with host mode: Skylake-Client on host: Haswell-noTSX/Broadwell/SandyBridge successfully. Should refuse to start the VM.
<cpu mode='host-model'>
 <model>Skylake-Client</model> or <model fallback='forbid'>Skylake-Client</model>
</cpu>

1)on host: Haswell-noTSX
# virsh start r7.3
Domain r7.3 started

# ps -ef|grep qemu-kvm
qemu      19302      1  4 22:07 ?        00:00:21 /usr/libexec/qemu-kvm -name guest=r7.3... -cpu Haswell-noTSX,+vme,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,
+monitor,+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+dca,+osxsave,+f16c,+rdrand,
+arat,+tsc_adjust,+xsaveopt,+pdpe1gb,+abm -m 1024

2)on host: Broadwell
# virsh start r7.3
Domain r7.3 started

# ps -ef|grep qemu
qemu      22036      1 40 01:00 ?        00:00:20 /usr/libexec/qemu-kvm -name guest=r7.3,...-cpu Broadwell,+vme,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,
+monitor,+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+dca,+osxsave,+f16c,+rdrand,
+arat,+tsc_adjust,+xsaveopt,+pdpe1gb,+abm,+rtm,+hle -m 1024

3)on host: SandyBridge
# virsh start r7.3
Domain r7.3 started

# ps -ef|grep qemu
qemu      4326     1 70 15:04 ?        00:00:20 /usr/libexec/qemu-kvm -name guest=r7.3,... -cpu SandyBridge,+vme,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,
+monitor,+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+pcid,+osxsave,+arat,
+xsaveopt -m 1024

Actual results:
Start a Skylake-Client VM on a non-Skylake CPU:Haswell-noTSX/Broadwell/SandyBridge successfully

Expected results:
Libvirt should refuse to start a Skylake-Client VM on a non-Skylake CPU.
Comment 4 Jiri Denemark 2016-09-22 13:56:59 UTC 
This should be fixed upstream as of

commit 7ce711a30eaf882ccd0217b2528362b563b6d670
Refs: v2.2.0-199-g7ce711a
Author:     Jiri Denemark <jdenemar>
AuthorDate: Wed Jun 22 15:53:48 2016 +0200
Commit:     Jiri Denemark <jdenemar>
CommitDate: Thu Sep 22 15:40:09 2016 +0200

    qemu: Update guest CPU def in live XML

    Storing the updated CPU definition in the live domain definition saves
    us from having to update it over and over when we need it. Not to
    mention that we will soon further update the CPU definition according to
    QEMU once it's started.

    A highly wanted side effect of this patch, libvirt will pass all CPU
    features explicitly specified in domain XML to QEMU, even those that are
    already included in the host model.

    This patch should fix the following bugs:
        https://bugzilla.redhat.com/show_bug.cgi?id=1207095
        https://bugzilla.redhat.com/show_bug.cgi?id=1339680
        https://bugzilla.redhat.com/show_bug.cgi?id=1371039
        https://bugzilla.redhat.com/show_bug.cgi?id=1373849
        https://bugzilla.redhat.com/show_bug.cgi?id=1375524
        https://bugzilla.redhat.com/show_bug.cgi?id=1377913

    Signed-off-by: Jiri Denemark <jdenemar>
Comment 6 Jing Qi 2017-03-03 07:12:15 UTC 
Verified with version: libvirt-2.5.0-1.el7.x86_64
                       qemu-kvm-rhev-2.8.0-3.el7.x86_64

On a host with CPU type of "SandyBridge" and the cpu part in domain xml file is as below, the guest vm can't be started.

<cpu mode='host-model'>
    <model fallback='allow'>Skylake-Client</model>
  </cpu>

#virsh start  rhel7
error: Failed to start domain rhel7
error: the CPU is incompatible with host CPU: Host CPU does not provide required features: fma, movbe, f16c, rdrand, fsgsbase, bmi1, hle, avx2, smep, bmi2, erms, invpcid, rtm, mpx, rdseed, adx, smap, xsavec, xgetbv1, abm, 3dnowprefetch
Comment 7 yalzhang@redhat.com 2017-06-18 07:10:00 UTC 
Hi Jiri,

I have a question about this bug. In libvirt.org page, it is said that for <cpu mode='host-model'> :

"The match attribute can't be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used."

Refer to: http://libvirt.org/formatdomain.html#elementsCPU

As it is described in comment 0, the guest started but with the host-model cpu(Haswell-noTSX/Broadwell/SandyBridge), not Skylake-Client which is specified in the xml. I think it is acceptable, and libvirt just need to ignore the model setting in the xml:

<model>Skylake-Client</model> or <model fallback='forbid'>Skylake-Client</model>

or report error to such setting such as "Specifying CPU model is not supported in <cpu mode='host-model'>"

If the customer want to use Skylake-Client, he need to set like this:

<cpu mode='custom'>
 <model>Skylake-Client</model>
</cpu>

Could you help to confirm? thank you very much!
Comment 8 Jiri Denemark 2017-06-19 08:41:17 UTC 
(In reply to yalzhang from comment #7)
> As it is described in comment 0, the guest started but with the host-model
> cpu(Haswell-noTSX/Broadwell/SandyBridge), not Skylake-Client which is
> specified in the xml. I think it is acceptable, and libvirt just need to
> ignore the model setting in the xml:
> 
> <model>Skylake-Client</model> or <model
> fallback='forbid'>Skylake-Client</model>
> 
> or report error to such setting such as "Specifying CPU model is not
> supported in <cpu mode='host-model'>"

No, we need to ignore it to maintain backward compatibility. Reporting an error could break existing domains.

> If the customer want to use Skylake-Client, he need to set like this:
> 
> <cpu mode='custom'>
>  <model>Skylake-Client</model>
> </cpu>

Yes.
Comment 9 yalzhang@redhat.com 2017-06-21 07:55:25 UTC 
(In reply to Jiri Denemark from comment #8)
> (In reply to yalzhang from comment #7)
> > As it is described in comment 0, the guest started but with the host-model
> > cpu(Haswell-noTSX/Broadwell/SandyBridge), not Skylake-Client which is
> > specified in the xml. I think it is acceptable, and libvirt just need to
> > ignore the model setting in the xml:
> > 
> > <model>Skylake-Client</model> or <model
> > fallback='forbid'>Skylake-Client</model>
> > 

How do you think about this proposal? 

As it is said in libvirt.org:
"The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, ..." and "Specifying CPU model is not supported either..." 

As in comment 6, libvirt check the model and fails, but in fact, the model will not be used in the running guest. So the checking just makes no sense.

For PowerISA, it seems different, may be current behavior is needed for the compatibility or the document need to be modified. Please help to confirm, thank you!

Refer to http://libvirt.org/formatdomain.html#elementsCPU  the "host-model" part
there is an example like this:
<cpu mode='host-model'>
  <model>power7</model>
</cpu>
Comment 10 Jiri Denemark 2017-06-21 11:25:36 UTC 
The documentation already says power is different: "PowerISA allows processors to run VMs in binary compatibility mode supporting an older version of ISA. Libvirt on PowerPC architecture uses the host-model to signify a guest mode CPU running in binary compatibility mode. Example: When a user needs a power7 VM to run in compatibility mode on a Power8 host, this can be described in XML as follows : 

<cpu mode='host-model'>
  <model>power7</model>
</cpu>"
Comment 11 yalzhang@redhat.com 2017-06-26 03:01:31 UTC 
Hi, Jiri, sorry for that I did not make it clear. Please help to confirm, thank you very much.

As it is described in comment 0, the guest started but with the host-model cpu(Haswell-noTSX/Broadwell/SandyBridge), not Skylake-Client which is specified in the xml. 

In addition，
As it is said in libvirt.org:
"The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, ..." and "Specifying CPU model is not supported either..." 

As in comment 6, libvirt check the model and fails, but in fact, the model will not be used in the running guest. So the checking just makes no sense.

I think libvirt may need to only ignore the model setting in the xml(may be only for x86_64 model, not for power).

Expected:
When set host-model with skylake-client model like this:
 <cpu mode='host-model'>
    <model fallback='allow'>Skylake-Client</model>
  </cpu>

libvirt ignore the model setting in inactive xml
 <cpu mode='host-model'/>

and the guest can start with the cpu xml copy from "# virsh domcapabilites"
Comment 12 Jiri Denemark 2017-06-27 07:59:03 UTC 
I somehow didn't notice the bugs description was talking about <cpu mode='host-model'><model>Skylake-Client</model></cpu>. This is a valid configuration sent by old libvirt during migration (new libvirt transforms host-model into custom CPU), but it's not something users should add into the XML. Libvirt will honor the CPU model for incoming migration, but it may just ignore it when starting a fresh domain. This applies to x86_64, of course. On Power, the model in a host-model CPU will always be honored.
Comment 13 errata-xmlrpc 2017-08-01 17:16:43 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Comment 14 errata-xmlrpc 2017-08-01 23:57:38 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Note You need to log in before you can comment on or make changes to this bug.