Bug 1375524

Summary: Failed to migrate guest back to rhel7.2 on host which support 'arat' with host-model
Product: Red Hat Enterprise Linux 7 Reporter: chhu
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED ERRATA QA Contact: Jing Qi <jinqi>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.3CC: dyuan, fjin, lhuang, rbalakri, xuzhang, yalzhang, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-2.5.0-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 17:14:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1199452    

Description chhu 2016-09-13 10:12:13 UTC
Description of problems:
Migrate a guest from rhel7.2 to rhel7.3 on host which support cpu feature 'arat', then try to migrate it back, met "error: internal error: Unknown CPU feature arat".


Version-Release number of selected component (if applicable):
libvirt-2.0.0-8.el7.x86_64
qemu-kvm-rhev-2.6.0-23.el7.x86_64

libvirt-1.2.17-13.el7_2.5.x86_64
qemu-kvm-rhev-2.3.0-31.el7_2.21.x86_64

How reproducible:
100%

1. Start a VM on rhel7.2 on host which support cpu feature 'arat'.
# virsh dumpxml r7|grep "<cpu" -A 5
  <cpu mode='host-model'>
    <model fallback='allow'/>
  </cpu>
# ps -ef|grep qemu
qemu      25019      1 93 01:53 ?        00:00:18 /usr/libexec/qemu-kvm -name r7... -cpu Haswell-noTSX,
+abm,+pdpe1gb,+rdrand,+f16c,+osxsave,+dca,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,
+tm,+ht,+ss,+acpi,+ds,+vme -m 1024

2. Migrate the VM to rhel7.3
# virsh migrate r7 qemu+ssh://ipaddress/system --live --verbose
Migration: [100 %]

On the target host:
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 1     r7                             running

# virsh dumpxml r7|grep "<cpu" -A 30
  <cpu mode='host-model'>
    <model fallback='allow'>Haswell-noTSX</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='abm'/>
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='rdrand'/>
    <feature policy='require' name='f16c'/>
    <feature policy='require' name='osxsave'/>
    <feature policy='require' name='dca'/>
    <feature policy='require' name='pdcm'/>
    <feature policy='require' name='xtpr'/>
    <feature policy='require' name='tm2'/>
    <feature policy='require' name='est'/>
    <feature policy='require' name='smx'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='ds_cpl'/>
    <feature policy='require' name='monitor'/>
    <feature policy='require' name='dtes64'/>
    <feature policy='require' name='pbe'/>
    <feature policy='require' name='tm'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='acpi'/>
    <feature policy='require' name='ds'/>
    <feature policy='require' name='vme'/>
  </cpu>
# ps -ef|grep qemu
qemu      35934      1  2 02:00 ?        00:00:03 /usr/libexec/qemu-kvm -name guest=r7...-cpu Haswell-noTSX,+vme,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,+monitor,
+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+dca,+osxsave,+f16c,+rdrand,+pdpe1gb,
+abm -m 1024

3. Tried to migrate VM back to rhel7.2
# virsh migrate r7 qemu+ssh://10.73.131.71/system --live --verbose
error: internal error: Unknown CPU feature arat

Check the dumpxml --migratable, there are 'arat','tsc_adjust','xsaveopt' added.
# virsh dumpxml r7 --migratable
<domain type='kvm'>
  <name>r7</name>
.....
  <cpu mode='host-model' match='exact'>
    <model fallback='allow'>Haswell-noTSX</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='ds'/>
    <feature policy='require' name='acpi'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='tm'/>
    <feature policy='require' name='pbe'/>
    <feature policy='require' name='dtes64'/>
    <feature policy='require' name='monitor'/>
    <feature policy='require' name='ds_cpl'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='smx'/>
    <feature policy='require' name='est'/>
    <feature policy='require' name='tm2'/>
    <feature policy='require' name='xtpr'/>
    <feature policy='require' name='pdcm'/>
    <feature policy='require' name='dca'/>
    <feature policy='require' name='osxsave'/>
    <feature policy='require' name='f16c'/>
    <feature policy='require' name='rdrand'/>
    <feature policy='require' name='arat'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='xsaveopt'/>
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='abm'/>
  </cpu>

Actual results:
In step3: Failed to migrate back to rhel7.2

Expected results:
In step3: Migrate back to rhel7.2 successfully.

Other information:
1) Test on host which support cpu features: arat, tsc_adjust,xsaveopt, they are new features in cpu_map.xml in rhel7.3
2) New features in rhel7.3 cpu_map.xml are:
tsc_adjust, cmt, clflushopt, xsaveopt, arat, xsavec,xgetbv1, xsaves, mbm_total, mbm_local.

Comment 2 chhu 2016-09-14 02:20:27 UTC
Migrate a guest from rhel7.2 to rhel7.3 on host:Skylake-Client, then try to migrate it back, met "error: internal error: Unknown CPU model Skylake-Client".

Comment 3 Jiri Denemark 2016-09-22 13:57:07 UTC
This should be fixed upstream as of

commit 7ce711a30eaf882ccd0217b2528362b563b6d670
Refs: v2.2.0-199-g7ce711a
Author:     Jiri Denemark <jdenemar>
AuthorDate: Wed Jun 22 15:53:48 2016 +0200
Commit:     Jiri Denemark <jdenemar>
CommitDate: Thu Sep 22 15:40:09 2016 +0200

    qemu: Update guest CPU def in live XML

    Storing the updated CPU definition in the live domain definition saves
    us from having to update it over and over when we need it. Not to
    mention that we will soon further update the CPU definition according to
    QEMU once it's started.

    A highly wanted side effect of this patch, libvirt will pass all CPU
    features explicitly specified in domain XML to QEMU, even those that are
    already included in the host model.

    This patch should fix the following bugs:
        https://bugzilla.redhat.com/show_bug.cgi?id=1207095
        https://bugzilla.redhat.com/show_bug.cgi?id=1339680
        https://bugzilla.redhat.com/show_bug.cgi?id=1371039
        https://bugzilla.redhat.com/show_bug.cgi?id=1373849
        https://bugzilla.redhat.com/show_bug.cgi?id=1375524
        https://bugzilla.redhat.com/show_bug.cgi?id=1377913

    Signed-off-by: Jiri Denemark <jdenemar>

Comment 5 Jing Qi 2017-03-03 09:32:10 UTC
Verified on RHEL7.3 with libvirt-2.5.0-1.el7.x86_64 & RHEL7.2 with libvirt-1.2.17-13.el7.x86_64.  
Started a guest VM in RHEL7.2 and migrated it RHEL7.3 , then migrated it back to 
RHEL7.2 , two migrations both passed and guest vm ran well after migration.

Comment 6 errata-xmlrpc 2017-08-01 17:14:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 7 errata-xmlrpc 2017-08-01 23:55:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 8 errata-xmlrpc 2017-08-02 01:27:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846