Bug 1138222 - Fail to managedsave while configure <cpu mode='host-model'> in the guest's xml
Summary: Fail to managedsave while configure <cpu mode='host-model'> in the guest's xml
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.6
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Ján Tomko
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 1138221
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-09-04 10:07 UTC by vivian zhang
Modified: 2014-10-14 04:23 UTC (History)
15 users (show)

Fixed In Version: libvirt-0.10.2-46.el6
Doc Type: Bug Fix
Doc Text:
Clone Of: 1138221
Environment:
Last Closed: 2014-10-14 04:23:30 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:1374 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2014-10-14 08:11:54 UTC

Description vivian zhang 2014-09-04 10:07:02 UTC
+++ This bug was initially created as a clone of Bug #1138221 +++

Description of problem:
Fail to managedsave while configure <cpu mode='host-model'> in the guest's xml

Version-Release number of selected component (if applicable):
libvirt-1.2.8-1.el7.x86_64
qemu-kvm-rhev-2.1.0-3.el7.x86_64
kernel-3.10.0-150.el7.x86_64


How reproducible:
100%

Steps to Reproduce:

1. prepare a guest with <cpu mode='host-model'>
   #virsh dumpxml rhel7
    --
  <vcpu placement='static'>16</vcpu>
    --
  <cpu mode='host-model'>
    <model fallback='forbid'/>
    <topology sockets='2' cores='4' threads='2'/>
  </cpu>

2. Start the guest
#virsh start rhel7

3. managedsave the guest, will report the following error
   #virsh managedsave rhel7
error: Failed to save domain rhel7 state
error: internal error: unable to execute QEMU command 'migrate': State blocked by non-migratable device 'cpu'

4. it works well on libvirt-1.2.6-1.el7

5.Also hit this issue in rhel6.6

Actual results:
Fail to managedsave while configure <cpu mode='host-model'> in the guest's xml

Expected results:
should succeed to do managedsave

Comment 2 vivian zhang 2014-09-04 10:27:15 UTC
bug also hit on rhel6.6:
libvirt-0.10.2-45.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.441.el6.x86_64
kernel-2.6.32-498.el6.x86_64

bug can not be reproduced on rhel6.5.z: libvirt-0.10.2-29.el6_5.12.x86_64

Comment 3 Jiri Denemark 2014-09-04 13:13:02 UTC
This is a result of invtsc support. When a host CPU supports invtsc, the guest CPU created by host-model will have it too, which results in the QEMU error mentioned in bug description. There is an easy workaround for upstream (adding <feature name='invtsc' policy='disable'/> to guest CPU definition) but it is not supported on RHEL-6. Thus the only workaround in RHEL-6 would be looking at what host-model is translated to (using virsh dumpxml --update-cpu DOMAIN), removing invtsc feature from there and using the result with CPU mode='custom' instead of the original host-model in the domain definition.

Comment 5 Ján Tomko 2014-09-05 12:22:46 UTC
Now pushed upstream:
commit de0aeafe9ce3eb414c8b5d3aa8995d776a2952de
Author:     Ján Tomko <jtomko@redhat.com>
CommitDate: 2014-09-05 12:45:19 +0200

    Don't include non-migratable features in host-model
    
    Commit fba6bc4 introduced support for the 'invtsc' feature,
    which blocks migration. We should not include it in the
    host-model CPU by default, because it's intended to be used
    with migration.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1138221

git describe: v1.2.8-32-gde0aeaf

Comment 7 Marcelo Tosatti 2014-09-05 14:20:20 UTC
(In reply to Jiri Denemark from comment #3)
> This is a result of invtsc support. When a host CPU supports invtsc, the
> guest CPU created by host-model will have it too, which results in the QEMU
> error mentioned in bug description. 

Jiri, this should not be the case: invtsc must be explicitly enabled, as it breaks migration.

IOW, host CPU with invariant TSC support should not expose invariant TSC to a guest, unless explicitly enabled.

Can you confirm this is not the case on RHEL-6.6 ?

> There is an easy workaround for upstream
> (adding <feature name='invtsc' policy='disable'/> to guest CPU definition)
> but it is not supported on RHEL-6. Thus the only workaround in RHEL-6 would
> be looking at what host-model is translated to (using virsh dumpxml
> --update-cpu DOMAIN), removing invtsc feature from there and using the
> result with CPU mode='custom' instead of the original host-model in the
> domain definition.

Comment 8 Jiri Denemark 2014-09-05 14:25:46 UTC
No, "host-model" is not equivalent of -cpu host ("host-passthrough" is its equivalent). This is a bug in libvirt which explicitly enables invtsc for host-model CPUs if a host CPU supports it. That is, we had to teach libvirt not to enable for host-model CPUs because it breaks migration.

Comment 9 Eduardo Habkost 2014-09-05 14:28:26 UTC
(In reply to Marcelo Tosatti from comment #7)
> (In reply to Jiri Denemark from comment #3)
> > This is a result of invtsc support. When a host CPU supports invtsc, the
> > guest CPU created by host-model will have it too, which results in the QEMU
> > error mentioned in bug description. 
> 
> Jiri, this should not be the case: invtsc must be explicitly enabled, as it
> breaks migration.

I don't know if you are talking about libvirt or QEMU. Note that QEMU seems to be doing the right thing, but libvirt was really explicitly enabling invtsc. That's because "host-model" queries the host CPU and generates a -cpu command-line argument enabling all features detected on the host (in other words, this is not about "-cpu host", but something different. "-cpu host" is called "host-passthrough" on libvirt).

Comment 11 zhe peng 2014-09-11 06:53:09 UTC
I can reproduce this with libvirt-0.10.2-45.el6.x86_64
# virsh managedsave rhel6
error: Failed to save domain rhel6 state
error: internal error unable to execute QEMU command 'migrate': An undefined error has ocurred

# virsh migrate rhel6 --live qemu+ssh://10.66.85.72/system --verbose
error: Requested operation is not valid: domain has CPU feature: invtsc

test with build libvirt-0.10.2-46.el6.x86_64

# virsh managedsave rhel6

Domain rhel6 state saved by libvirt

# virsh migrate rhel6 --live qemu+ssh://10.66.85.72/system --verbose
error: unsupported configuration: guest and host CPU are not compatible: Host CPU does not provide required features: rtm, invpcid, erms, bmi2, smep, avx2, hle, bmi1, fsgsbase, abm, pdpe1gb, rdrand, f16c, movbe, fma, monitor

I thought this error because of two host cpu features flags are not same,by now element <feature> not support in host-model. "invtsc" not listed.

so i update xml to:
 <cpu mode='custom' match='exact'>
    <model fallback='allow'>Haswell</model>
    <vendor>Intel</vendor>
    <feature policy='disable' name='rtm'/>
    <feature policy='disable' name='invpcid'/>
    <feature policy='disable' name='erms'/>
    <feature policy='disable' name='bmi2'/>
    <feature policy='disable' name='smep'/>
    <feature policy='disable' name='avx2'/>
    <feature policy='disable' name='hle'/>
    <feature policy='disable' name='bmi1'/>
    <feature policy='disable' name='fsgsbase'/>
    <feature policy='disable' name='abm'/>
    <feature policy='disable' name='pdpe1gb'/>
    <feature policy='disable' name='rdrand'/>
    <feature policy='disable' name='f16c'/>
    <feature policy='disable' name='movbe'/>
    <feature policy='disable' name='fma'/>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='dtes64'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='xtpr'/>
    <feature policy='require' name='est'/>
    <feature policy='require' name='smx'/>
    <feature policy='require' name='tm'/>
    <feature policy='require' name='acpi'/>
    <feature policy='require' name='osxsave'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='pdcm'/>
    <feature policy='require' name='ds'/>
    <feature policy='require' name='tm2'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='pbe'/>
    <feature policy='require' name='ds_cpl'/>
  </cpu>

# virsh migrate test --live qemu+ssh://10.66.85.72/system --verbose
root@10.66.85.72's password: 
Migration: [100 %]

if add <feature policy='require' name='invtsc'/> 
migration will fialed.


Hi Jiri
 Could you help to confirm that verification step is enough? thanks in advance.

Comment 12 Jiri Denemark 2014-09-11 08:03:25 UTC
The guest CPU should be <cpu mode='host-model'/>. The patch for this bug does not do anything for custom mode CPUs.

Comment 13 zhe peng 2014-09-11 08:22:27 UTC
Thanks Jiri's help, my reproduce step and verification used guest xml:
.....  
<cpu mode='host-model'>
    <model fallback='forbid'/>
    <topology sockets='2' cores='4' threads='2'/>
  </cpu>
.....

ignore mode='custom'

managedsave worked well, so move to verified.

Comment 15 errata-xmlrpc 2014-10-14 04:23:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1374.html


Note You need to log in before you can comment on or make changes to this bug.