RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1138222 - Fail to managedsave while configure <cpu mode='host-model'> in the guest's xml
Summary: Fail to managedsave while configure <cpu mode='host-model'> in the guest's xml
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.6
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Ján Tomko
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 1138221
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-09-04 10:07 UTC by vivian zhang
Modified: 2014-10-14 04:23 UTC (History)
15 users (show)

Fixed In Version: libvirt-0.10.2-46.el6
Doc Type: Bug Fix
Doc Text:
Clone Of: 1138221
Environment:
Last Closed: 2014-10-14 04:23:30 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:1374 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2014-10-14 08:11:54 UTC

Description vivian zhang 2014-09-04 10:07:02 UTC
+++ This bug was initially created as a clone of Bug #1138221 +++

Description of problem:
Fail to managedsave while configure <cpu mode='host-model'> in the guest's xml

Version-Release number of selected component (if applicable):
libvirt-1.2.8-1.el7.x86_64
qemu-kvm-rhev-2.1.0-3.el7.x86_64
kernel-3.10.0-150.el7.x86_64


How reproducible:
100%

Steps to Reproduce:

1. prepare a guest with <cpu mode='host-model'>
   #virsh dumpxml rhel7
    --
  <vcpu placement='static'>16</vcpu>
    --
  <cpu mode='host-model'>
    <model fallback='forbid'/>
    <topology sockets='2' cores='4' threads='2'/>
  </cpu>

2. Start the guest
#virsh start rhel7

3. managedsave the guest, will report the following error
   #virsh managedsave rhel7
error: Failed to save domain rhel7 state
error: internal error: unable to execute QEMU command 'migrate': State blocked by non-migratable device 'cpu'

4. it works well on libvirt-1.2.6-1.el7

5.Also hit this issue in rhel6.6

Actual results:
Fail to managedsave while configure <cpu mode='host-model'> in the guest's xml

Expected results:
should succeed to do managedsave

Comment 2 vivian zhang 2014-09-04 10:27:15 UTC
bug also hit on rhel6.6:
libvirt-0.10.2-45.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.441.el6.x86_64
kernel-2.6.32-498.el6.x86_64

bug can not be reproduced on rhel6.5.z: libvirt-0.10.2-29.el6_5.12.x86_64

Comment 3 Jiri Denemark 2014-09-04 13:13:02 UTC
This is a result of invtsc support. When a host CPU supports invtsc, the guest CPU created by host-model will have it too, which results in the QEMU error mentioned in bug description. There is an easy workaround for upstream (adding <feature name='invtsc' policy='disable'/> to guest CPU definition) but it is not supported on RHEL-6. Thus the only workaround in RHEL-6 would be looking at what host-model is translated to (using virsh dumpxml --update-cpu DOMAIN), removing invtsc feature from there and using the result with CPU mode='custom' instead of the original host-model in the domain definition.

Comment 5 Ján Tomko 2014-09-05 12:22:46 UTC
Now pushed upstream:
commit de0aeafe9ce3eb414c8b5d3aa8995d776a2952de
Author:     Ján Tomko <jtomko>
CommitDate: 2014-09-05 12:45:19 +0200

    Don't include non-migratable features in host-model
    
    Commit fba6bc4 introduced support for the 'invtsc' feature,
    which blocks migration. We should not include it in the
    host-model CPU by default, because it's intended to be used
    with migration.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1138221

git describe: v1.2.8-32-gde0aeaf

Comment 7 Marcelo Tosatti 2014-09-05 14:20:20 UTC
(In reply to Jiri Denemark from comment #3)
> This is a result of invtsc support. When a host CPU supports invtsc, the
> guest CPU created by host-model will have it too, which results in the QEMU
> error mentioned in bug description. 

Jiri, this should not be the case: invtsc must be explicitly enabled, as it breaks migration.

IOW, host CPU with invariant TSC support should not expose invariant TSC to a guest, unless explicitly enabled.

Can you confirm this is not the case on RHEL-6.6 ?

> There is an easy workaround for upstream
> (adding <feature name='invtsc' policy='disable'/> to guest CPU definition)
> but it is not supported on RHEL-6. Thus the only workaround in RHEL-6 would
> be looking at what host-model is translated to (using virsh dumpxml
> --update-cpu DOMAIN), removing invtsc feature from there and using the
> result with CPU mode='custom' instead of the original host-model in the
> domain definition.

Comment 8 Jiri Denemark 2014-09-05 14:25:46 UTC
No, "host-model" is not equivalent of -cpu host ("host-passthrough" is its equivalent). This is a bug in libvirt which explicitly enables invtsc for host-model CPUs if a host CPU supports it. That is, we had to teach libvirt not to enable for host-model CPUs because it breaks migration.

Comment 9 Eduardo Habkost 2014-09-05 14:28:26 UTC
(In reply to Marcelo Tosatti from comment #7)
> (In reply to Jiri Denemark from comment #3)
> > This is a result of invtsc support. When a host CPU supports invtsc, the
> > guest CPU created by host-model will have it too, which results in the QEMU
> > error mentioned in bug description. 
> 
> Jiri, this should not be the case: invtsc must be explicitly enabled, as it
> breaks migration.

I don't know if you are talking about libvirt or QEMU. Note that QEMU seems to be doing the right thing, but libvirt was really explicitly enabling invtsc. That's because "host-model" queries the host CPU and generates a -cpu command-line argument enabling all features detected on the host (in other words, this is not about "-cpu host", but something different. "-cpu host" is called "host-passthrough" on libvirt).

Comment 11 zhe peng 2014-09-11 06:53:09 UTC
I can reproduce this with libvirt-0.10.2-45.el6.x86_64
# virsh managedsave rhel6
error: Failed to save domain rhel6 state
error: internal error unable to execute QEMU command 'migrate': An undefined error has ocurred

# virsh migrate rhel6 --live qemu+ssh://10.66.85.72/system --verbose
error: Requested operation is not valid: domain has CPU feature: invtsc

test with build libvirt-0.10.2-46.el6.x86_64

# virsh managedsave rhel6

Domain rhel6 state saved by libvirt

# virsh migrate rhel6 --live qemu+ssh://10.66.85.72/system --verbose
error: unsupported configuration: guest and host CPU are not compatible: Host CPU does not provide required features: rtm, invpcid, erms, bmi2, smep, avx2, hle, bmi1, fsgsbase, abm, pdpe1gb, rdrand, f16c, movbe, fma, monitor

I thought this error because of two host cpu features flags are not same,by now element <feature> not support in host-model. "invtsc" not listed.

so i update xml to:
 <cpu mode='custom' match='exact'>
    <model fallback='allow'>Haswell</model>
    <vendor>Intel</vendor>
    <feature policy='disable' name='rtm'/>
    <feature policy='disable' name='invpcid'/>
    <feature policy='disable' name='erms'/>
    <feature policy='disable' name='bmi2'/>
    <feature policy='disable' name='smep'/>
    <feature policy='disable' name='avx2'/>
    <feature policy='disable' name='hle'/>
    <feature policy='disable' name='bmi1'/>
    <feature policy='disable' name='fsgsbase'/>
    <feature policy='disable' name='abm'/>
    <feature policy='disable' name='pdpe1gb'/>
    <feature policy='disable' name='rdrand'/>
    <feature policy='disable' name='f16c'/>
    <feature policy='disable' name='movbe'/>
    <feature policy='disable' name='fma'/>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='dtes64'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='xtpr'/>
    <feature policy='require' name='est'/>
    <feature policy='require' name='smx'/>
    <feature policy='require' name='tm'/>
    <feature policy='require' name='acpi'/>
    <feature policy='require' name='osxsave'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='pdcm'/>
    <feature policy='require' name='ds'/>
    <feature policy='require' name='tm2'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='pbe'/>
    <feature policy='require' name='ds_cpl'/>
  </cpu>

# virsh migrate test --live qemu+ssh://10.66.85.72/system --verbose
root.85.72's password: 
Migration: [100 %]

if add <feature policy='require' name='invtsc'/> 
migration will fialed.


Hi Jiri
 Could you help to confirm that verification step is enough? thanks in advance.

Comment 12 Jiri Denemark 2014-09-11 08:03:25 UTC
The guest CPU should be <cpu mode='host-model'/>. The patch for this bug does not do anything for custom mode CPUs.

Comment 13 zhe peng 2014-09-11 08:22:27 UTC
Thanks Jiri's help, my reproduce step and verification used guest xml:
.....  
<cpu mode='host-model'>
    <model fallback='forbid'/>
    <topology sockets='2' cores='4' threads='2'/>
  </cpu>
.....

ignore mode='custom'

managedsave worked well, so move to verified.

Comment 15 errata-xmlrpc 2014-10-14 04:23:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1374.html


Note You need to log in before you can comment on or make changes to this bug.