Bug 1470582
Summary: | [UPDATES] Target CPU mode custom does not match source host-model | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | Jaroslav Reznik <jreznik> |
Component: | libvirt | Assignee: | Jiri Denemark <jdenemar> |
Status: | CLOSED ERRATA | QA Contact: | Luyao Huang <lhuang> |
Severity: | high | Docs Contact: | |
Priority: | high | ||
Version: | 7.4 | CC: | berrange, chhu, dasmith, dyuan, eglynn, ehabkost, jdenemar, jishao, jreznik, jsuchane, kchamart, knoel, lhuang, lyarwood, mcornea, mtessun, rbalakri, rbryant, salmy, sasha, sbauza, sferdjao, sgordon, srevivo, vromanso, xuzhang, yalzhang, yanqzhan, yprokule |
Target Milestone: | rc | Keywords: | ZStream |
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | libvirt-3.2.0-14.el7_4.1 | Doc Type: | Bug Fix |
Doc Text: |
Cause: Previously libvirt transformed a host-model CPU into a real model everytime it needed to know the actual CPU model. Current libvirt does this transformation only once when a new domain with a host-model CPU is started to make sure the same CPU model is used for the whole life time of the domain.
Consequence: When libvirtd is restarted after an upgrade from an earlier version, it reconnects to running domains some of which might have been started with a host-model CPU that was not transformed into an actual model. Current libvirt does not expect a host-model CPU in a definition of a running domain which caused several issues when trying to migrate such domain.
Fix: Libvirt transforms host-model CPUs into actual CPU models of all running domains when reconnecting to them.
Result: Domains started on RHEL-7.3 can be successfully migrated even after libvirt is upgraded to a newer version.
|
Story Points: | --- |
Clone Of: | 1463957 | Environment: | |
Last Closed: | 2017-08-01 11:30:30 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1463957 | ||
Bug Blocks: |
Description
Jaroslav Reznik
2017-07-13 09:06:28 UTC
Verify this bug with libvirt-3.2.0-14.el7_4.1.x86_64: S1: start guest on 7.3 -> update to 7.4 -> migrate guest to 7.4 host 1. prepare 3 guest, which have custom, host-model and host-passthrough cpu: # virsh dumpxml r7-mig ... <cpu mode='custom' match='minimum'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> ... # virsh dumpxml r7-mig2 ... <cpu mode='host-model'> <model fallback='allow'/> ... # virsh dumpxml r7-mig3 ... <cpu mode='host-passthrough'> ... 2. start guest: # virsh start r7-mig3 Domain r7-mig3 started # virsh start r7-mig2 Domain r7-mig2 started # virsh start r7-mig Domain r7-mig started 3. update libvirt to 7.4: yum update libvirt* qemu* 4. recheck guest xml: # virsh dumpxml r7-mig <cpu mode='custom' match='minimum' check='partial'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <numa> <cell id='0' cpus='0-2' memory='524288' unit='KiB'/> <cell id='1' cpus='3-5' memory='524288' unit='KiB'/> </numa> </cpu> # virsh dumpxml r7-mig2 <cpu mode='custom' match='exact' check='full'> <model fallback='allow'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig3 <cpu mode='host-passthrough' check='none'> <numa> <cell id='0' cpus='0-2' memory='524288' unit='KiB'/> <cell id='1' cpus='3-5' memory='524288' unit='KiB'/> </numa> </cpu> 5. migrate to rhel7.4 host (same cpu with source): # virsh migrate r7-mig qemu+ssh://target/system --live # virsh migrate r7-mig2 qemu+ssh://target/system --live # virsh migrate r7-mig3 qemu+ssh://target/system --live 6. recheck guest one the target host: # virsh dumpxml r7-mig | grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='disable' name='invtsc'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig2 | grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig3 | grep -A28 '<cpu' <cpu mode='host-passthrough' check='none'> <numa> <cell id='0' cpus='0-2' memory='524288' unit='KiB'/> <cell id='1' cpus='3-5' memory='524288' unit='KiB'/> </numa> </cpu> 7. migrate back to source: # virsh migrate r7-mig qemu+ssh://source/system --live # virsh migrate r7-mig2 qemu+ssh://source/system --live # virsh migrate r7-mig3 qemu+ssh://source/system --live 8. recheck guest in the source host: # virsh dumpxml r7-mig2 |grep -A28 "<cpu" <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig |grep -A28 "<cpu" <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='disable' name='invtsc'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig3 |grep -A28 "<cpu" <cpu mode='host-passthrough' check='none'> <numa> <cell id='0' cpus='0-2' memory='524288' unit='KiB'/> <cell id='1' cpus='3-5' memory='524288' unit='KiB'/> </numa> </cpu> 9. check the qemu command and log in guest log S2: guest migrate from 7.3 host -> update to 7.4 -> migrate to 7.4 1. prepare 3 guest, which have custom, host-model and host-passthrough cpu: # virsh dumpxml r7-mig ... <cpu mode='custom' match='minimum'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> ... # virsh dumpxml r7-mig2 ... <cpu mode='host-model'> <model fallback='allow'/> ... # virsh dumpxml r7-mig3 ... <cpu mode='host-passthrough'> ... 2. start and migrate to 7.3 host: # virsh start r7-mig3 Domain r7-mig3 started # virsh start r7-mig2 Domain r7-mig2 started # virsh start r7-mig Domain r7-mig started # virsh migrate r7-mig qemu+ssh://target/system --live # virsh migrate r7-mig2 qemu+ssh://target/system --live # virsh migrate r7-mig3 qemu+ssh://target/system --live 3. check the guest xml in target: # virsh dumpxml r7-mig3 ... <cpu mode='host-passthrough'> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> <feature policy='require' name='tce'/> <feature policy='require' name='nodeid_msr'/> <feature policy='require' name='topoext'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='perfctr_nb'/> ... # virsh dumpxml r7-mig2 ... <cpu mode='host-model'> <model fallback='allow'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> <feature policy='require' name='tce'/> <feature policy='require' name='nodeid_msr'/> <feature policy='require' name='topoext'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='perfctr_nb'/> ... # virsh dumpxml r7-mig <cpu mode='custom' match='exact'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> 4. update target and source host to 7.4: # yum update libvirt* qemu* 5. recheck guest xml: # virsh dumpxml r7-mig |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='partial'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> <feature policy='require' name='tce'/> <feature policy='require' name='nodeid_msr'/> <feature policy='require' name='topoext'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='perfctr_nb'/> # virsh dumpxml r7-mig2 |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='allow'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig3 |grep -A28 '<cpu' <cpu mode='host-passthrough' check='none'> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> <feature policy='require' name='tce'/> <feature policy='require' name='nodeid_msr'/> <feature policy='require' name='topoext'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='perfctr_nb'/> 9. migrate to source host: 10. recheck the guest xml: # virsh dumpxml r7-mig |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig2 |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> 11. check the qemu command and log in guest log S3: start guest on 7.3 -> update to 7.4 -> migrate guest to 7.4 host with --xml 1. prepare 3 guest, which have custom, host-model and host-passthrough cpu: # virsh dumpxml r7-mig ... <cpu mode='custom' match='minimum'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> ... # virsh dumpxml r7-mig2 ... <cpu mode='host-model'> <model fallback='allow'/> ... # virsh dumpxml r7-mig3 ... <cpu mode='host-passthrough'> ... 2. start guest: # virsh start r7-mig3 Domain r7-mig3 started # virsh start r7-mig2 Domain r7-mig2 started # virsh start r7-mig Domain r7-mig started 3. update libvirt to 7.4: yum update libvirt* qemu* 4. recheck guest xml: # virsh dumpxml r7-mig <cpu mode='custom' match='minimum' check='partial'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <numa> <cell id='0' cpus='0-2' memory='524288' unit='KiB'/> <cell id='1' cpus='3-5' memory='524288' unit='KiB'/> </numa> </cpu> # virsh dumpxml r7-mig2 <cpu mode='custom' match='exact' check='full'> <model fallback='allow'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig3 <cpu mode='host-passthrough' check='none'> <numa> <cell id='0' cpus='0-2' memory='524288' unit='KiB'/> <cell id='1' cpus='3-5' memory='524288' unit='KiB'/> </numa> </cpu> 5. migrate to rhel7.4 host the --xml (same cpu with source): # virsh dumpxml r7-mig2 --migratable > /tmp/r7-mig2.xml # virsh migrate r7-mig2 qemu+ssh://target/system --live --xml /tmp/r7-mig2.xml # virsh dumpxml r7-mig --migratable > /tmp/r7-mig.xml # virsh migrate r7-mig qemu+ssh://target/system --live --xml /tmp/r7-mig.xml # virsh dumpxml r7-mig3 --migratable > /tmp/r7-mig3.xml # virsh migrate r7-mig3 qemu+ssh://target/system --live --xml /tmp/r7-mig3.xml 6. recheck guest one the target host: # virsh dumpxml r7-mig | grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='disable' name='invtsc'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig2 | grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig3 | grep -A28 '<cpu' <cpu mode='host-passthrough' check='none'> <numa> <cell id='0' cpus='0-2' memory='524288' unit='KiB'/> <cell id='1' cpus='3-5' memory='524288' unit='KiB'/> </numa> </cpu> 7. migrate back to source: # virsh dumpxml r7-mig3 --migratable > /tmp/r7-mig3.xml # virsh migrate r7-mig3 qemu+ssh://source/system --live --xml /tmp/r7-mig3.xml # virsh dumpxml r7-mig2 --migratable > /tmp/r7-mig2.xml # virsh migrate r7-mig2 qemu+ssh://source/system --live --xml /tmp/r7-mig2.xml # virsh dumpxml r7-mig --migratable > /tmp/r7-mig.xml # virsh migrate r7-mig qemu+ssh://source/system --live --xml /tmp/r7-mig.xml 8. recheck guest in the source host: # virsh dumpxml r7-mig2 |grep -A28 "<cpu" <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig |grep -A28 "<cpu" <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='disable' name='invtsc'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig3 |grep -A28 "<cpu" <cpu mode='host-passthrough' check='none'> <numa> <cell id='0' cpus='0-2' memory='524288' unit='KiB'/> <cell id='1' cpus='3-5' memory='524288' unit='KiB'/> </numa> </cpu> 9. check the qemu command and log in guest log S4: guest migrate from 7.3 host -> update to 7.4 -> migrate to 7.4 with --xml 1. prepare 3 guest, which have custom, host-model and host-passthrough cpu: # virsh dumpxml r7-mig ... <cpu mode='custom' match='minimum'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> ... # virsh dumpxml r7-mig2 ... <cpu mode='host-model'> <model fallback='allow'/> ... # virsh dumpxml r7-mig3 ... <cpu mode='host-passthrough'> ... 2. start and migrate to 7.3 host: # virsh start r7-mig3 Domain r7-mig3 started # virsh start r7-mig2 Domain r7-mig2 started # virsh start r7-mig Domain r7-mig started # virsh migrate r7-mig qemu+ssh://target/system --live # virsh migrate r7-mig2 qemu+ssh://target/system --live # virsh migrate r7-mig3 qemu+ssh://target/system --live 3. check the guest xml in target: # virsh dumpxml r7-mig3 ... <cpu mode='host-passthrough'> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> <feature policy='require' name='tce'/> <feature policy='require' name='nodeid_msr'/> <feature policy='require' name='topoext'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='perfctr_nb'/> ... # virsh dumpxml r7-mig2 ... <cpu mode='host-model'> <model fallback='allow'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> <feature policy='require' name='tce'/> <feature policy='require' name='nodeid_msr'/> <feature policy='require' name='topoext'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='perfctr_nb'/> ... # virsh dumpxml r7-mig <cpu mode='custom' match='exact'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> 4. update target and source host to 7.4: # yum update libvirt* qemu* 5. recheck guest xml: # virsh dumpxml r7-mig |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='partial'> <model fallback='allow'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> <feature policy='require' name='tce'/> <feature policy='require' name='nodeid_msr'/> <feature policy='require' name='topoext'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='perfctr_nb'/> # virsh dumpxml r7-mig2 |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='allow'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig3 |grep -A28 '<cpu' <cpu mode='host-passthrough' check='none'> <feature policy='require' name='vme'/> <feature policy='require' name='ht'/> <feature policy='require' name='monitor'/> <feature policy='require' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='require' name='ibs'/> <feature policy='require' name='skinit'/> <feature policy='require' name='wdt'/> <feature policy='require' name='lwp'/> <feature policy='require' name='tce'/> <feature policy='require' name='nodeid_msr'/> <feature policy='require' name='topoext'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='perfctr_nb'/> 9. migrate to source host: # virsh dumpxml r7-mig --migratable > /tmp/r7-mig.xml # virsh migrate r7-mig qemu+ssh://source/system --live --xml /tmp/r7-mig.xml # virsh dumpxml r7-mig2 --migratable > /tmp/r7-mig2.xml # virsh migrate r7-mig2 qemu+ssh://source/system --live --xml /tmp/r7-mig2.xml # virsh dumpxml r7-mig3 --migratable > /tmp/r7-mig3.xml # virsh migrate r7-mig3 qemu+ssh://source/system --live --xml /tmp/r7-mig3.xml 10. recheck the guest xml: # virsh dumpxml r7-mig |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig2 |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> # virsh dumpxml r7-mig |grep -A28 '<cpu' <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <feature policy='disable' name='invtsc'/> <feature policy='require' name='vme'/> <feature policy='disable' name='ht'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='osxsave'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='disable' name='extapic'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='ibs'/> <feature policy='disable' name='skinit'/> <feature policy='disable' name='wdt'/> <feature policy='disable' name='lwp'/> <feature policy='disable' name='tce'/> <feature policy='disable' name='nodeid_msr'/> <feature policy='disable' name='topoext'/> <feature policy='disable' name='perfctr_core'/> <feature policy='disable' name='perfctr_nb'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> 11. check the qemu command and log in guest log Also retest with qemu-kvm (qemu-kvm-1.5.3-141.el7_4.1.x86_64) and libvirt (libvirt-3.2.0-14.el7_4.1.x86_64) with the same steps in comment 5, migrate success and no regression. Test some basic test scenario with r7-mig, r7-mig2 and r7-mig3 (xml check comment 5), no regression been found. Hi Jirka, Our auto job catch a problem on an old cpu model machine, the managedsave and migration have been broken on that machine, here is some info and reproduce steps: virsh # capabilities <capabilities> <host> <uuid>00020003-0004-0005-0006-000700080009</uuid> <cpu> <arch>x86_64</arch> <model>Opteron_G2</model> <vendor>AMD</vendor> <topology sockets='1' cores='2' threads='1'/> <feature name='vme'/> <feature name='ht'/> <feature name='mmxext'/> <feature name='fxsr_opt'/> <feature name='3dnowext'/> <feature name='3dnow'/> <feature name='cmp_legacy'/> <feature name='extapic'/> <feature name='cr8legacy'/> <pages unit='KiB' size='4'/> <pages unit='KiB' size='2048'/> </cpu> # virsh dumpxml avocado-vt-vm1 ... <cpu mode='host-model' check='partial'> <model fallback='allow'/> </cpu> ... # virsh dumpxml avocado-vt-vm1 ... <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>phenom</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='require' name='cx16'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='tsc-deadline'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='arat'/> <feature policy='require' name='lahf_lm'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='cr8legacy'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='svm'/> <feature policy='disable' name='popcnt'/> <feature policy='disable' name='pdpe1gb'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='abm'/> <feature policy='disable' name='sse4a'/> </cpu> ... # virsh dumpxml avocado-vt-vm1 --migratable ... <cpu mode='custom' match='exact' check='partial'> <model fallback='forbid'>phenom</model> <vendor>AMD</vendor> <feature policy='require' name='vme'/> <feature policy='require' name='cx16'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='tsc-deadline'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='arat'/> <feature policy='require' name='lahf_lm'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='cr8legacy'/> <feature policy='disable' name='monitor'/> <feature policy='disable' name='svm'/> </cpu> ... # virsh managedsave avocado-vt-vm1 Domain avocado-vt-vm1 state saved by libvirt # virsh start avocado-vt-vm1 error: Failed to start domain avocado-vt-vm1 error: operation failed: guest CPU doesn't match specification And i have test on the libvirt 3.2.0-14, still can reproduce it. Could you please help to check if this problem is a critical problem that need been fixed on the 7.4.z ? And more information, i tried with different libvirt version and found libvirt-3.2.0-13.el7.x86_64 didn't have this problem, so it is a regression issue introduced in the libvirt-3.2.0-14.el7.x86_64. (In reply to Luyao Huang from comment #9) > # virsh start avocado-vt-vm1 > error: Failed to start domain avocado-vt-vm1 > error: operation failed: guest CPU doesn't match specification Could you please file a new BZ for this issue? According to the log, the phenom CPU model defined in QEMU contains "npt" feature while libvirt doesn't know anything about the feature. When a new domain is started, QEMU disables this feature on your host, which is ignored by libvirt (it doesn't know the feature), but once restoring the domain it sees QEMU unexpectedly disabled some feature and complains about it. (In reply to Jiri Denemark from comment #13) > (In reply to Luyao Huang from comment #9) > > # virsh start avocado-vt-vm1 > > error: Failed to start domain avocado-vt-vm1 > > error: operation failed: guest CPU doesn't match specification > > Could you please file a new BZ for this issue? According to the log, the > phenom CPU model defined in QEMU contains "npt" feature while libvirt > doesn't know anything about the feature. When a new domain is started, QEMU > disables this feature on your host, which is ignored by libvirt (it doesn't > know the feature), but once restoring the domain it sees QEMU unexpectedly > disabled some feature and complains about it. Got it ! thanks a lot for your help. New bug is bug 1473516 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:2334 |