Bug 1254124 - -cpu $cpu-model,+invtsc doesn't support migration
-cpu $cpu-model,+invtsc doesn't support migration
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev (Show other bugs)
7.2
x86_64 Linux
medium Severity medium
: rc
: ---
Assigned To: Marcelo Tosatti
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-17 04:37 EDT by Pei Zhang
Modified: 2016-03-28 05:18 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-08-17 12:43:34 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Pei Zhang 2015-08-17 04:37:44 EDT
Description of problem:
'-cpu $cpu-model,+invtsc' doesn't support migration. 

Version-Release number of selected component (if applicable):
Host:
Kernel:3.10.0-304.el7.x86_64
qemu-kvm-rhev:qemu-kvm-rhev-2.3.0-18.el7.x86_64
/proc/cpuinfo:
processor	: 31
vendor_id	: AuthenticAMD
cpu family	: 21
model		: 2
model name	: AMD Opteron(tm) Processor 6376                 
stepping	: 0
microcode	: 0x6000832
cpu MHz		: 2300.000
cache size	: 2048 KB
physical id	: 1
siblings	: 16
core id		: 7
cpu cores	: 8
apicid		: 79
initial apicid	: 47
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips	: 4587.94
TLB size	: 1536 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro


How reproducible:
100%

Steps to Reproduce:
1.
# /usr/libexec/qemu-kvm -cpu Opteron_G5,+invtsc -monitor stdio

2.
# /usr/libexec/qemu-kvm -cpu Opteron_G5,+invtsc -monitor stdio -incoming tcp:0:6666 

3.
(qemu) migrate -d tcp:10.66.9.120:6666
migrate: State blocked by non-migratable device 'cpu'


Actual results:
migrate: State blocked by non-migratable device 'cpu'

Expected results:
Should migrate successfully.

Additional info:
When using '-cpu host,+invtsc', migration works well. And Using '-cpu Opteron_G5,fpu' also works well.
Comment 1 FuXiangChun 2015-08-17 04:56:04 EDT
Eduardo,

Is it a valid problem?
Comment 3 Karen Noel 2015-08-17 10:44:55 EDT
> When using '-cpu host,+invtsc', migration works well.

I thought any use of invtsc should disallow live migration. Marcelo?
Comment 4 Eduardo Habkost 2015-08-17 12:43:34 EDT
This is expected. We must block migration if invtsc is enabled, and it would be a bug if we didn't block it.
Comment 5 Pei Zhang 2015-08-17 20:24:20 EDT
(In reply to Eduardo Habkost from comment #4)
> This is expected. We must block migration if invtsc is enabled, and it would
> be a bug if we didn't block it.

According to Comment 4, using '-cpu host,+invtsc' should also disallow live migration, but it it allowed. Only using'-cpu $cpu-model,+invtsc' can not migrate.
Comment 6 Eduardo Habkost 2015-08-19 12:10:47 EDT
(In reply to zhangpei from comment #5)
> (In reply to Eduardo Habkost from comment #4)
> > This is expected. We must block migration if invtsc is enabled, and it would
> > be a bug if we didn't block it.
> 
> According to Comment 4, using '-cpu host,+invtsc' should also disallow live
> migration, but it it allowed. Only using'-cpu $cpu-model,+invtsc' can not
> migrate.

You should always test using either "enforce" or "check" flags ("enforce" is preferred). invtsc is being silently disabled because "-cpu host" runs with migratable=on by default, and it prevents unmigratable features from being enabled. With -cpu host, you need "-cpu host,migratable=off,+invtsc,enforce". I know it is confusing, but we had to do it to make sure "-cpu host" is a migratable CPU model by default (as long as host CPU, host QEMU version and host kernel stay the same).
Comment 7 Pei Zhang 2015-08-20 07:13:43 EDT
(In reply to Eduardo Habkost from comment #6)
> (In reply to zhangpei from comment #5)
> > (In reply to Eduardo Habkost from comment #4)
> > > This is expected. We must block migration if invtsc is enabled, and it would
> > > be a bug if we didn't block it.
> > 
> > According to Comment 4, using '-cpu host,+invtsc' should also disallow live
> > migration, but it it allowed. Only using'-cpu $cpu-model,+invtsc' can not
> > migrate.
> 
> You should always test using either "enforce" or "check" flags ("enforce" is
> preferred). invtsc is being silently disabled because "-cpu host" runs with
> migratable=on by default, and it prevents unmigratable features from being
> enabled. With -cpu host, you need "-cpu
> host,migratable=off,+invtsc,enforce". I know it is confusing, but we had to
> do it to make sure "-cpu host" is a migratable CPU model by default (as long
> as host CPU, host QEMU version and host kernel stay the same).

Hi Eduardo,
Thanks for your explain. I tried according to Comments 6 and now understand this cpu flag well.

# /usr/libexec/qemu-kvm -cpu host,+invtsc,enforce -monitor stdio
QEMU 2.3.0 monitor - type 'help' for more information
(qemu) warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 0]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 1]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 2]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 3]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 4]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 5]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 6]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 7]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 8]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 9]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 12]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 13]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 14]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 15]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 16]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 17]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 23]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 24]
warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]
qemu-kvm: Host doesn't support requested features


# /usr/libexec/qemu-kvm -cpu host,migratable=off,+invtsc,enforce -monitor stdio
QEMU 2.3.0 monitor - type 'help' for more information
(qemu) VNC server running on `::1:5900'
(qemu) migrate -d tcp:10.66.12.153:6666
migrate: State blocked by non-migratable device 'cpu'

Note You need to log in before you can comment on or make changes to this bug.