Bug 1254124

Summary:	-cpu $cpu-model,+invtsc doesn't support migration
Product:	Red Hat Enterprise Linux 7	Reporter:	Pei Zhang <pezhang>
Component:	qemu-kvm-rhev	Assignee:	Marcelo Tosatti <mtosatti>
Status:	CLOSED NOTABUG	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	7.2	CC:	chayang, ehabkost, juzhang, knoel, michen, mtosatti, qzhang, virt-maint, xfu
Target Milestone:	rc
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-08-17 16:43:34 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Pei Zhang 2015-08-17 08:37:44 UTC

Description of problem:
'-cpu $cpu-model,+invtsc' doesn't support migration. 

Version-Release number of selected component (if applicable):
Host:
Kernel:3.10.0-304.el7.x86_64
qemu-kvm-rhev:qemu-kvm-rhev-2.3.0-18.el7.x86_64
/proc/cpuinfo:
processor	: 31
vendor_id	: AuthenticAMD
cpu family	: 21
model		: 2
model name	: AMD Opteron(tm) Processor 6376                 
stepping	: 0
microcode	: 0x6000832
cpu MHz		: 2300.000
cache size	: 2048 KB
physical id	: 1
siblings	: 16
core id		: 7
cpu cores	: 8
apicid		: 79
initial apicid	: 47
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips	: 4587.94
TLB size	: 1536 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro


How reproducible:
100%

Steps to Reproduce:
1.
# /usr/libexec/qemu-kvm -cpu Opteron_G5,+invtsc -monitor stdio

2.
# /usr/libexec/qemu-kvm -cpu Opteron_G5,+invtsc -monitor stdio -incoming tcp:0:6666 

3.
(qemu) migrate -d tcp:10.66.9.120:6666
migrate: State blocked by non-migratable device 'cpu'


Actual results:
migrate: State blocked by non-migratable device 'cpu'

Expected results:
Should migrate successfully.

Additional info:
When using '-cpu host,+invtsc', migration works well. And Using '-cpu Opteron_G5,fpu' also works well.

Comment 1 FuXiangChun 2015-08-17 08:56:04 UTC

Eduardo,

Is it a valid problem?

Comment 3 Karen Noel 2015-08-17 14:44:55 UTC

> When using '-cpu host,+invtsc', migration works well.

I thought any use of invtsc should disallow live migration. Marcelo?

Comment 4 Eduardo Habkost 2015-08-17 16:43:34 UTC

This is expected. We must block migration if invtsc is enabled, and it would be a bug if we didn't block it.

Comment 5 Pei Zhang 2015-08-18 00:24:20 UTC

(In reply to Eduardo Habkost from comment #4)
> This is expected. We must block migration if invtsc is enabled, and it would
> be a bug if we didn't block it.

According to Comment 4, using '-cpu host,+invtsc' should also disallow live migration, but it it allowed. Only using'-cpu $cpu-model,+invtsc' can not migrate.

Comment 6 Eduardo Habkost 2015-08-19 16:10:47 UTC

(In reply to zhangpei from comment #5)
> (In reply to Eduardo Habkost from comment #4)
> > This is expected. We must block migration if invtsc is enabled, and it would
> > be a bug if we didn't block it.
> 
> According to Comment 4, using '-cpu host,+invtsc' should also disallow live
> migration, but it it allowed. Only using'-cpu $cpu-model,+invtsc' can not
> migrate.

You should always test using either "enforce" or "check" flags ("enforce" is preferred). invtsc is being silently disabled because "-cpu host" runs with migratable=on by default, and it prevents unmigratable features from being enabled. With -cpu host, you need "-cpu host,migratable=off,+invtsc,enforce". I know it is confusing, but we had to do it to make sure "-cpu host" is a migratable CPU model by default (as long as host CPU, host QEMU version and host kernel stay the same).

Comment 7 Pei Zhang 2015-08-20 11:13:43 UTC

(In reply to Eduardo Habkost from comment #6)
> (In reply to zhangpei from comment #5)
> > (In reply to Eduardo Habkost from comment #4)
> > > This is expected. We must block migration if invtsc is enabled, and it would
> > > be a bug if we didn't block it.
> > 
> > According to Comment 4, using '-cpu host,+invtsc' should also disallow live
> > migration, but it it allowed. Only using'-cpu $cpu-model,+invtsc' can not
> > migrate.
> 
> You should always test using either "enforce" or "check" flags ("enforce" is
> preferred). invtsc is being silently disabled because "-cpu host" runs with
> migratable=on by default, and it prevents unmigratable features from being
> enabled. With -cpu host, you need "-cpu
> host,migratable=off,+invtsc,enforce". I know it is confusing, but we had to
> do it to make sure "-cpu host" is a migratable CPU model by default (as long
> as host CPU, host QEMU version and host kernel stay the same).

Hi Eduardo,
Thanks for your explain. I tried according to Comments 6 and now understand this cpu flag well.

# /usr/libexec/qemu-kvm -cpu host,+invtsc,enforce -monitor stdio
QEMU 2.3.0 monitor - type 'help' for more information
(qemu) warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 0]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 1]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 2]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 3]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 4]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 5]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 6]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 7]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 8]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 9]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 12]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 13]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 14]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 15]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 16]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 17]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 23]
warning: host doesn't support requested feature: CPUID.80000001H:EDX [bit 24]
warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]
qemu-kvm: Host doesn't support requested features


# /usr/libexec/qemu-kvm -cpu host,migratable=off,+invtsc,enforce -monitor stdio
QEMU 2.3.0 monitor - type 'help' for more information
(qemu) VNC server running on `::1:5900'
(qemu) migrate -d tcp:10.66.12.153:6666
migrate: State blocked by non-migratable device 'cpu'