This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours
Bug 892101 - Guest consumes 100% cpu after migration with xbzrle enabled (migrate during installation)
Guest consumes 100% cpu after migration with xbzrle enabled (migrate during i...
Status: CLOSED DUPLICATE of bug 1063417
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
7.0
Unspecified Unspecified
high Severity high
: rc
: ---
Assigned To: Dr. David Alan Gilbert
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-01-05 03:30 EST by Qunfang Zhang
Modified: 2014-03-19 07:06 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-03-19 07:06:17 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
ftrace log when the guest consume 100% cpu after migration (951.68 KB, application/x-tar)
2013-01-07 04:24 EST, Qunfang Zhang
no flags Details

  None (edit)
Description Qunfang Zhang 2013-01-05 03:30:17 EST
Description of problem:
Migrate guest during installation(at the installing packages stage), xbzrle is enabled. After guest finish installation, qemu-kvm process consumes 100% cpu and the installation can not proceed.

Version-Release number of selected component (if applicable):
Host install tree: RHEL-7.0-20121217.0
kernel-3.6.0-0.29.el7.x86_64
qemu-kvm-1.2.0-21.el7.x86_64

Guest: 
RHEL6.4-20121212.1

How reproducible:
4/5

Steps to Reproduce:
1. Boot guest on source host:
#  /usr/libexec/qemu-kvm -cpu SandyBridge -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -enable-kvm -name t2-rhel6.4-32 -uuid 61b6c504-5a8b-4fe1-8347-6c929b750dde -k en-us -rtc base=localtime,clock=host,driftfix=slew -no-kvm-pit-reinjection -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-tablet,id=input0 -drive file=/root/installation.qcow2,if=none,id=disk0,format=qcow2,werror=stop,rerror=stop,aio=native -device ide-drive,bus=ide.0,unit=1,drive=disk0,id=disk0  -drive file=/root/boot.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,drive=drive-ide0-1-0,bus=ide.1,unit=0,id=cdrom -netdev tap,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=44:37:E6:5E:91:85,bus=pci.0,addr=0x5 -monitor stdio -qmp tcp:0:6666,server,nowait -chardev socket,path=/tmp/isa-serial,server,nowait,id=isa1 -device isa-serial,chardev=isa1,id=isa-serial1 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x8 -chardev socket,id=charchannel0,path=/tmp/serial-socket,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,path=/tmp/foo,server,nowait,id=foo -device virtconsole,chardev=foo,id=console0  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x9 -vnc :10 -k en-us -boot dc -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga0 -device virtserialport,bus=virtio-serial0.0,chardev=qga0,name=org.qemu.guest_agent.0  -global  PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0


2. Boot guest with listening mode on dst host "-incoming tcp:0:5800"

3. On both source and destination host:
(qemu)migrate_set_capability xbzrle on

4. On source host:
(qemu) migrate_set_cache_size 2G

5. Install guest:
Boot from the boot.iso and install from URL, use all the default configuration to install.

6. At last, during guest is installing the packages, start migration.
(qemu) migrate -d tcp:t2:5800
  
Actual results:
Guest consumes 100% cpu after migration. The installation process can not continue. 

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                              
10613 root      20   0 2511m 1.2g 5876 S 100.1 16.9   2:49.80 qemu-kvm    

Expected results:
Guest works well after migration and can finish the installation.

Additional info:
(1) disable xbzrle and re-test, can not reproduce (migrate during installing packages stage too.)

(2) (qemu) migrate_set_cache_size 2G ==> easy to reproduce
(qemu) migrate_set_cache_size 1G     ==> can not reproduce after twice attempts.

Host cpu info: (src and dst are the same hosts)

4 cpus and 8G memory.

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
stepping	: 7
microcode	: 0x28
cpu MHz		: 1600.000
cache size	: 6144 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 6186.50
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:


[root@localhost ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          7556       1860       5695          0          4       1658
-/+ buffers/cache:        197       7358
Swap:         7823          0       7823
Comment 2 Orit Wasserman 2013-01-07 03:42:47 EST
Can you run strace/ftrace on the dest QEMU?
Comment 3 Qunfang Zhang 2013-01-07 04:24:29 EST
Created attachment 673836 [details]
ftrace log when the guest consume 100% cpu after migration
Comment 4 Qunfang Zhang 2013-01-07 04:42:52 EST
Also tested qemu-kvm-1.3.0-3.el7.x86_64, still can reproduce.
Comment 5 CongLi 2013-02-21 05:31:02 EST
The same problem during installation also happened in checking disk & fomatting disk when do the guest migration, with xbzrle is enabled. 
After guest finish installation, qemu-kvm process consumes 100% cpu and the installation can not proceed.
Comment 6 Orit Wasserman 2013-12-17 08:59:39 EST
Can you try to reproduce with the latest rhel7 qemu version (1.5.3)?
Comment 7 Qunfang Zhang 2013-12-19 01:34:18 EST
(In reply to Orit Wasserman from comment #6)
> Can you try to reproduce with the latest rhel7 qemu version (1.5.3)?

Hi, Orit

Re-test on the following version:
Host:
kernel-3.10.0-63.el7.x86_64
qemu-kvm-1.5.3-24.el7.x86_64

The issue still could be reproduced but need to try lots of times. 
(1) Tried a rhel7.0 guest for 6 times: 5 times passed and reproduce once.
(2) Tried a rhel6.5 guest for about 20 times:
i) reproduced this bug once.
ii) hit bug 1044853 for about 2~3 times
iii) passed in other attempts.

Thanks,
Qunfang
Comment 11 Dr. David Alan Gilbert 2014-03-05 06:52:17 EST
Please test with qemu-kvm-1.5.3-51.el7; if it passes then please dupe to 
bz 1063417
Comment 12 juzhang 2014-03-18 23:57:47 EDT
(In reply to Dr. David Alan Gilbert from comment #11)
> Please test with qemu-kvm-1.5.3-51.el7; if it passes then please dupe to 
> bz 1063417

Hi Qzhang,

Could you have a try and update the testing result?

Best Regards,
Junyi
Comment 13 Qunfang Zhang 2014-03-19 07:06:17 EDT
Verified this bug with qemu-kvm-1.5.3-53.el7, do migration during guest installation for more than 20 times, could not reproduce the original issue.  The steps are the same as comment 0.  So I will close it according to comment 11.

*** This bug has been marked as a duplicate of bug 1063417 ***

Note You need to log in before you can comment on or make changes to this bug.