RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 921465 - Migration can not finished even the "remaining ram" is already 0 kb
Summary: Migration can not finished even the "remaining ram" is already 0 kb
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm
Version: 7.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Juan Quintela
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-03-14 09:30 UTC by Qunfang Zhang
Modified: 2014-06-18 03:23 UTC (History)
10 users (show)

Fixed In Version: qemu-kvm-1.5.3-10.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-06-13 10:14:53 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Qunfang Zhang 2013-03-14 09:30:59 UTC 
Description of problem:
This is a regression of qemu-1.4 and does not exist in qemu-1.3. Migrate an idle guest or load a little and then do live migration with default migration speed and downtime. But migration can not finish even the "remain ram" is 0. Please check the following info:

(qemu) info migrate
capabilities: xbzrle: off 
Migration status: active
total time: 77536 milliseconds
expected downtime: 0 milliseconds
transferred ram: 2524940 kbytes
remaining ram: 0 kbytes
total ram: 4211072 kbytes
duplicate: 1588975 pages
normal: 1187467 pages
normal bytes: 4749868 kbytes
dirty pages rate: 6974 pages
(qemu) 
(qemu)info migrate
capabilities: xbzrle: off 
Migration status: active
total time: 77905 milliseconds
expected downtime: 0 milliseconds
transferred ram: 2572072 kbytes
remaining ram: 0 kbytes
total ram: 4211072 kbytes
duplicate: 1589055 pages
normal: 1199227 pages
normal bytes: 4796908 kbytes
dirty pages rate: 9300 pages


Version-Release number of selected component (if applicable):
kenrel-3.8.0-0.40.el7.x86_64
qemu-kvm-1.4.0-1.el7.x86_6

How reproducible:
Always

Steps to Reproduce:
1. Boot a guest on host
/usr/libexec/qemu-kvm -M pc-i440fx-1.4 -enable-kvm -m 4096 -smp 2,sockets=2,cores=1,threads=1 -name rhel7.0 -uuid 9a0e67ec-f286-d8e7-0548-0c1c9ec93009 -nodefconfig -nodefaults -monitor stdio -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 -drive file=/opt/rhel7.0-64-virtio.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/opt/boot.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:d5:51:8a,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0  -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-tablet,id=input0 -vnc :10 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

2. Boot the guest with listening mode on dst host.

3. Run some application but do not load too much inside guest. 
(1)eg:  open firefox and open a website with dynamic content.
or 
(2) eg: #while true; do dd if=/dev/zero of=file bs=1M count=100; rm -rf file; done  ("top" inside guest shows the cpu and mem usage is < 10% for the dd operation.)

4. Migrate guest to dst host
(qemu) migrate -d tcp:$dst_host_ip:5800

5. Keep checking the migration status
(qemu) info migrate

  
Actual results:
Migration can not finish even the "remain ram" is already 0.

Expected results:
Migration could finish

Additional info:

(1) I downgrade the qemu-kvm to qemu-kvm-1.3.0-8, and re-test with same steps on same hosts, migration could finish successfully.

(2) Host info

#free -m 
     total       used       free     shared    buffers     cached
Mem:          7556       2259       5296          0          1        729
-/+ buffers/cache:       1529       6026
Swap:         7823          0       7823


#top
(here only list the last cpu info)

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
stepping	: 7
microcode	: 0x28
cpu MHz		: 1600.000
cache size	: 6144 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 6185.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:
Comment 3 Orit Wasserman 2013-10-07 07:14:51 UTC 
Can you check with the latest rhel7 qemu version (1.5)?
Comment 4 Jun Li 2013-10-08 04:50:24 UTC 
(In reply to Orit Wasserman from comment #3)
> Can you check with the latest rhel7 qemu version (1.5)?

verify this issue with qemu-kvm-1.5.3-2.el7 and test case as following.
https://tcms.engineering.redhat.com/case/189296/?from_plan=6435

since this test case is pass status. so this bug is fixed.
Comment 5 Miroslav Rezanina 2013-10-16 11:52:12 UTC 
There's patches for this bz on the queue.
Comment 6 Miroslav Rezanina 2013-10-17 10:01:39 UTC 
Fix included in qemu-kvm-1.5.3-10.el7
Comment 8 Qian Guo 2013-10-31 09:41:48 UTC 
Reproduced this bug with component : qemu-kvm-1.4.0-1.el7.x86_64

Steps:

1.Boot guest as src:
# /usr/libexec/qemu-kvm -M pc-i440fx-1.4 -enable-kvm -m 4096 -smp 2,sockets=2,cores=1,threads=1 -name rhel7.0 -uuid 9a0e67ec-f286-d8e7-0548-0c1c9ec93009 -nodefconfig -nodefaults -monitor stdio -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 -drive file=/home/rhel7base.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -device usb-tablet,id=input0 -vnc :10 -vga std

2.Inside guest, do following script:
#while true; do dd if=/dev/zero of=file bs=1M count=100; rm -rf file; done

3.Launch dst listening mode qemuL:

#/usr/libexec/qemu-kvm -M pc-i440fx-1.4 -enable-kvm -m 4096 -smp 2,sockets=2,cores=1,threads=1 -name rhel7.0 -uuid 9a0e67ec-f286-d8e7-0548-0c1c9ec93009 -nodefconfig -nodefaults -monitor stdio -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 -drive file=/home/rhel7base.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -device usb-tablet,id=input0 -vnc :20 -vga std -incoming tcp:0:4000

4.Migrate guest form src to dst:
(qemu) migrate -d tcp:$dst_ip:4000

5.Monitor the status of migration:


In src host:
(qemu) info migrate
capabilities: xbzrle: off 
Migration status: active
total time: 533446 milliseconds
expected downtime: 0 milliseconds
transferred ram: 17469171 kbytes
remaining ram: 0 kbytes
total ram: 4211008 kbytes
duplicate: 12791329 pages
normal: 4330726 pages
normal bytes: 17322904 kbytes
dirty pages rate: 32198 pages


In dst host:
(qemu) info status
VM status: paused (inmigrate)

Result: migration can not finished even the remaining ram is "0".
So, according above, this bug is reproduced.


Verify this bug with component: qemu-kvm-1.5.3-10.el7.x86_64

Steps as above
Monitor the status of migration:

In src host
(qemu) info migrate
capabilities: xbzrle: off auto-converge: off zero-blocks: off 
Migration status: completed
total time: 374794 milliseconds
downtime: 923 milliseconds
transferred ram: 12208164 kbytes
remaining ram: 0 kbytes
total ram: 4211008 kbytes
duplicate: 12462410 pages
skipped: 0 pages
normal: 3018754 pages
normal bytes: 12075016 kbytes

In dst host:
(qemu) info status
VM status: running

So, according above, this bug is fixed by qemu-kvm-1.5.3-10.el7.x86_64
Comment 10 Ludek Smid 2014-06-13 10:14:53 UTC 
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.
Note You need to log in before you can comment on or make changes to this bug.