Bug 1363998 - Live migration via a compressed file causes the guest desktop to freeze
Summary: Live migration via a compressed file causes the guest desktop to freeze
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Dr. David Alan Gilbert
QA Contact: Qianqian Zhu
URL:
Whiteboard:
: 1363929 1369687 (view as bug list)
Depends On: 1367716
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-04 08:49 UTC by HongyangDai
Modified: 2018-05-13 23:40 UTC (History)
8 users (show)

Fixed In Version: qemu-kvm-rhev-2.6.0-26.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-07 21:28:41 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1577680 None None None 2019-05-07 06:29:13 UTC
Red Hat Product Errata RHBA-2016:2673 normal SHIPPED_LIVE qemu-kvm-rhev bug fix and enhancement update 2016-11-08 01:06:13 UTC

Internal Links: 1577680

Description HongyangDai 2016-08-04 08:49:41 UTC
Description of problem:
I just try to migrate a windows guest via a compressed file, however the guest desktop freezes after it is migrated to the destination host.

Version-Release number of selected component (if applicable):
kernel-3.10.0-481.el7.x86_64
qemu-kvm-rhev-2.6.0-16.el7.x86_64


How reproducible:
4/4

Steps to Reproduce:
1. Boot a windows guest on src host .

/usr/libexec/qemu-kvm  -name win -cpu Westmere,check -m 2048 -realtime mlock=off -smp 8,sockets=4,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot order=c,menu=on -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/mnt/nfs/win10x64.iso,if=none,media=cdrom,id=drive-virtio-win0,readonly=on,format=raw -device ide-drive,drive=drive-virtio-win0,id=virtio-win0 -drive file=/mnt/nfs/virtio-win.iso,if=none,media=cdrom,id=drive-virtio-win1,readonly=on,format=raw -device ide-drive,drive=drive-virtio-win1,id=virtio-win1 -drive file=/mnt/nfs/winsys.qcow2,if=none,id=drive-virtio-blk0,format=qcow2,werror=stop,rerror=stop -device virtio-scsi-pci,id=virtio-blk0,disable-legacy=true,disable-modern=false -device scsi-disk,drive=drive-virtio-blk0,scsi-id=0,lun=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=3C:D9:2B:09:AB:44,bus=pci.0,addr=0x7 -qmp tcp:0:5555,server,nowait  -boot order=d

2.Save VM state into a compressed file in host
{"execute":"stop"}
{ "execute": "migrate_set_speed", "arguments": { "value": 104857600 } }
{"execute": "migrate","arguments":{"uri": "exec:gzip -c > /mnt/nfs/STATEFILE.gz"}}

3.3. Load the file in dest host.
<qemu-command-line> -incoming "exec: gzip -c -d /mnt/nfsSTATEFILE.gz"

Actual results:
After migration, guest desktop freezes.The screen will not change any more. And the mouse point could not move.

Expected results:
Guest desktop should not freeze. 


Additional info:
None

Comment 1 HongyangDai 2016-08-04 09:38:37 UTC
After step3  ,I execute the command "cont" in the (qemu). Then the guest still hangs.

Comment 5 HongyangDai 2016-08-09 07:23:54 UTC
ok,I will answer your questions in the following.

1.Yes,I try the same steps with RHEL7.3 guests and win2012. I hit another issue.

In the des guest`s hmp,it shows :

"(qemu) 2016-08-09T03:12:14.113798Z qemu-kvm: Unknown savevm section or instance 'cpu_common' 4
2016-08-09T03:12:14.113845Z qemu-kvm: warning: TSC frequency mismatch between VM (3392289 kHz) and host (3392292 kHz), and TSC scaling unavailable
2016-08-09T03:12:14.113919Z qemu-kvm: warning: TSC frequency mismatch between VM (3392289 kHz) and host (3392292 kHz), and TSC scaling unavailable
2016-08-09T03:12:14.113988Z qemu-kvm: warning: TSC frequency mismatch between VM (3392289 kHz) and host (3392292 kHz), and TSC scaling unavailable
2016-08-09T03:12:14.114050Z qemu-kvm: warning: TSC frequency mismatch between VM (3392289 kHz) and host (3392292 kHz), and TSC scaling unavailable

gzip: stdout: Broken pipe
2016-08-09T03:12:14.114253Z qemu-kvm: load of migration failed: Invalid argument"



2.I reproduced this case with win10 and win2012 separately.

Boot a win2012 guest on src host . 

/usr/libexec/qemu-kvm  -name win -cpu Westmere,check -m 2048 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot order=c,menu=on -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6   -drive file=/mnt/nfs/win2012-64r2-virtio-scsi.qcow2,if=none,id=drive-virtio-blk0,format=qcow2,werror=stop,rerror=stop -device virtio-scsi-pci,id=virtio-blk0 -device scsi-disk,drive=drive-virtio-blk0,scsi-id=0,lun=0  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=3C:D9:2B:09:AB:44,bus=pci.0,addr=0x7 -qmp tcp:0:5555,server,nowait  -boot order=d


The other steps are same.


Actual results:
In the des guest`s hmp,it shows 

"(qemu) cot2016-08-09T06:08:27.052884Z qemu-kvm: Unknown savevm section or instance 'cpu_common' 4
2016-08-09T06:08:27.052926Z qemu-kvm: warning: TSC frequency mismatch between VM (3392289 kHz) and host (3392292 kHz), and TSC scaling unavailable
2016-08-09T06:08:27.053002Z qemu-kvm: warning: TSC frequency mismatch between VM (3392289 kHz) and host (3392292 kHz), and TSC scaling unavailable
2016-08-09T06:08:27.053061Z qemu-kvm: warning: TSC frequency mismatch between VM (3392289 kHz) and host (3392292 kHz), and TSC scaling unavailable
2016-08-09T06:08:27.053114Z qemu-kvm: warning: TSC frequency mismatch between VM (3392289 kHz) and host (3392292 kHz), and TSC scaling unavailable

gzip: stdout: Broken pipe
2016-08-09T06:08:27.053334Z qemu-kvm: load of migration failed: Invalid argument"

Then, Boot a win10 guest on src host ,and the  step are same with the win2012.
It appears the same result.


3.Yes  ,I run this case ,using Win10 guest with -cpu Westmere on the source  host and the destination .

Comment 6 Dr. David Alan Gilbert 2016-08-09 13:43:57 UTC
Hi Hongyang,
  Can you give me the full command line pair that you used for the RHEL7.3 test case please.

Dave

Comment 7 HongyangDai 2016-08-10 03:03:38 UTC
ok,the command line as the following :

/usr/libexec/qemu-kvm  -name win -cpu Westmere,check -m 2048 -realtime mlock=off -smp 8,sockets=4,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot order=c,menu=on -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/mnt/nfs/win10x64.iso,if=none,media=cdrom,id=drive-virtio-win0,readonly=on,format=raw -device ide-drive,drive=drive-virtio-win0,id=virtio-win0 -drive file=/mnt/nfs/virtio-win.iso,if=none,media=cdrom,id=drive-virtio-win1,readonly=on,format=raw -device ide-drive,drive=drive-virtio-win1,id=virtio-win1 -drive file=/mnt/nfs/RHEL-Server-7.3-64-virtio-scsi.qcow2,if=none,id=drive-virtio-blk0,format=qcow2,werror=stop,rerror=stop -device virtio-scsi-pci,id=virtio-blk0,disable-legacy=true,disable-modern=false -device scsi-disk,drive=drive-virtio-blk0,scsi-id=0,lun=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=3C:D9:2B:09:AB:44,bus=pci.0,addr=0x7 -qmp tcp:0:5555,server,nowait  -boot order=d

Comment 8 HongyangDai 2016-08-10 03:10:24 UTC
and the  command line in the des guest:

/usr/libexec/qemu-kvm  -name win -cpu Westmere,check -m 2048 -realtime mlock=off -smp 8,sockets=4,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot order=c,menu=on -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/mnt/nfs/win10x64.iso,if=none,media=cdrom,id=drive-virtio-win0,readonly=on,format=raw -device ide-drive,drive=drive-virtio-win0,id=virtio-win0 -drive file=/mnt/nfs/virtio-win.iso,if=none,media=cdrom,id=drive-virtio-win1,readonly=on,format=raw -device ide-drive,drive=drive-virtio-win1,id=virtio-win1 -drive file=/mnt/nfs/RHEL-Server-7.3-64-virtio-scsi.qcow2,if=none,id=drive-virtio-blk0,format=qcow2,werror=stop,rerror=stop -device virtio-scsi-pci,id=virtio-blk0,disable-legacy=true,disable-modern=false -device scsi-disk,drive=drive-virtio-blk0,scsi-id=0,lun=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=3C:D9:2B:09:AB:44,bus=pci.0,addr=0x7 -qmp tcp:0:5555,server,nowait  -boot order=d  -incoming "exec: gzip -c -d /mnt/nfs/STATEFILE.gz"

Comment 9 Dr. David Alan Gilbert 2016-08-11 17:47:46 UTC
I can't reproduce the problem here with a rhel guest - it all seems to work fine.
I've spent a few hours trying different options and it's all happy.

One thing I don't understand about the line that you pasted is that you have a 
-boot order=c,menu=on  AND a -boot order=d   at the end; doesn't that mean it boots off the CD rather the image?

I'm using the line:
/usr/libexec/qemu-kvm -cpu Westmere,check -m 2048 -realtime mlock=off -smp 8,sockets=4,cores=2,threads=1 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot order=c,menu=on -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/home/vms/Fedora-Live-Xfce-i686-20-1.iso,if=none,media=cdrom,id=drive-virtio-iso0,readonly=on,format=raw -device ide-drive,drive=drive-virtio-iso0,id=virtio-iso0 -drive file=/home/vms/Fedora-Live-Xfce-i686-20-1.iso,if=none,media=cdrom,id=drive-virtio-iso1,readonly=on,format=raw -device ide-drive,drive=drive-virtio-iso1,id=virtio-iso1 -M pc,accel=kvm -drive file=/home/vms/7.2a.qcow2,cache=none,format=qcow2,if=none,id=drive-disk0,werror=stop,rerror=stop -device virtio-scsi-pci,id=scsi0,disable-legacy=true,disable-modern=false -device scsi-hd,id=disk0,bus=scsi0.0,scsi-id=0,lun=0,drive=drive-disk0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4 -netdev user,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=3C:D9:2B:09:AB:44,bus=pci.0,addr=0x7

I'm using qemu-kvm-rhev-2.6.0-16.

Please grab me on irc and see if we can get it to reproduce.

Comment 10 HongyangDai 2016-08-16 08:43:29 UTC
I try it with windows10,windows2012 and rhel7.3 again, but I don`t hit the issue on comment2:"gzip: stdout: Broken pipe".

But I was able to reproduce this freeze issue with win10 and win2012 , while RHEL7.3 works well after migration. So it should be windows only issue.

Could you please try it with windows guest?

Here is my command line:
/usr/libexec/qemu-kvm -name win -cpu Westmere,check -m 2048 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew  -drive file=/mnt/nfs/win10-64-virtio-scsi.qcow2,if=none,id=drive-virtio-blk0,format=qcow2,werror=stop,rerror=stop,cache=none -device virtio-scsi-pci,id=virtio-blk0 -device scsi-disk,drive=drive-virtio-blk0,bootindex=0,scsi-id=0,lun=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=3C:D9:2B:09:AB:44,bus=pci.0,addr=0x7 -qmp tcp:0:5555,server,nowait

Comment 11 Dr. David Alan Gilbert 2016-08-16 10:07:16 UTC
(In reply to HongyangDai from comment #10)
> I try it with windows10,windows2012 and rhel7.3 again, but I don`t hit the
> issue on comment2:"gzip: stdout: Broken pipe".
> 
> But I was able to reproduce this freeze issue with win10 and win2012 , while
> RHEL7.3 works well after migration. So it should be windows only issue.
> 
> Could you please try it with windows guest?
> 
> Here is my command line:
> /usr/libexec/qemu-kvm -name win -cpu Westmere,check -m 2048 -realtime
> mlock=off -smp 4,sockets=2,cores=2,threads=1 -uuid
> 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc
> base=localtime,driftfix=slew  -drive
> file=/mnt/nfs/win10-64-virtio-scsi.qcow2,if=none,id=drive-virtio-blk0,
> format=qcow2,werror=stop,rerror=stop,cache=none -device
> virtio-scsi-pci,id=virtio-blk0 -device
> scsi-disk,drive=drive-virtio-blk0,bootindex=0,scsi-id=0,lun=0 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on -spice
> port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432
> -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4 -netdev
> tap,id=hostnet0,vhost=on -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=3C:D9:2B:09:AB:44,bus=pci.0,
> addr=0x7 -qmp tcp:0:5555,server,nowait

Interesting.
I ran this test 6 times at the windows installer, at the screen where it asks for the license key.
I had it fail 2 times and pass 4 - so for me it doesn't always fail, but I have had two failures, so confirming.
My test is:

/usr/libexec/qemu-kvm -name win -cpu Westmere,check -m 2048 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew -drive file=/home/vms/win2k12r2.qcow2,if=none,id=drive-virtio-blk0,format=qcow2,werror=stop,rerror=stop,cache=none -device virtio-scsi-pci,id=virtio-blk0 -device scsi-disk,drive=drive-virtio-blk0,bootindex=0,scsi-id=0,lun=0 -msg timestamp=on -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4 -netdev user,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=3C:D9:2B:09:AB:44,bus=pci.0,addr=0x7 -cdrom /home/vms/isos/en_windows_server_2012_r2_with_update_x64_dvd_6052708.iso

then select the keyboard type, click 'install now' and wait for the box
to come up where it asks for the license key.

migrate_set_speed 100M
migrate "exec: gzip -c > /home/vms/migfile.gz"

and then on the destination the command line same as above with:
migrate "exec: gzip -c -d < /home/vms/migfile.gz"

Comment 12 Dr. David Alan Gilbert 2016-08-17 19:08:06 UTC
and it seems to work fine both on a  2.6.0 upstream and a current 2.7.0-rc3; so it looks like something we've added/enabled.
My current commandline is:

/usr/libexec/qemu-kvm -name win -cpu Westmere,check -M pc-i440fx-rhel7.2.0,accel=kvm -m 2048 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew -drive file=/home/vms/win2k12r2.qcow2,if=none,id=drive-virtio-blk0,format=qcow2,werror=stop,rerror=stop,cache=none -device virtio-scsi-pci,id=virtio-blk0 -device scsi-disk,drive=drive-virtio-blk0,bootindex=0,scsi-id=0,lun=0 -msg timestamp=on -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4  -cdrom /home/vms/isos/en_windows_server_2012_r2_with_update_x64_dvd_6052708.iso -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1

Comment 13 Dr. David Alan Gilbert 2016-08-18 16:20:28 UTC
It seems broken for me on qemu-kvm-rhev-2.6.0-1.el7 but work fine on 2.6.0 upstream;  I've got it down to caaa2b731cf73681fe49 which looks OK.
My current command line is:

./x86_64-softmmu/qemu-system-x86_64 -name pp44473-4 -cpu Westmere,check -M pc-i440fx-2.6,accel=kvm -m 2048 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew -msg timestamp=on -vnc :0 -vga cirrus -cdrom /home/vms/isos/en_windows_server_2012_r2_with_update_x64_dvd_6052708.iso  -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 

so no virtio, no disks (other than the ISO).

Comment 14 Dr. David Alan Gilbert 2016-08-18 17:40:43 UTC
Hmm.
Somewhere around there we turn hpet off; and turning hpet off breaks this migration on both 2.6.0 and 2.7.0 - so the real culprit here is whatever happens
when hpet is off.

Comment 15 HongyangDai 2016-08-19 04:01:32 UTC
Live migration via unix protocol causes the guest desktop to freeze . 

Description of problem:
I just try to migrate a windows(win10) guest via unix protocol, however the guest desktop freezes after it is migrated to the destination host.

Version-Release number of selected component (if applicable):
kernel-3.10.0-481.el7.x86_64
qemu-kvm-rhev-2.6.0-21.el7.x86_64


How reproducible:
5/5

Steps to Reproduce:
1. Boot a windows guest on src host .

/usr/libexec/qemu-kvm -name win -cpu Westmere,check -m 2048 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew  -drive file=/mnt/nfs/win10-64.qcow2,if=none,id=drive-virtio-blk0,format=qcow2,werror=stop,rerror=stop,cache=none -device virtio-scsi-pci,id=virtio-blk0 -device scsi-disk,drive=drive-virtio-blk0,bootindex=0,scsi-id=0,lun=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x4 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=3C:D9:2B:09:AB:44,bus=pci.0,addr=0x7


2.start listenning port in the same host
 <commandLine> -incoming unix:/tmp/qzhang

3.do live migration
(qemu)migrate -d unix:/tmp/qzhang


Actual results:
After migration, guest desktop freezes.The screen will not change any more. And the mouse point could not move.

Comment 16 Dr. David Alan Gilbert 2016-08-19 08:44:36 UTC
Hongyang: That may well be a separate bug - different windows, different migration scheme; and I think we already have Windows10 migration bugs filed.

Comment 17 Dr. David Alan Gilbert 2016-08-23 15:00:11 UTC
Hi Hongyang,
  Can you retry your cases with the added qemu flag:
      -no-kvm-irqchip

for me it fixes the problem.

Dave

Comment 20 HongyangDai 2016-08-24 07:43:08 UTC
(In reply to Dr. David Alan Gilbert from comment #17)
> Hi Hongyang,
>   Can you retry your cases with the added qemu flag:
>       -no-kvm-irqchip
> 
> for me it fixes the problem.
> 
> Dave

Hi,Dave:
I retry this case with the added qemu flag : -no-kvm-irqchip.
And the guest works well after the migration is completed.

Comment 22 Qianqian Zhu 2016-08-24 08:18:28 UTC
*** Bug 1369687 has been marked as a duplicate of this bug. ***

Comment 24 HongyangDai 2016-08-25 10:04:47 UTC
 I reproduce this issue  on qemu-kvm-rhev-2.3.0-6.el7.x86_64  .The guest  desktop freezes  after migration finishes.

Comment 27 Dr. David Alan Gilbert 2016-09-08 17:20:32 UTC
prior to migration I see the rtc raising and lowering interrupts regularly (when the guest reads the ioport).
after migration I never see it lowering the interrupt - so either the interrupt isn't getting to the guest, or for some reason it's not getting back to the rtc code.

Comment 35 Karen Noel 2016-09-16 12:49:09 UTC
Upstream patch: http://marc.info/?l=qemu-devel&m=147376060707877&w=2

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

Load the LAPIC state during post_load (rather than when the CPU
starts).

This allows an interrupt to be delivered from the ioapic to
the lapic prior to cpu loading, in particular the RTC that starts
ticking as soon as we load it's state.

Partially fixes a case where Windows hangs after migration due to RTC interrupts
disappearing; it survived ~30 iterations of my test where as without
it we didn't get to 2.  Still something else to be found yet.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>

Comment 36 Paolo Bonzini 2016-09-16 12:50:24 UTC
Note to PM: this is just a recipe to make the bug more deterministic.  You can still reproduce it with normal migration, just only 1 in 3-5 times depending on bad luck.

Comment 37 Qianqian Zhu 2016-09-18 03:59:52 UTC
Reproduced with:
qemu-kvm-rhev-debuginfo-2.6.0-24.el7.1363998a.x86_64

Steps:
1. Launch Win2012 guest with:
/usr/libexec/qemu-kvm  -name win -cpu Westmere,check -m 2048 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1 -uuid 7bef3814-631a-48bb-bae8-2b1de75f7a13 -nodefaults -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot menu=on -device virtio-scsi-pci,id=scsi_pci_bus0 -drive file=/mntnfs/virtio-win-prewhql.iso,if=none,cache=none,media=cdrom,id=drive_wincd,readonly=on,format=raw  -device ide-cd,bus=ide.0,drive=drive_wincd,id=device_wincd  -drive file=/mntnfs/win2012-64r2-virtio-scsi.qcow2,format=qcow2,id=drive_sysdisk,if=none,cache=none,aio=native,werror=stop,rerror=stop  -device scsi-hd,drive=drive_sysdisk,bus=scsi_pci_bus0.0,id=device_sysdisk,bootindex=0 -monitor stdio -spice port=5800,disable-ticketing -vga qxl -qmp tcp:0:8888,server,nowait  -netdev tap,id=netdev0,vhost=on -device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pci.0

2. (qemu) migrate_set_speed 100M
(qemu) migrate -d "exec:gzip -c > /mntnfs/STATEFILE.gz"

3. Launch guest on destination host with -incoming "exec: gzip -c -d /mntnfs/STATEFILE.gz"

Result:
Guest freeze, mouse can move but no response for clicking.

Comment 38 Dr. David Alan Gilbert 2016-09-19 08:39:46 UTC
OK, now test with Paolo's new kernel (on source and destination) and see if it helps (with that same qemu)

https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=11755217

If it still breaks then please place the statefile.gz somewhere I can get to it
together with the exact wincd mimage

Comment 39 Qianqian Zhu 2016-09-20 02:00:53 UTC
(In reply to Dr. David Alan Gilbert from comment #38)
> OK, now test with Paolo's new kernel (on source and destination) and see if
> it helps (with that same qemu)
> 
> https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=11755217
> 
> If it still breaks then please place the statefile.gz somewhere I can get to
> it
> together with the exact wincd mimage

This build seems to be closed, I can't get the package, could you help post it again? Thanks.

Qianqian

Comment 41 Dr. David Alan Gilbert 2016-09-20 08:56:20 UTC
(In reply to qianqianzhu from comment #39)
> (In reply to Dr. David Alan Gilbert from comment #38)
> > OK, now test with Paolo's new kernel (on source and destination) and see if
> > it helps (with that same qemu)
> > 
> > https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=11755217
> > 
> > If it still breaks then please place the statefile.gz somewhere I can get to
> > it
> > together with the exact wincd mimage
> 
> This build seems to be closed, I can't get the package, could you help post
> it again? Thanks.
> 
> Qianqian

I've just built a new one (from the same srpm):
https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=11779565

also, please use the qemu from:
http://file.rdu.redhat.com/~dgilbert/for_qizhu/

Comment 42 Miroslav Rezanina 2016-09-20 12:29:18 UTC
Fix included in qemu-kvm-rhev-2.6.0-26.el7

Comment 44 Qianqian Zhu 2016-09-21 06:45:37 UTC
This issue has gone with kernel-3.10.0-506.el7.test5.x86_64 and qemu-kvm-rhev-2.6.0-25.el7.1363998c.x86_64.

But I can still reproduce it with qemu-kvm-rhev-2.6.0-26.el7.x86_64 plus kernel-3.10.0-506.el7.x86_64/kernel-3.10.0-509.el7.x86_64. So moving to ASSIGNED as FAIL QA.

Comment 45 Qianqian Zhu 2016-09-21 07:18:17 UTC
Guest works well with qemu-kvm-rhev-2.6.0-26.el7.x86_64 and kernel-3.10.0-506.el7.test5.x86_64, so qemu-kvm-rhev part should be fixed.
Moving to Verified.

Comment 46 juzhang 2016-09-21 07:24:57 UTC
I would like summarize QE's testing result according to comment44 and comment45.

1. Guest works well with fixed official build qemu-kvm-rhev-2.6.0-26.el7.x86_64 and Develop private build(3.10.0-506.el7.test5.x86_64) 

2. Still could reproduce this issue with fixed official build qemu-kvm-rhev-2.6.0-26.el7.x86_64 and latest official kernel build(kernel-3.10.0-509.el7.x86_64).  

In QE understanding, we need to fix both qemu-kvm-rhev and kernel component for this issue.

According to QE testing, qemu-kvm-rhev part is fixed. How about kernel part? Do we need to file new bz for tracking? or we have already existing bz for tracking? 

Best Regards,
Junyi

Comment 47 Dr. David Alan Gilbert 2016-09-21 08:01:42 UTC
(In reply to juzhang from comment #46)
> I would like summarize QE's testing result according to comment44 and
> comment45.
> 
> 1. Guest works well with fixed official build
> qemu-kvm-rhev-2.6.0-26.el7.x86_64 and Develop private
> build(3.10.0-506.el7.test5.x86_64) 
> 
> 2. Still could reproduce this issue with fixed official build
> qemu-kvm-rhev-2.6.0-26.el7.x86_64 and latest official kernel
> build(kernel-3.10.0-509.el7.x86_64).  
> 
> In QE understanding, we need to fix both qemu-kvm-rhev and kernel component
> for this issue.
> 
> According to QE testing, qemu-kvm-rhev part is fixed. How about kernel part?
> Do we need to file new bz for tracking? or we have already existing bz for
> tracking? 
> 
> Best Regards,
> Junyi

The kernel fix is going in via bz 1367716.
As long as the bug is gone with both the kernel and qemu parts applied then I'm happy.
Now, check all other windows migration hangs with this kernel/qemu pair - hopefully a lot of the other bugs are all fixed with the same code.

Dave

Comment 48 Dr. David Alan Gilbert 2016-09-23 09:54:29 UTC
*** Bug 1363929 has been marked as a duplicate of this bug. ***

Comment 50 errata-xmlrpc 2016-11-07 21:28:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2673.html


Note You need to log in before you can comment on or make changes to this bug.