Bug 1357765
| Summary: | windows10 guest hangs after migration | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | yduan | ||||||
| Component: | qemu-kvm-rhev | Assignee: | Amit Shah <amit.shah> | ||||||
| Status: | CLOSED DUPLICATE | QA Contact: | Qianqian Zhu <qizhu> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | 7.3 | CC: | amit.shah, chayang, dgilbert, jasowang, jinzhao, juzhang, knoel, mst, pezhang, qizhu, quintela, virt-maint, xfu, yduan | ||||||
| Target Milestone: | rc | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2016-09-27 11:25:31 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Additional info: Not reproducible with '-machine pc' machine type. CMD is as attachment. Created attachment 1183613 [details]
Not reproducible with '-machine pc' machine type
Can you please try with the latest qemu-kvm-rhev? Also, let me know if it reproduces - I'd like access to your test systems. Migration succeed when without CMD below: -netdev tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ -device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pcie.0 \ (In reply to yduan from comment #6) > Migration succeed when without CMD below: > > -netdev > tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ > -device > virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on, > bus=pcie.0 \ It might be worth trying just changing that to vhost=off ? Dave (In reply to yduan from comment #6) > Migration succeed when without CMD below: > > -netdev > tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ > -device > virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on, > bus=pcie.0 \ Thanks for digging into this! Can you please try vhost=off as Dave mentions above? Michael, Jason, are you aware of any virtio or vhost related regressions that should be backported? (In reply to Amit Shah from comment #8) > (In reply to yduan from comment #6) > > Migration succeed when without CMD below: > > > > -netdev > > tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ > > -device > > virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on, > > bus=pcie.0 \ > > Can you please try vhost=off as Dave mentions above? > Using the latest qemu-kvm-rhev, the destination qemu monitor quits with following prompts as soon as migration starts in the source, no matter what "vhost" is set ("on" or "off"). (qemu) qemu-kvm: Missing section footer for 0000:00:03.0/virtio-net qemu-kvm: load of migration failed: Invalid argument red_channel_client_disconnect_dummy: rcc=0x7f908f903000 (channel=0x7f90918ec260 type=5 id=0) snd_channel_put: SndChannel=0x7f909129a000 freed red_channel_client_disconnect_dummy: rcc=0x7f90915e0000 (channel=0x7f90918ec100 type=6 id=0) snd_channel_put: SndChannel=0x7f90902dc000 freed red_channel_client_disconnect: rcc=0x7f9091252000 (channel=0x7f908fd26c00 type=2 id=0) qemu-kvm: network script /etc/ifdown_script failed with status 256 red_channel_client_disconnect: rcc=0x7f9090112000 (channel=0x7f90900bfb80 type=4 id=0) [root@dhcp-9-49 backup]# uname -r 3.10.0-489.el7.x86_64 [root@dhcp-9-49 backup]# rpm -q qemu-kvm-rhev qemu-kvm-rhev-2.6.0-20.el7.x86_64 (In reply to Amit Shah from comment #8) > (In reply to yduan from comment #6) > > Migration succeed when without CMD below: > > > > -netdev > > tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ > > -device > > virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on, > > bus=pcie.0 \ > > Thanks for digging into this! > > Can you please try vhost=off as Dave mentions above? > > Michael, Jason, are you aware of any virtio or vhost related regressions > that should be backported? Seems not. No - how about disabling the modern interface and using the legacy one? Does this work around the issue? (In reply to Michael S. Tsirkin from comment #11) > No - how about disabling the modern interface and using the legacy one? > Does this work around the issue? There is no difference without "disable-legacy=on,disable-modern=off". (qemu) qemu-kvm: Missing section footer for 0000:00:03.0/virtio-net qemu-kvm: load of migration failed: Invalid argument red_channel_client_disconnect_dummy: rcc=0x7f4efdc53000 (channel=0x7f4effc3c260 type=5 id=0) snd_channel_put: SndChannel=0x7f4efe79e000 freed red_channel_client_disconnect_dummy: rcc=0x7f4eff7a6000 (channel=0x7f4effc3c100 type=6 id=0) snd_channel_put: SndChannel=0x7f4efe5c8000 freed red_channel_client_disconnect: rcc=0x7f4eff9e4000 (channel=0x7f4efe076c00 type=2 id=0) red_channel_client_disconnect: rcc=0x7f4eff616000 (channel=0x7f4efe40fb80 type=4 id=0) qemu-kvm: network script /etc/ifdown_script failed with status 256 [root@dhcp-9-49 backup]# uname -r 3.10.0-489.el7.x86_64 [root@dhcp-9-49 backup]# rpm -q qemu-kvm-rhev qemu-kvm-rhev-2.6.0-20.el7.x86_64 I think the problem here could be due to us missing two backports, so can you please try: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=11568745 Michael, Jason, this has the following two commits added: 62cee1a28aada2cce4b0e1fb835d8fc830aed7ac 6c6668232e71b7cf7ff39fa1a7abf660c40f9cea (In reply to Dr. David Alan Gilbert from comment #14) > I think the problem here could be due to us missing two backports, so can > you please try: > > https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=11568745 > virtio-win-prewhql-0.1-125 # uname -r 3.10.0-489.el7.x86_64 # rpm -q qemu-kvm-rhev qemu-kvm-rhev-2.6.0-20.el7.1365747a.x86_64 After migration completed, windows10 guest hangs with no any error prompt. # uname -r 3.10.0-489.el7.x86_64 # rpm -q qemu-kvm-rhev **qemu-kvm-rhev-2.6.0-20.el7.1365747a.x86_64** Migration succeeds after updating network adapter driver to latest **virtio-win-prewhql-0.1-126**. (In reply to yduan from comment #17) > # uname -r > 3.10.0-489.el7.x86_64 > # rpm -q qemu-kvm-rhev > **qemu-kvm-rhev-2.6.0-20.el7.1365747a.x86_64** > > Migration succeeds after updating network adapter driver to latest > **virtio-win-prewhql-0.1-126**. Do you know exactly which version you had previously? (In reply to Dr. David Alan Gilbert from comment #18) > (In reply to yduan from comment #17) > > # uname -r > > 3.10.0-489.el7.x86_64 > > # rpm -q qemu-kvm-rhev > > **qemu-kvm-rhev-2.6.0-20.el7.1365747a.x86_64** > > > > Migration succeeds after updating network adapter driver to latest > > **virtio-win-prewhql-0.1-126**. > > Do you know exactly which version you had previously? I think you are asking for virtio-win version. As metioned in comment 15, it is virtio-win-prewhql-0.1-125. OK, that set of fixes that was in that test brew are in qemu-kvm-rhev-2.6.0-21.el7, so if that works in combination with the -126 driver then I think it's probably worth closing this, however please run the test a few times since I suspect we might have more intermittent bugs. # uname -r 3.10.0-492.el7.x86_64 # rpm -q qemu-kvm-rhev qemu-kvm-rhev-2.6.0-21.el7.x86_64 virtio-win-prewhql-0.1-126 It is not reproducible when windows 10 guest is installed from en_windows_10_enterprise_version_1511_updated_apr_2016_x64_dvd_8711771.iso (ISO link: http://tree.englab.nay.redhat.com/pub/windows/win10/en_windows_10_enterprise_version_1511_updated_apr_2016_x64_dvd_8711771.iso). It is reproducible when windows 10 guest is installed from en_windows_10_enterprise_version_1607_updated_jul_2016_x64_dvd_9054264.iso (ISO link: http://tree.englab.nay.redhat.com/pub/windows/win10/en_windows_10_enterprise_version_1607_updated_jul_2016_x64_dvd_9054264.iso). CMD: /usr/libexec/qemu-kvm \ -S \ -name 'rhel7.3' \ -machine q35,accel=kvm,vmport=off \ -m 4096 \ -smp 4,maxcpus=4,sockets=1,cores=2,threads=2 \ -cpu SandyBridge,enforce \ -rtc base=localtime,clock=host,driftfix=slew \ -nodefaults \ -vga qxl \ -device AC97,bus=pcie.0 \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idSWJ5gV \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151214-111528-C6FB1EaX,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_log,path=/tmp/seabios-log,server,nowait \ -device isa-debugcon,chardev=seabioslog_log,iobase=0x402 \ -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pcie.0 \ -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pcie.0 \ -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pcie.0 \ -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pcie.0 \ -device usb-tablet,id=usb-tablet1 \ -boot menu=on \ -enable-kvm \ -monitor stdio \ -spice port=5900,disable-ticketing \ -qmp tcp:0:9999,server,nowait \ -netdev tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ -device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pcie.0 \ -drive file=/home/backup/virtio-win-prewhql-0.1-126.iso,if=none,media=cdrom,id=drive_wincd,readonly=on,format=raw \ -device ide-cd,bus=ide.0,drive=drive_wincd,id=device_wincd \ -device ioh3420,bus=pcie.0,id=root.0,slot=0 \ -device ioh3420,bus=pcie.0,id=root.1,slot=1 \ -device ioh3420,bus=pcie.0,id=root.2,slot=2 \ -device x3130-upstream,bus=root.2,id=upstream0 \ -device xio3130-downstream,bus=upstream0,id=downstream0,chassis=1 \ -device xio3130-downstream,bus=upstream0,id=downstream1,chassis=2 \ -device virtio-scsi-pci,bus=downstream0,id=scsi_pci_bus0,disable-legacy=on,disable-modern=off \ -drive file=/home/sysdisk.qcow2,format=qcow2,id=drive_sysdisk,if=none,cache=none,aio=native,werror=stop,rerror=stop \ -device scsi-hd,drive=drive_sysdisk,bus=scsi_pci_bus0.0,id=device_sysdisk,bootindex=0,physical_block_size=4096,logical_block_size=512 \ -drive file=/home/backup/en_windows_10_enterprise_version_1511_updated_apr_2016_x64_dvd_8711771.iso,if=none,media=cdrom,id=drive_syscd,readonly=on,format=raw \ -device scsi-disk,bus=scsi_pci_bus0.0,drive=drive_syscd,id=device_syscd,bootindex=1 \ I met this issue both "q35" and "pc" recently, so i retry two cases here. virtio-win-prewhql-0.1-126 # uname -r 3.10.0-492.el7.x86_64 # rpm -q qemu-kvm-rhev qemu-kvm-rhev-2.6.0-22.el7.x86_64 Description of problem: After migration completed, windows10 guest hangs with no any error prompt. Q35 How reproducible: 2/2 PC How reproducible: 2/2 Q35 CMD: /usr/libexec/qemu-kvm \ -S \ -name 'Win10-x64' \ -machine q35,accel=kvm,vmport=off \ -m 4096 \ -smp 4,maxcpus=4,sockets=1,cores=4,threads=1 \ -cpu Opteron_G5,enforce \ -rtc base=localtime,clock=host,driftfix=slew \ -nodefaults \ -vga qxl \ -device AC97,bus=pcie.0 \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idSWJ5gV \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151214-111528-C6FB1EaX,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_log,path=/tmp/seabios-log,server,nowait \ -device isa-debugcon,chardev=seabioslog_log,iobase=0x402 \ -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pcie.0 \ -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pcie.0 \ -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pcie.0 \ -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pcie.0 \ -device usb-tablet,id=usb-tablet1 \ -boot menu=on \ -enable-kvm \ -monitor stdio \ -spice port=5900,disable-ticketing \ -qmp tcp:0:9999,server,nowait \ -netdev tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ -device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pcie.0 \ -drive file=/home/virtio-win-prewhql-0.1-126.iso,if=none,media=cdrom,id=drive_wincd,readonly=on,format=raw \ -device ide-cd,bus=ide.0,drive=drive_wincd,id=device_wincd \ -device ioh3420,bus=pcie.0,id=root.0,slot=0 \ -device ioh3420,bus=pcie.0,id=root.1,slot=1 \ -device ioh3420,bus=pcie.0,id=root.2,slot=2 \ -device x3130-upstream,bus=root.2,id=upstream0 \ -device xio3130-downstream,bus=upstream0,id=downstream0,chassis=1 \ -device xio3130-downstream,bus=upstream0,id=downstream1,chassis=2 \ -device virtio-scsi-pci,bus=downstream0,id=scsi_pci_bus0 \ -drive file=/mnt/sysdisk.qcow2,format=qcow2,id=drive_sysdisk,if=none,cache=none,aio=native,werror=stop,rerror=stop \ -device scsi-hd,drive=drive_sysdisk,bus=scsi_pci_bus0.0,id=device_sysdisk,bootindex=0 \ -drive file=/home/en_windows_10_enterprise_version_1607_updated_jul_2016_x64_dvd_9054264.iso,if=none,media=cdrom,id=drive_syscd,readonly=on,format=raw \ -device scsi-disk,bus=scsi_pci_bus0.0,drive=drive_syscd,id=device_syscd,bootindex=1 \ PC CMD: /usr/libexec/qemu-kvm \ -S \ -name 'win10' \ -machine pc \ -m 4096 \ -smp 4,maxcpus=4,sockets=1,cores=4,threads=1 \ -cpu Opteron_G5,check \ -rtc base=localtime,clock=host,driftfix=slew \ -nodefaults \ -vga qxl \ -device AC97,bus=pci.0 \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idSWJ5gV \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151214-111528-C6FB1EaX,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_log,path=/tmp/seabios-log,server,nowait \ -device isa-debugcon,chardev=seabioslog_log,iobase=0x402 \ -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pci.0 \ -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pci.0 \ -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pci.0 \ -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pci.0 \ -device usb-tablet,id=usb-tablet1 \ -boot menu=on \ -enable-kvm \ -monitor stdio \ -spice port=5900,disable-ticketing \ -qmp tcp:0:9999,server,nowait \ -netdev tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ -device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pci.0 \ -drive file=/home/virtio-win-prewhql-0.1-126.iso,if=none,cache=none,media=cdrom,id=drive_wincd,readonly=on,format=raw \ -device ide-cd,bus=ide.0,drive=drive_wincd,id=device_wincd \ -device virtio-scsi-pci,id=scsi_pci_bus0 \ -drive file=/mnt/sysdisk.qcow2,format=qcow2,id=drive_sysdisk,if=none,cache=none,aio=native,werror=stop,rerror=stop \ -device scsi-hd,drive=drive_sysdisk,bus=scsi_pci_bus0.0,id=device_sysdisk,bootindex=0 \ -drive file=/home/en_windows_10_enterprise_version_1607_updated_jul_2016_x64_dvd_9054264.iso,if=none,cache=none,media=cdrom,id=drive_syscd,readonly=on,format=raw \ -device scsi-disk,bus=scsi_pci_bus0.0,drive=drive_syscd,id=device_syscd,bootindex=1 \ (In reply to yduan from comment #22) > I met this issue both "q35" and "pc" recently, so i retry two cases here. > > virtio-win-prewhql-0.1-126 > # uname -r > 3.10.0-492.el7.x86_64 > # rpm -q qemu-kvm-rhev > qemu-kvm-rhev-2.6.0-22.el7.x86_64 > > Description of problem: > After migration completed, windows10 guest hangs with no any error prompt. > In this situation, guest works well after "system_reset" in HMP. FYI. In this testing, the windows 10 migration works well. Versions: Host: 3.10.0-510.el7.x86_64 qemu-kvm-rhev-2.6.0-27.el7.x86_64 Guest: Windows10 Steps: 1. Boot win10 guest /usr/libexec/qemu-kvm -name win10 \ -cpu SandyBridge -m 4G -numa node \ -smp 4,sockets=2,cores=2,threads=1 \ -netdev tap,id=SandyBridgenet0 \ -device virtio-net-pci,netdev=SandyBridgenet0,id=net0,mac=12:54:00:5c:77:01 \ -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16 \ -spice port=5902,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on \ -monitor stdio \ -serial unix:/tmp/monitor,server,nowait \ -qmp tcp:0:5551,server,nowait \ -drive file=/mnt/win10-64-virtio.qcow2,format=qcow2,if=none,id=drive-virtio-blk0,werror=stop,rerror=stop \ -device virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0,ioeventfd=off \ -usbdevice tablet 2. Boot guest in des with '-incoming' <same as step1> -incoming tcp:0:5555 \ 3. Do migrate (qemu) migrate -d tcp:10.73.72.154:5555 4. After migration, check guest status, works well. 5. Reboot guest several times and check guest status, works well. # reboot network keeps working well # ping google.com PING google.com (216.58.196.174) 56(84) bytes of data. 64 bytes from sin04s05-in-f174.1e100.net (216.58.196.174): icmp_seq=1 ttl=49 time=68.6 ms ... Great, thanks for confirming! So It looks like Dave's fix worked for this bug too. I'll close it as a dup. *** This bug has been marked as a duplicate of bug 1377920 *** |
Created attachment 1181414 [details] destination guest hangs Description of problem: After migration completed, windows10 guest hangs with no any error prompt. Version-Release number of selected component (if applicable): Host: kernel: 3.10.0-469.el7.x86_64 qemu-kvm-rhev-2.6.0-13.el7.x86_64 Guest: windows 10 How reproducible: 100% Steps to Reproduce: 1.Start VM with following commands: /usr/libexec/qemu-kvm \ -S \ -name 'rhel7.3' \ -machine q35,accel=kvm,vmport=off \ -m 4096 \ -smp 4,maxcpus=4,sockets=1,cores=2,threads=2 \ -cpu SandyBridge,enforce \ -rtc base=localtime,clock=host,driftfix=slew \ -nodefaults \ -vga qxl \ -device AC97,bus=pcie.0 \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idSWJ5gV \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151214-111528-C6FB1EaX,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_log,path=/tmp/seabios-log,server,nowait \ -device isa-debugcon,chardev=seabioslog_log,iobase=0x402 \ -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pcie.0 \ -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pcie.0 \ -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pcie.0 \ -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pcie.0 \ -device usb-tablet,id=usb-tablet1 \ -boot menu=on \ -enable-kvm \ -monitor stdio \ -spice port=5900,disable-ticketing \ -qmp tcp:0:9999,server,nowait \ -drive file=/home/win10-seabios-sysdisk.qcow2,format=qcow2,id=drive_sysdisk,if=none,cache=none,aio=native,werror=stop,rerror=stop \ -device ide-drive,drive=drive_sysdisk,bus=ide.0,id=device_sysdisk,bootindex=0,physical_block_size=4096,logical_block_size=512 \ -netdev tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ -device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pcie.0 \ -device ioh3420,bus=pcie.0,id=root.0,slot=0 \ -device virtio-scsi-pci,bus=root.0,id=scsi_pci_bus,disable-legacy=on,disable-modern=off \ -drive file=/home/datadisk5G.qcow2,format=qcow2,id=drive_datadisk5G,if=none,cache=none,aio=native,werror=stop,rerror=stop \ -device scsi-hd,drive=drive_datadisk5G,bus=scsi_pci_bus.0,id=device_datadisk5G,physical_block_size=4096,logical_block_size=4096 \ -drive file=/home/cdimg.iso,if=none,media=cdrom,id=drive_cd,readonly=on \ -device scsi-cd,bus=scsi_pci_bus.0,drive=drive_cd,id=device_cd \ -device ioh3420,bus=pcie.0,id=root.1,slot=1 \ -device ioh3420,bus=pcie.0,id=root.2,slot=2 \ -device x3130-upstream,bus=root.2,id=upstream0 \ -device xio3130-downstream,bus=upstream0,id=downstream0,chassis=1 \ -device xio3130-downstream,bus=upstream0,id=downstream1,chassis=2 \ -device virtio-scsi-pci,bus=downstream0,id=scsi_pci_bus0 \ -drive file=/home/backup/en_windows_10_enterprise_version_1511_updated_feb_2016_x64_dvd_8378876.iso,if=none,media=cdrom,id=drive_syscd,readonly=on,format=raw \ -device scsi-disk,bus=scsi_pci_bus0.0,drive=drive_syscd,id=device_syscd,bootindex=1 \ -device virtio-scsi-pci,bus=downstream1,id=scsi_pci_bus1,disable-legacy=on,disable-modern=off \ -drive file=/home/datadisk2G.qcow2,format=qcow2,id=drive_datadisk2G,if=none,cache=none,aio=native,werror=stop,rerror=stop \ -device scsi-hd,drive=drive_datadisk2G,bus=scsi_pci_bus1.0,id=device_datadisk2G,physical_block_size=512,logical_block_size=512 \ 2.Boot the guest on destination host with incoming option: /usr/libexec/qemu-kvm \ -S \ -name 'rhel7.3' \ -machine q35,accel=kvm,vmport=off \ -m 4096 \ -smp 4,maxcpus=4,sockets=1,cores=2,threads=2 \ -cpu SandyBridge,enforce \ -rtc base=localtime,clock=host,driftfix=slew \ -nodefaults \ -vga qxl \ -device AC97,bus=pcie.0 \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151214-111528-C6FB1EaX,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idSWJ5gV \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151214-111528-C6FB1EaX,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_log,path=/tmp/seabios-log,server,nowait \ -device isa-debugcon,chardev=seabioslog_log,iobase=0x402 \ -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pcie.0 \ -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pcie.0 \ -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pcie.0 \ -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pcie.0 \ -device usb-tablet,id=usb-tablet1 \ -boot menu=on \ -enable-kvm \ -monitor stdio \ -spice port=5800,disable-ticketing \ -qmp tcp:0:8888,server,nowait \ -drive file=/home/win10-seabios-sysdisk.qcow2,format=qcow2,id=drive_sysdisk,if=none,cache=none,aio=native,werror=stop,rerror=stop \ -device ide-drive,drive=drive_sysdisk,bus=ide.0,id=device_sysdisk,bootindex=0,physical_block_size=4096,logical_block_size=512 \ -netdev tap,id=netdev0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/ifdown_script \ -device virtio-net-pci,mac=BA:BC:13:83:4F:BD,id=net0,netdev=netdev0,status=on,bus=pcie.0 \ -device ioh3420,bus=pcie.0,id=root.0,slot=0 \ -device virtio-scsi-pci,bus=root.0,id=scsi_pci_bus,disable-legacy=on,disable-modern=off \ -drive file=/home/datadisk5G.qcow2,format=qcow2,id=drive_datadisk5G,if=none,cache=none,aio=native,werror=stop,rerror=stop \ -device scsi-hd,drive=drive_datadisk5G,bus=scsi_pci_bus.0,id=device_datadisk5G,physical_block_size=4096,logical_block_size=4096 \ -drive file=/home/cdimg.iso,if=none,media=cdrom,id=drive_cd,readonly=on \ -device scsi-cd,bus=scsi_pci_bus.0,drive=drive_cd,id=device_cd \ -device ioh3420,bus=pcie.0,id=root.1,slot=1 \ -device ioh3420,bus=pcie.0,id=root.2,slot=2 \ -device x3130-upstream,bus=root.2,id=upstream0 \ -device xio3130-downstream,bus=upstream0,id=downstream0,chassis=1 \ -device xio3130-downstream,bus=upstream0,id=downstream1,chassis=2 \ -device virtio-scsi-pci,bus=downstream0,id=scsi_pci_bus0 \ -drive file=/home/backup/en_windows_10_enterprise_version_1511_updated_feb_2016_x64_dvd_8378876.iso,if=none,media=cdrom,id=drive_syscd,readonly=on,format=raw \ -device scsi-disk,bus=scsi_pci_bus0.0,drive=drive_syscd,id=device_syscd,bootindex=1 \ -device virtio-scsi-pci,bus=downstream1,id=scsi_pci_bus1,disable-legacy=on,disable-modern=off \ -drive file=/home/datadisk2G.qcow2,format=qcow2,id=drive_datadisk2G,if=none,cache=none,aio=native,werror=stop,rerror=stop \ -device scsi-hd,drive=drive_datadisk2G,bus=scsi_pci_bus1.0,id=device_datadisk2G,physical_block_size=512,logical_block_size=512 \ -incoming tcp:0:1234 3.Migrate to the destination: {"execute": "migrate","arguments":{"uri": "tcp:0:1234"}} Actual results: In source HMP: (qemu) info migrate capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: off compress: off events: off postcopy-ram: off Migration status: completed total time: 103948 milliseconds downtime: 14 milliseconds setup: 17 milliseconds transferred ram: 3414497 kbytes throughput: 269.11 mbps remaining ram: 0 kbytes total ram: 4326224 kbytes duplicate: 793549 pages skipped: 0 pages normal: 850218 pages normal bytes: 3400872 kbytes dirty sync count: 25 (qemu) (qemu) info status VM status: paused (postmigrate) In dest HMP: (qemu) info status VM status: running When migration is completed, destination guest hangs as attachment. Expected results: Destination guest should work after migration. Additional info: With qemu-kvm-rhev-2.3.0-31.el7_2.18.x86_64 (qemu) migrate -d tcp:0:1234 migrate: State blocked by non-migratable device '0000:00:1f.2/ich9_ahci'