Red Hat Bugzilla – Bug 1086987
src qemu crashed when starting migration in inmigrate mode
Last modified: 2015-03-05 03:05:45 EST
Description of problem: src qemu crashed when starting migration in inmigrate mode Version-Release number of selected component (if applicable): kernel-3.10.0-121.el7.x86_64 qemu-kvm-1.5.3-60.el7.x86_64 How reproducible: 1/1 Steps to Reproduce: 1.boot a src vm in incoming mode(this is a false operation, but qemu shouldn't crash) /root/staf-kvm-devel/autotest-devel/client/tests/virt/qemu/qemu -monitor stdio \ -S \ -name 'virt-tests-vm1' \ -sandbox off \ -M pc \ -nodefaults \ -vga qxl \ -global qxl-vga.vram_size=33554432 \ -device intel-hda,bus=pci.0,addr=03 \ -device hda-duplex \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140411-165804-A7sX6JQ2,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20140411-165804-A7sX6JQ2,path=/tmp/seabios-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20140411-165804-A7sX6JQ2,iobase=0x402 \ -device nec-usb-xhci,id=usb1,bus=pci.0,addr=04 \ -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/RHEL-Server-7.0-64-virtio.qcow2 \ -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=05 \ -device virtio-net-pci,mac=9a:77:78:79:7a:7b,id=idVJAHXq,mq=on,vectors=10,netdev=idfvC0SQ,bus=pci.0,addr=06 \ -netdev tap,id=idfvC0SQ,vhost=on,queues=4 \ -drive id=drive_cd1,if=none,snapshot=off,aio=native,media=cdrom,file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso \ -device ide-cd,id=cd1,drive=drive_cd1,bootindex=1,bus=ide.0,unit=0 \ -m 4096 \ -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \ -cpu 'SandyBridge',+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -spice port=3000,password=123456,addr=0,seamless-migration=on,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off \ -device virtio-balloon-pci,id=ballooning \ -enable-kvm \ -incoming tcp:0:5300 2. and lauch another one as dst vm 3. start the mig migrate_set_capability auto-converge on migrate -d tcp:ip:port Actual results: src qemu crashes: (gdb) bt #0 0x00007ff66b83a989 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007ff66b83c098 in __GI_abort () at abort.c:90 #2 0x00007ff670d90267 in runstate_set (new_state=new_state@entry=RUN_STATE_FINISH_MIGRATE) at vl.c:678 #3 0x00007ff670d96733 in vm_stop_force_state (state=state@entry=RUN_STATE_FINISH_MIGRATE) at /usr/src/debug/qemu-1.5.3/cpus.c:1152 #4 0x00007ff670d2e1b9 in migration_thread (opaque=0x7ff67127cf80 <current_migration.27431>) at migration.c:619 #5 0x00007ff66ebefdf3 in start_thread (arg=0x7ff6637ac700) at pthread_create.c:308 #6 0x00007ff66b8fb3dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Expected results: qemu should either prompt friendly this is not the support way, or quit or something except crash Additional info:
Nice, still does it on upstream 2.0.0rc2 as well. (It's minor though since this is an invalid thing to do anyway and you've not got any data in the guest).
Posted patch upstream.
Fix included in qemu-kvm-1.5.3-66.el7
Reproduce this bug using the following version: qemu-kvm-1.5.3-63.el7.x86_64 kernel-3.10.0-140.el7.x86_64 Steps to Reproduce: 1. boot a src vm in incoming mode # /usr/libexec/qemu-kvm \ -monitor stdio \ -S \ -name 'virt-tests-vm1' \ -sandbox off \ -M pc \ -nodefaults \ -vga qxl \ -global qxl-vga.vram_size=33554432 \ -device intel-hda,bus=pci.0,addr=03 \ -device hda-duplex \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140411-165804-A7sX6JQ2,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20140411-165804-A7sX6JQ2,path=/tmp/seabios-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20140411-165804-A7sX6JQ2,iobase=0x402 \ -device nec-usb-xhci,id=usb1,bus=pci.0,addr=04 \ -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/mnt/rhel7-64-ga-3.qcow2 \ -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=05 \ -device virtio-net-pci,mac=9a:77:78:79:7a:7b,id=idVJAHXq,mq=on,vectors=10,netdev=idfvC0SQ,bus=pci.0,addr=06 \ -netdev tap,id=idfvC0SQ,vhost=on,queues=4 \ -drive id=drive_cd1,if=none,snapshot=off,aio=native,media=cdrom,file=/mnt/driver.iso \ -device ide-cd,id=cd1,drive=drive_cd1,bootindex=1,bus=ide.0,unit=0 \ -m 4096 \ -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \ -cpu 'SandyBridge',+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -spice port=3000,password=123456,addr=0,seamless-migration=on,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off \ -device virtio-balloon-pci,id=ballooning \ -enable-kvm \ -incoming tcp:0:5300 2. and lauch another one as dst vm # /usr/libexec/qemu-kvm \ -monitor stdio \ -S \ -name 'virt-tests-vm1' \ -sandbox off \ -M pc \ -nodefaults \ -vga qxl \ -global qxl-vga.vram_size=33554432 \ -device intel-hda,bus=pci.0,addr=03 \ -device hda-duplex \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140411-165804-A7sX6JQ2,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20140411-165804-A7sX6JQ2,path=/tmp/seabios-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20140411-165804-A7sX6JQ2,iobase=0x402 \ -device nec-usb-xhci,id=usb1,bus=pci.0,addr=04 \ -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/mnt/rhel7-64-ga-3.qcow2 \ -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=05 \ -device virtio-net-pci,mac=9a:77:78:79:7a:7b,id=idVJAHXq,mq=on,vectors=10,netdev=idfvC0SQ,bus=pci.0,addr=06 \ -netdev tap,id=idfvC0SQ,vhost=on,queues=4 \ -drive id=drive_cd1,if=none,snapshot=off,aio=native,media=cdrom,file=/mnt/driver.iso \ -device ide-cd,id=cd1,drive=drive_cd1,bootindex=1,bus=ide.0,unit=0 \ -m 4096 \ -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \ -cpu 'SandyBridge',+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -spice port=4000,password=123456,addr=0,seamless-migration=on,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off \ -device virtio-balloon-pci,id=ballooning \ -enable-kvm \ -incoming tcp:0:5800 3. start the mig (qemu)migrate_set_capability auto-converge on (qemu)migrate -d tcp:0:5800 Results: after step3, src qemu crashes: (gdb) bt #0 0x00007ffff2c91989 in raise () from /lib64/libc.so.6 #1 0x00007ffff2c93098 in abort () from /lib64/libc.so.6 #2 0x00005555557435d7 in runstate_set () #3 0x0000555555749c23 in vm_stop_force_state () #4 0x00005555556e14c9 in migration_thread () #5 0x00007ffff604ddf3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ffff2d523dd in clone () from /lib64/libc.so.6
Test this bug on an intel host using the following version: qemu-kvm-1.5.3-66.el7.x86_64 kernel-3.10.0-140.el7.x86_64 Steps to Test: 1. boot a src vm in incoming mode # /usr/libexec/qemu-kvm \ -monitor stdio \ -S \ -name 'virt-tests-vm1' \ -sandbox off \ -M pc \ -nodefaults \ -vga qxl \ -global qxl-vga.vram_size=33554432 \ -device intel-hda,bus=pci.0,addr=03 \ -device hda-duplex \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140411-165804-A7sX6JQ2,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20140411-165804-A7sX6JQ2,path=/tmp/seabios-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20140411-165804-A7sX6JQ2,iobase=0x402 \ -device nec-usb-xhci,id=usb1,bus=pci.0,addr=04 \ -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/mnt/rhel7-64-ga-3.qcow2 \ -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=05 \ -device virtio-net-pci,mac=9a:77:78:79:7a:7b,id=idVJAHXq,mq=on,vectors=10,netdev=idfvC0SQ,bus=pci.0,addr=06 \ -netdev tap,id=idfvC0SQ,vhost=on,queues=4 \ -drive id=drive_cd1,if=none,snapshot=off,aio=native,media=cdrom,file=/mnt/driver.iso \ -device ide-cd,id=cd1,drive=drive_cd1,bootindex=1,bus=ide.0,unit=0 \ -m 4096 \ -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \ -cpu 'SandyBridge',+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -spice port=3000,password=123456,addr=0,seamless-migration=on,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off \ -device virtio-balloon-pci,id=ballooning \ -enable-kvm \ -incoming tcp:0:5300 2. and lauch another one as dst vm # /usr/libexec/qemu-kvm \ -monitor stdio \ -S \ -name 'virt-tests-vm1' \ -sandbox off \ -M pc \ -nodefaults \ -vga qxl \ -global qxl-vga.vram_size=33554432 \ -device intel-hda,bus=pci.0,addr=03 \ -device hda-duplex \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140411-165804-A7sX6JQ2,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20140411-165804-A7sX6JQ2,path=/tmp/seabios-20140411-165804-A7sX6JQ2,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20140411-165804-A7sX6JQ2,iobase=0x402 \ -device nec-usb-xhci,id=usb1,bus=pci.0,addr=04 \ -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/mnt/rhel7-64-ga-3.qcow2 \ -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=05 \ -device virtio-net-pci,mac=9a:77:78:79:7a:7b,id=idVJAHXq,mq=on,vectors=10,netdev=idfvC0SQ,bus=pci.0,addr=06 \ -netdev tap,id=idfvC0SQ,vhost=on,queues=4 \ -drive id=drive_cd1,if=none,snapshot=off,aio=native,media=cdrom,file=/mnt/driver.iso \ -device ide-cd,id=cd1,drive=drive_cd1,bootindex=1,bus=ide.0,unit=0 \ -m 4096 \ -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \ -cpu 'SandyBridge',+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -spice port=4000,password=123456,addr=0,seamless-migration=on,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off \ -device virtio-balloon-pci,id=ballooning \ -enable-kvm \ -incoming tcp:0:5800 3. start the mig (qemu)migrate_set_capability auto-converge on (qemu)migrate -d tcp:0:5800 Results: after step3, qemu-kvm does not crash and migration is failed with the following info: (qemu)migrate -d tcp:0:5800 migrate: Guest is waiting for an incoming migration I also test qmp command, qemu-kvm does not crash and the qmp command return the following info: {"execute": "migrate","arguments":{"uri": "tcp:0:5800"}} {"error": {"class": "GenericError", "desc": "Guest is waiting for an incoming migration"}}
Test this bug on an intel host using the following version: qemu-kvm-rhev-2.1.0-3.el7ev.preview kernel-3.10.0-140.el7.x86_64 Use the same steps of comment 6, test HMP and QMP command, the results are that qemu-kvm doesn't crash and migration is failed with the error info: Guest is waiting for an incoming migration
Test qemu-kvm-rhev-2.1.0-3.el7ev.preview and qemu-kvm-1.5.3-66.el7.x86_64 on an amd host. Use the same steps of comment 6, test HMP and QMP command, the results are that qemu-kvm doesn't crash and migration is failed with the error info: Guest is waiting for an incoming migration
*** Bug 1006663 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0349.html