Red Hat Bugzilla – Bug 836135
spice migration: prevent race with libvirt
Last modified: 2013-02-21 02:18:11 EST
+++ This bug was initially created as a clone of Bug #836133 +++ Description of problem: In order to support seamless migration (bug 836123) Spice needs to migrate its state, from the migration src to the target, when migration is completed. However, when libvirt identifies that migration has completed, it closes the src qemu, preventing spice from completing its migration. We want to add a QMP event for signaling libvirt when spice migration has completed. libvirt will need to wait for this event before closing the src. (spice api already contains a migration_completion callback for qemu). In order for spice to recognize whether the installed libvirt supports this qmp event, we will also need to add a "seamless-migration=on" parameter to spice arguments in qemu. Only if this parameter is set, spice will execute the seamless migration pathway.
Hi Michal, In https://bugzilla.redhat.com/show_bug.cgi?id=836133#c6, Luiz raised a concern that a QMP event triggered by spice is not enough. What do you think? Are there any other requirements? Thanks, Yonit.
Yonit, Michal is on vacation this week, do you need an immediate answer?
Yonit, I think we can extend the output of 'query-migrate' command by a boolean called 'spice-migration-complete', for instance. Because libvirt uses this command to poll on qemu like every 50ms. Like Luiz said, an event is a kind of tricky as libvirt might miss it. The scenario looks like this: with libvirt running an user starts an migration. Then he kills libvirtd and start it again after while. If qemu would send an event meanwhile it won't reach libvirt. However, when libvirt starts the second time, it knows there was a migration running and starts to poll again.
(In reply to comment #4) > Yonit, > > I think we can extend the output of 'query-migrate' command by a boolean > called 'spice-migration-complete', for instance. Because libvirt uses this ok, I'll send a patch to qemu. I hope it will be accepted. > command to poll on qemu like every 50ms. Like Luiz said, an event is a kind > of tricky as libvirt might miss it. The scenario looks like this: with > libvirt running an user starts an migration. Then he kills libvirtd and > start it again after while. If qemu would send an event meanwhile it won't > reach libvirt. However, when libvirt starts the second time, it knows there > was a migration running and starts to poll again.
Martin, can we get QA ack?
(In reply to comment #6) > Martin, can we get QA ack? QA ack is given. Remove needinfo flag.
Patch proposed upstream: https://www.redhat.com/archives/libvir-list/2012-September/msg00941.html
Yet another patch proposed upstream: https://www.redhat.com/archives/libvir-list/2012-September/msg01417.html
Moving to POST: http://post-office.corp.redhat.com/archives/rhvirt-patches/2012-September/msg00695.html
Created attachment 624651 [details] libvirtd.log libvirtd.log
Hi Michal: So quickly :), very thanks! verify with build: libvirt-0.10.2-2.el6.x86_64 qemu-kvm-0.12.1.2-2.320.el6.x86_64 spice-server-0.12.0-1.el6.x86_64 spice-gtk-0.14-2.el6.x86_64 spice-protocol-0.12.2-1.el6.noarch step: 1: prepare two machine for migrate 2: on source, create a windows guest with spice ..... <graphics type='spice' autoport='yes' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> ..... 3: migrate guest to target #virsh migrate win7_64 qemu+ssh://target.redhat.com/system 4: check qemu process,make sure seamless-migration=on #ps -ef | grep win7_64 qemu 8278 1 53 17:47 ? 00:26:51 /usr/libexec/qemu-kvm -name win7_64 -S -M rhel6.2.0 -enable-kvm -m 2048 -smp 4,sockets=4,cores=1,threads=1 -uuid d551e24f-1368-6ffb-02fd-01eb28301ab5 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/win7_64.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x4.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x4.0x2 -drive file=/var/lib/libvirt/migrate/win7-64-virtio.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,fd=26,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:38:7e:d1,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5906,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us -vga qxl -global qxl-vga.vram_size=67108864 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x7 -device intel-hda,id=sound0,bus=pci.0,addr=0x6 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 5: after migrate finished, check libvirt log ..... 61634 2012-10-10 21:40:18.071+0000: 1751: debug : qemuMonitorJSONIOProcessLine:166 : QEMU_MONITOR_RECV_REPLY: mon=0x7f53ac0cf700 reply={"return": {"status": "completed"}, "id": "libvirt-95"} ..... 61668 2012-10-10 21:40:18.071+0000: 1756: debug : virJSONValueToString:1133 : result={"execute":"query-spice","id":"libvirt-96"} 61669 2012-10-10 21:40:18.071+0000: 1756: debug : qemuMonitorJSONCommandWithFd:259 : Send command '{"execute":"query-spice","id":"libvirt-96"}' for write with FD -1 ..... 61763 2012-10-10 21:40:18.073+0000: 1751: debug : qemuMonitorIOProcess:354 : QEMU_MONITOR_IO_PROCESS: mon=0x7f53ac0cf700 buf={"return": {"migrated": tr ue, "enabled": true, "auth": "none", "port": 5906, "host": "0.0.0.0", "channels": []}, "id": "libvirt-96"}^M 61764 len=134 61765 2012-10-10 21:40:18.073+0000: 1751: debug : qemuMonitorJSONIOProcessLine:146 : Line [{"return": {"migrated": true, "enabled": true, "auth": "none ", "port": 5906, "host": "0.0.0.0", "channels": []}, "id": "libvirt-96"}] 61766 2012-10-10 21:40:18.073+0000: 1751: debug : virJSONValueFromString:975 : string={"return": {"migrated": true, "enabled": true, "auth": "none", "p ort": 5906, "host": "0.0.0.0", "channels": []}, "id": "libvirt-96"} verification passed.
Hi zhe, Have you verified the bug fix with a remote viewer connection? Spice session migration is relevant only with a spice client session. According to the report in Bug 836133 comment #21, there is still a problem.
Hi Yonit, No,when verify this bug, i dont use virt-viewer or remote-viewer to connect guest. I try it yesterday, use virt-viewer to connect guest then doing migrate,i always got error like "{"migrated": false," in libvirt log, but migrate can complete, I will follow up bug: https://bugzilla.redhat.com/show_bug.cgi?id=836133 to track the issue.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2013-0276.html