Bug 836135
Summary: | spice migration: prevent race with libvirt | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 6 | Reporter: | Yonit Halperin <yhalperi> | ||||
Component: | libvirt | Assignee: | Michal Privoznik <mprivozn> | ||||
Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | ||||
Severity: | unspecified | Docs Contact: | |||||
Priority: | unspecified | ||||||
Version: | 6.4 | CC: | acathrow, bsarathy, dallan, dblechte, dyasny, dyuan, juzhang, mkenneth, mzhan, qzhang, rwu, shu, virt-maint, weizhan, ydu, zhpeng, zpeng | ||||
Target Milestone: | rc | ||||||
Target Release: | --- | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | libvirt-0.10.2-2.el6 | Doc Type: | Bug Fix | ||||
Doc Text: |
Cause:
Spice needs some time at the end of migration to transfer internal state to destination. However, it needs helping hand from libvirt.
Consequence:
With previous code it was possible to kill source qemu (and spice server) before internal state was transmitted. This lead into unresponsive client on destination side.
Fix:
After qemu reports migration has finished, libvirt waits for spice server to migrate. Libvirt checks repeatedly on the monitor until the spice server reports success. Qemu is not terminated meanwhile.
Result:
Spice client won't get unresponsive anymore.
|
Story Points: | --- | ||||
Clone Of: | 836133 | Environment: | |||||
Last Closed: | 2013-02-21 07:18:11 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | 836133 | ||||||
Bug Blocks: | 836123, 846910, 846911, 894020 | ||||||
Attachments: |
|
Description
Yonit Halperin
2012-06-28 07:48:52 UTC
Hi Michal, In https://bugzilla.redhat.com/show_bug.cgi?id=836133#c6, Luiz raised a concern that a QMP event triggered by spice is not enough. What do you think? Are there any other requirements? Thanks, Yonit. Yonit, Michal is on vacation this week, do you need an immediate answer? Yonit, I think we can extend the output of 'query-migrate' command by a boolean called 'spice-migration-complete', for instance. Because libvirt uses this command to poll on qemu like every 50ms. Like Luiz said, an event is a kind of tricky as libvirt might miss it. The scenario looks like this: with libvirt running an user starts an migration. Then he kills libvirtd and start it again after while. If qemu would send an event meanwhile it won't reach libvirt. However, when libvirt starts the second time, it knows there was a migration running and starts to poll again. (In reply to comment #4) > Yonit, > > I think we can extend the output of 'query-migrate' command by a boolean > called 'spice-migration-complete', for instance. Because libvirt uses this ok, I'll send a patch to qemu. I hope it will be accepted. > command to poll on qemu like every 50ms. Like Luiz said, an event is a kind > of tricky as libvirt might miss it. The scenario looks like this: with > libvirt running an user starts an migration. Then he kills libvirtd and > start it again after while. If qemu would send an event meanwhile it won't > reach libvirt. However, when libvirt starts the second time, it knows there > was a migration running and starts to poll again. Martin, can we get QA ack? (In reply to comment #6) > Martin, can we get QA ack? QA ack is given. Remove needinfo flag. Patch proposed upstream: https://www.redhat.com/archives/libvir-list/2012-September/msg00941.html Yet another patch proposed upstream: https://www.redhat.com/archives/libvir-list/2012-September/msg01417.html Moving to POST: http://post-office.corp.redhat.com/archives/rhvirt-patches/2012-September/msg00695.html Created attachment 624651 [details]
libvirtd.log
libvirtd.log
Hi Michal: So quickly :), very thanks! verify with build: libvirt-0.10.2-2.el6.x86_64 qemu-kvm-0.12.1.2-2.320.el6.x86_64 spice-server-0.12.0-1.el6.x86_64 spice-gtk-0.14-2.el6.x86_64 spice-protocol-0.12.2-1.el6.noarch step: 1: prepare two machine for migrate 2: on source, create a windows guest with spice ..... <graphics type='spice' autoport='yes' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> ..... 3: migrate guest to target #virsh migrate win7_64 qemu+ssh://target.redhat.com/system 4: check qemu process,make sure seamless-migration=on #ps -ef | grep win7_64 qemu 8278 1 53 17:47 ? 00:26:51 /usr/libexec/qemu-kvm -name win7_64 -S -M rhel6.2.0 -enable-kvm -m 2048 -smp 4,sockets=4,cores=1,threads=1 -uuid d551e24f-1368-6ffb-02fd-01eb28301ab5 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/win7_64.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x4.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x4.0x2 -drive file=/var/lib/libvirt/migrate/win7-64-virtio.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,fd=26,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:38:7e:d1,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5906,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us -vga qxl -global qxl-vga.vram_size=67108864 -device qxl,id=video1,vram_size=67108864,bus=pci.0,addr=0x7 -device intel-hda,id=sound0,bus=pci.0,addr=0x6 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 5: after migrate finished, check libvirt log ..... 61634 2012-10-10 21:40:18.071+0000: 1751: debug : qemuMonitorJSONIOProcessLine:166 : QEMU_MONITOR_RECV_REPLY: mon=0x7f53ac0cf700 reply={"return": {"status": "completed"}, "id": "libvirt-95"} ..... 61668 2012-10-10 21:40:18.071+0000: 1756: debug : virJSONValueToString:1133 : result={"execute":"query-spice","id":"libvirt-96"} 61669 2012-10-10 21:40:18.071+0000: 1756: debug : qemuMonitorJSONCommandWithFd:259 : Send command '{"execute":"query-spice","id":"libvirt-96"}' for write with FD -1 ..... 61763 2012-10-10 21:40:18.073+0000: 1751: debug : qemuMonitorIOProcess:354 : QEMU_MONITOR_IO_PROCESS: mon=0x7f53ac0cf700 buf={"return": {"migrated": tr ue, "enabled": true, "auth": "none", "port": 5906, "host": "0.0.0.0", "channels": []}, "id": "libvirt-96"}^M 61764 len=134 61765 2012-10-10 21:40:18.073+0000: 1751: debug : qemuMonitorJSONIOProcessLine:146 : Line [{"return": {"migrated": true, "enabled": true, "auth": "none ", "port": 5906, "host": "0.0.0.0", "channels": []}, "id": "libvirt-96"}] 61766 2012-10-10 21:40:18.073+0000: 1751: debug : virJSONValueFromString:975 : string={"return": {"migrated": true, "enabled": true, "auth": "none", "p ort": 5906, "host": "0.0.0.0", "channels": []}, "id": "libvirt-96"} verification passed. Hi zhe, Have you verified the bug fix with a remote viewer connection? Spice session migration is relevant only with a spice client session. According to the report in Bug 836133 comment #21, there is still a problem. Hi Yonit, No,when verify this bug, i dont use virt-viewer or remote-viewer to connect guest. I try it yesterday, use virt-viewer to connect guest then doing migrate,i always got error like "{"migrated": false," in libvirt log, but migrate can complete, I will follow up bug: https://bugzilla.redhat.com/show_bug.cgi?id=836133 to track the issue. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2013-0276.html |