Bug 1442929
| Summary: | virt-viewer cannot keep connection while seamless migration | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Yanqiu Zhang <yanqzhan> |
| Component: | virt-viewer | Assignee: | Pavel Grunt <pgrunt> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.4 | CC: | dblechte, dyuan, fjin, juzhou, kuwei, lizhu, mxie, mzhan, pgrunt, rbalakri, tzheng, victortoso, xiaodwan, xuzhang, yafu, yanqzhan, zpeng |
| Target Milestone: | rc | Keywords: | Regression |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | virt-viewer-5.0-4.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-08-01 15:06:41 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1449577 | ||
Since you say that it is a regression, which version of spice-gtk & virt-viewer works ? Also try using --direct: virt-viewer V --debug --spice-debug --direct Last time, we tested rhel7.4 libvirt-3.1.0-1 with virt-viewer-2.0-13.el7.x86_64.rpm & spice-gtk3-0.31-8.el7.x86_64.rpm, test passed. Also, now rhel7.3 testing on libvirt-2.0.0-10 passed, with following pkgs version: virt-viewer-2.0-12.el7.x86_64 spice-gtk3-0.31-6.el7_3.2.x86_64 Hi,
I used remote-viewer to connect to a rhv 4.1 instance + migration between hosts and I was not able to reproduce it virt-viewer-5.0-2.el7.x86_64 and spice-gtk3-0.33-3.el7.x86_64.
Both spice-gtk and spice server were rebased this cycle. From the logs nothing poped up from the client side.
Could you provide the qemu+spice logs from both src & target hosts? (/var/log/libvirt/qemu/<domain>.log)
Could you try downgrading spice to rhel-7.3 version, which is 0.12.4-20 (both src/target hosts) and trying again?
The only bit I found suspicious in the client-side log is:
> (virt-viewer:9553): GSpice-DEBUG: spice-session.c:1701
> migration channels left:8 (in migration:8)
But this is *AFTER* the migration was already completed. Something seems off indeed. I'm still looking into the reason.
(In reply to Victor Toso from comment #6) Hi > I used remote-viewer to connect to a rhv 4.1 instance + migration between > hosts and I was not able to reproduce it virt-viewer-5.0-2.el7.x86_64 and > spice-gtk3-0.33-3.el7.x86_64. Yes, remote-viewer will not reproduce it, only reproduce with virt-viewer. As descripted in comment 0 additional info: "remote-viewer spice://{source_ip}:5900" works well > Both spice-gtk and spice server were rebased this cycle. From the logs > nothing poped up from the client side. all 7.4 test are with spice-server-0.12.8-1.el7.x86_64 > Could you provide the qemu+spice logs from both src & target hosts? > (/var/log/libvirt/qemu/<domain>.log) Pls refer to the new attachment "qemu-log" in comment 7. > Could you try downgrading spice to rhel-7.3 version, which is 0.12.4-20 > (both src/target hosts) and trying again? Retest by downgrading some pkgs, here are additional test scenarios: a)works well on : libvirt-3.1.0-1.el7.x86_64 virt-viewer-2.0-13.el7.x86_64 spice-gtk3-0.31-8.el7.x86_64 b)issue can also reproduce on: libvirt-3.1.0-1.el7.x86_64 virt-viewer-5.0-2.el7.x86_64 spice-gtk3-0.33-3.el7.x86_64 So the issue is caused by virt-viewer-5.0-2.el7.x86_64 and spice-gtk3-0.33-3.el7.x86_64. the versions of virt-viewer and spice-gtk3 are interdependent, can not individually downgrade one and leave another one high version. Why need to downgrade spice to rhel-7.3 version(0.12.4-20) ? btw:also test with '--direct', no help, still reproduce. > Yes, remote-viewer will not reproduce it, only reproduce with virt-viewer. > As descripted in comment 0 additional info: > "remote-viewer spice://{source_ip}:5900" works well Sorry, I missed that! > > Could you provide the qemu+spice logs from both src & target hosts? > > (/var/log/libvirt/qemu/<domain>.log) > > Pls refer to the new attachment "qemu-log" in comment 7. If possible, can you enable debug using SPICE_DEBUG=1 ? I usually do that by including export SPICE_DEBUG=1 in a custom qemu-kvm script. > > > Could you try downgrading spice to rhel-7.3 version, which is 0.12.4-20 > > (both src/target hosts) and trying again? > > Retest by downgrading some pkgs, here are additional test scenarios: > a)works well on : > libvirt-3.1.0-1.el7.x86_64 > virt-viewer-2.0-13.el7.x86_64 > spice-gtk3-0.31-8.el7.x86_64 > > b)issue can also reproduce on: > libvirt-3.1.0-1.el7.x86_64 > virt-viewer-5.0-2.el7.x86_64 > spice-gtk3-0.33-3.el7.x86_64 > > So the issue is caused by virt-viewer-5.0-2.el7.x86_64 and > spice-gtk3-0.33-3.el7.x86_64. the versions of virt-viewer and spice-gtk3 are > interdependent, can not individually downgrade one and leave another one > high version. > > Why need to downgrade spice to rhel-7.3 version(0.12.4-20) ? In 7.3 last version is 0.12.4-20, in 7.4 last version is 0.12.8-2 (today). The bump from 0.12.4 to 0.12.8 might have changed something too (but I'm skeptical now as this is not reproducible with remote-viewer, only with virt-viewer). The migration messages comes from spice. > btw:also test with '--direct', no help, still reproduce. Thanks > If possible, can you enable debug using SPICE_DEBUG=1 ? I usually do that by > including export SPICE_DEBUG=1 in a custom qemu-kvm script. I tried with "export SPICE_DEBUG=1" in terminal, or add following xml in domain: # virsh dumpxml V|grep kvm <domain type='kvm' id='12' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> ... <qemu:commandline> <qemu:env name='SPICE_DEBUG' value='1'/> </qemu:commandline> can see "SPICE_DEBUG=1" was set successfully in the qemu log(refer to new attachment "SPICE_DEBUG_qemu_log"), but still no any SPICE_DEBUG messages . Could you help to check? > In 7.3 last version is 0.12.4-20, in 7.4 last version is 0.12.8-2 (today). > The bump from 0.12.4 to 0.12.8 might have changed something too (but I'm > skeptical now as this is not reproducible with remote-viewer, only with > virt-viewer). > As in commment 8, all rhel7.4 scenarios were tested with the same spice-server-0.12.8-1.el7.x86_64, if as following, virt-viewer will keep connection well while seamless migration: virt-viewer-2.0-13.el7.x86_64 spice-gtk3-0.31-8.el7.x86_64 if as following, issue reproduces: virt-viewer-5.0-2.el7.x86_64 spice-gtk3-0.33-3.el7.x86_64 Both pass and fail result can occur on the same spice-server-0.12.8, so, I don't think need to downgrade to spice-server-0.12.4-20. Hi,
it regression in virt-viewer. I have bisected it to:
first bad commit: [a62827d28c6b69e90102e4c1c8043cbddad8929a] virt-viewer: ensure we close when seeing domain stop event
commit a62827d28c6b69e90102e4c1c8043cbddad8929a
Author: Daniel P. Berrange <berrange>
Date: Tue Jul 12 11:27:09 2016 +0100
virt-viewer: ensure we close when seeing domain stop event
Normally virt-viewer relies on the VNC/SPICE widget seeing
an EOF on its underlying connection to detect when the
session is closed.
When tunnelling to a remote guest over SSH though, this
EOF can be delayed for a very long time, leaving a dead
session open.
This can be seen with
virt-viewer -c qemu+ssh://remotehost/system guestname
when on the remote shell run
virsh destroy guestname
and notice that virt-viewer does not see the shutdown
immediately.
When we get a domain stopped event we know the session
should be dead, so forceably close it, if not already
closed.
Signed-off-by: Daniel P. Berrange <berrange>
I can reproduce it with virt-viewer-5.0-2.el7.x86_64. then i verified it with virt-viewer-5.0-4.el7.x86_64, virt-viewer can still keep connecting after migration. so move the bug from ON_QA to VERIFIED. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:1849 |
Description of problem: virt-viewer cannot keep connection while seamless migration Version-Release number of selected component (if applicable): virt-viewer-5.0-2.el7.x86_64 spice-gtk3-0.33-3.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.prepare migration env, and a domain with images on nfs storage 2.Dispatch ssh public key of source host to target host. # ssh-keygen -t rsa # ssh-copy-id -i ~/.ssh/id_rsa.pub root@{target_ip} 3.Define a domain with default spice graphic on source host. ... <graphics type='spice' autoport='yes'> <listen type='address'/> </graphics> … 4.add the following line in /etc/libvirt/qemu.conf both in source host and target host, and restart libvirtd # echo 'spice_listen = "0.0.0.0"' >> /etc/libvirt/qemu.conf # systemctl restart libvirtd 5.Start the domain on source host # virsh start V Domain V started # virsh dumpxml V|grep graphic -A3 <graphics type='spice' port='5900' autoport='yes' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics> ... 6.Use virt-viewer to connect the domain on source host. # virt-viewer V --debug --spice-debug 7.Migrate the domain to destination host and check virt-viewer connection # virsh migrate V qemu+ssh://{target_ip}/system --live --unsafe --verbose Migration: [100 %] virt-viewer disconnect after migration. # cat seamless-spice-debug.log|grep WARNING (virt-viewer:7106): Spice-WARNING **: channel-display-gst.c:303:create_pipeline: GStreamer error: no element "avdec_h264" (virt-viewer:7106): Gtk-WARNING **: Allocating size to SpiceDisplay 0x55b5489a63c0 without calling gtk_widget_get_preferred_width/height(). How does the code know the size to allocate? (virt-viewer:7106): Spice-WARNING **: channel-display-gst.c:303:create_pipeline: GStreamer error: no element "avdec_h264" (virt-viewer:7106): Spice-WARNING **: channel-display-gst.c:303:create_pipeline: GStreamer error: no element "avdec_h264" (virt-viewer:7106): Spice-WARNING **: channel-display-gst.c:303:create_pipeline: GStreamer error: no element "avdec_h264" Actual results: As in step7, virt-viewer cannot keep connection while seamless migration Expected results: virt-viewer should keep connection while seamless migration Additional info: "remote-viewer spice://{source_ip}:5900" works well