Bug 886686 - vnc tunneling is broken
vnc tunneling is broken
Status: CLOSED WORKSFORME
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: gtk-vnc (Show other bugs)
6.4
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Daniel Berrange
Desktop QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-12-12 16:27 EST by Gerd Hoffmann
Modified: 2017-02-08 09:45 EST (History)
16 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-02-08 09:45:54 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
virt viewer logfile (266.54 KB, text/plain)
2012-12-13 08:28 EST, Gerd Hoffmann
no flags Details
qemu log file (6.14 KB, text/plain)
2012-12-13 10:49 EST, Gerd Hoffmann
no flags Details

  None (edit)
Description Gerd Hoffmann 2012-12-12 16:27:01 EST
Description of problem:
$subject

Version-Release number of selected component (if applicable):
virt-viewer-0.5.2-17.el6.x86_64
libvirt-0.10.2-11.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1. connect with virt-viewer to a linux guest, using --attach
2. trigger lots of text mode scrolling
2a. switch to a text console, login, dmesg will do.
2b. remove quiet + rhgb from the kernel command line + reboot works too.
  
Actual results:
virt-viewer behaves in strange ways.  segfaults, hangs, fatal glib errors, all seen.

Expected results:
virt-viewer works fine.

Additional info:
virt-viewer --direct doesn't show those problems,
thats why I suspect the libvirt tunneling being the culprit.
Comment 2 Jiri Denemark 2012-12-13 07:50:35 EST
AFAICT, libvirt is not involved in any kind of tunnelling done by virt-viewer. Thus, I'm changing component to virt-viewer. However, since --attach results in virDomainOpenGraphics, which just calls add_client monitor command, it's possible the problem lies somewhere in that area of QEMU.
Comment 3 Daniel Berrange 2012-12-13 08:15:29 EST
Can you re-run virt-viewer with the  --debug and --gtk-vnc-debug flags and capture both stdout + stderr to one file so it shows the errors included with the logs
Comment 4 Gerd Hoffmann 2012-12-13 08:28:02 EST
(In reply to comment #2)
> AFAICT, libvirt is not involved in any kind of tunnelling done by
> virt-viewer.

For remote connections?  Doesn't it tunnel using libvirt streams then?
Comment 5 Gerd Hoffmann 2012-12-13 08:28:57 EST
Created attachment 662966 [details]
virt viewer logfile
Comment 6 Daniel Berrange 2012-12-13 08:34:39 EST
(In reply to comment #4)
> For remote connections?  Doesn't it tunnel using libvirt streams then?

No, virt-viewer launches its own SSH tunnel to the remote host.
Comment 7 Daniel Berrange 2012-12-13 08:36:10 EST
Looks like the problem is handling of re-connecting to a guest when it restarts 

(virt-viewer:28349): gvnc-DEBUG: vncconnection.c Requesting framebuffer update at 0,0 size 720x400, incremental 1
(virt-viewer:28349): gvnc-DEBUG: vncconnection.c Closing the connection: vnc_connection_flush 32
(virt-viewer:28349): gvnc-DEBUG: vncconnection.c Aborting message processing on error
(virt-viewer:28349): gvnc-DEBUG: vncconnection.c Doing final VNC cleanup
(virt-viewer:28349): gvnc-DEBUG: vncconnection.c Close VncConnection=0xcc3fe0
(virt-viewer:28349): gvnc-DEBUG: vncconnection.c Emit main context 14
(virt-viewer:28349): gtk-vnc-DEBUG: vncdisplay.c Disconnected from VNC server

(virt-viewer:28349): Gtk-WARNING **: Attempting to add a widget with type VncDisplay to a container of type VirtViewerDisplayVnc, but the widget is already inside a container of type VirtViewerDisplayVnc, the GTK+ FAQ at http://library.gnome.org/devel/gtk-faq/stable/ explains how to reparent a widget.
** (virt-viewer:28349): DEBUG: Disconnected
** (virt-viewer:28349): DEBUG: close vnc=0xb9a1b0
(virt-viewer:28349): gvnc-DEBUG: vncconnection.c Init VncConnection=0xdf17f0
(virt-viewer:28349): gtk-vnc-DEBUG: vncdisplaykeymap.c Using evdev keycode mapping
** (virt-viewer:28349): DEBUG: notebook show status 0xb221d0
** (virt-viewer:28349): DEBUG: Guest rhel6-org-virtio display has disconnected, shutting down
** (virt-viewer:28349): DEBUG: Disposing window 0xb238a0

(virt-viewer:28349): gtk-vnc-DEBUG: vncdisplay.c Display destroy, requesting that VNC connection close
(virt-viewer:28349): gtk-vnc-DEBUG: vncdisplay.c Releasing VNC widget
(virt-viewer:28349): gvnc-DEBUG: vncconnection.c Finalize VncConnection=0xdf17f0
** (virt-viewer:28349): DEBUG: Set connect info: (null),(null),(null),-1,(null),(null),(null),0
Comment 8 Gerd Hoffmann 2012-12-13 09:33:19 EST
(In reply to comment #7)
> Looks like the problem is handling of re-connecting to a guest when it
> restarts 

There is no guest restart involved.
Comment 9 Daniel Berrange 2012-12-13 09:38:13 EST
Hmm, is QEMU dropping the connection then, because the logs show that the VNC server connection is closing. What is in /var/log/libvirt/qemu/GUEST.log if anything ?
Comment 10 Gerd Hoffmann 2012-12-13 10:49:35 EST
Created attachment 663022 [details]
qemu log file

With "#define _VNC_DEBUG 1"

So, yes, qemu closes the connection because it sees a protocol error.
Comment 11 Daniel Berrange 2012-12-13 11:06:31 EST
Hmm, that's interesting, I've not seen this kind of problem with gtk-vnc before.

That said, we did just have a coverity report which identified an issue with gtk-vnc's send buffer handling, which could conceivably cause something like this. I was not entirely convinced it was possible actually hit the problem, but its the only idea I have thus far for this behaviour. If you have toime, it'd be helpful if you could try applying the following patch from gtk-vnc GIT upstream to your RHEL build of GTK-VNC  (git://git.gnome.org/gtk-vnc)

  commit 10ce6d8f96e650d0b946622db22baac2fcfaadfb
  Author: Daniel P. Berrange <berrange@redhat.com>
  Date:   Fri Dec 7 15:38:55 2012 +0000

    Fix size check on available write buffer space
    
    The vnc_connection_write() function did not consider the current
    write offset when calculating how much space was available in
    the write buffer. This could lead to an array overrun.
Comment 13 RHEL Product and Program Management 2012-12-18 01:48:58 EST
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.
Comment 14 tingting zheng 2012-12-31 04:20:10 EST
Tried to reproduce the issue on my machine,but failed to reproduce:
Steps:
1.Use virt-viewer --attach to connect a rhel6 guest:
#virt-viewer --attach -c qemu+ssh://localhost/system test
2.Trigger lots of text mode scrolling in host.I guess step2 is executed on host instead of guest,as comment 8 shows.However I have tried the step on both host and guest,still faild to reproduce.

As steps show,virt-viewer works well,is there anything I have missed?

Also I found that when use virt-viewer --attach to connect a guest,libvirtd will crash after that.
# service libvirtd status
libvirtd dead but pid file exists

And there are lots of info in libvirtd.log:Caught Segmentation violation dumping internal log buffer.
Comment 15 Gerd Hoffmann 2013-01-03 03:07:15 EST
(In reply to comment #14)
> Tried to reproduce the issue on my machine,but failed to reproduce:
> Steps:
> 1.Use virt-viewer --attach to connect a rhel6 guest:
> #virt-viewer --attach -c qemu+ssh://localhost/system test
> 2.Trigger lots of text mode scrolling in host.I guess step2 is executed on
> host instead of guest,as comment 8 shows.However I have tried the step on
> both host and guest,still faild to reproduce.

Step 2 should be executed in the guest, on a text console (not an xterm or gnome-terminal).
Comment 16 tingting zheng 2013-01-03 21:57:09 EST
(In reply to comment #15)
> (In reply to comment #14)
> > Tried to reproduce the issue on my machine,but failed to reproduce:
> > Steps:
> > 1.Use virt-viewer --attach to connect a rhel6 guest:
> > #virt-viewer --attach -c qemu+ssh://localhost/system test
> > 2.Trigger lots of text mode scrolling in host.I guess step2 is executed on
> > host instead of guest,as comment 8 shows.However I have tried the step on
> > both host and guest,still faild to reproduce.
> 
> Step 2 should be executed in the guest, on a text console (not an xterm or
> gnome-terminal).

I still can not reproduce this issue:
version:
virt-viewer-0.5.2-17.el6.x86_64
libvirt-0.10.2-11.el6.x86_64

steps:
1.Use virt-viewer --attach to connect a rhel6 guest:
#virt-viewer --attach -c qemu+ssh://localhost/system test
2.In guest,click Send Key and choose Ctrl+ALt+F2,login the guest,and type dmesg.
3.Also I tried in guest,type init 3 in console,then login and type dmesg.

virt-viewer still works well.
Comment 20 Christophe Fergeau 2014-12-12 08:19:15 EST
Gerd, are you still hitting that bug?
Comment 21 Gerd Hoffmann 2014-12-15 02:03:23 EST
Yes.
Comment 22 Daniel Berrange 2017-02-08 09:45:54 EST
Never been able to reproduce the problems in this bug, so impossible to say what might be broken. The code upstream has changed hugely since the version of gtk-vnc in rhel-6, and no equivalent bug has been reported against rhel-7, so assuming this is something obscure in the old codebase.

Note You need to log in before you can comment on or make changes to this bug.