Bug 692048

Summary: Fail to connect to the specified address with VNC client vinagre when using vnc method in anaconda 15.25
Product: [Fedora] Fedora Reporter: Tao Wu <twu>
Component: tigervncAssignee: Adam Tkac <atkac>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 15CC: atkac, awilliam, berrange, bnocera, jlaska, jonathan, mclasen, ovasik, rhe, sdharane, vanmeeuwen+fedora
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: AcceptedNTH
Fixed In Version: tigervnc-1.0.90-2.fc15 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-04-08 22:13:50 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 657619    
Attachments:
Description Flags
Connection to host was closed
none
/tmp/vncserver.log
none
/tmp/anaconda.log
none
/tmp/program.log
none
vinagre strace output
none
vinagre --gtk-vnc-debug none

Description Tao Wu 2011-03-30 05:51:15 EDT
Created attachment 488722 [details]
Connection to host was closed

Description of problem:
In an installation using the graphical installer over VNC, the VNC client vinagre can not connect to the specified address given in the anaconda.

Version-Release number of selected component (if applicable):
anaconda 15.25

How reproducible:
always

Steps to Reproduce:
1.Boot the installer with the command-line option vnc 
2.Connect with VNC vinagre to the specified address
  
Actual results:
No graphical installation is shown and it prompts: 'Connection to host xxxxx was closed'.

Expected results:
The graphical installation is shown.

Additional info:
Comment 1 He Rui 2011-03-30 06:25:48 EDT
Created attachment 488731 [details]
/tmp/vncserver.log

Reproduced and attached logs.
Comment 2 He Rui 2011-03-30 06:26:19 EDT
Created attachment 488732 [details]
/tmp/anaconda.log
Comment 3 He Rui 2011-03-30 06:27:12 EDT
Created attachment 488733 [details]
/tmp/program.log
Comment 4 Ales Kozumplik 2011-03-30 09:10:59 EDT
What command did you use to try to connect?
Comment 5 James Laska 2011-03-30 09:19:41 EDT
(In reply to comment #4)
> What command did you use to try to connect?

PASS - $ vncviewer test106.test.redhat.com:1
PASS - $ vncviewer test106.test.redhat.com:5901
FAIL - $ vinagre test106.test.redhat.com:1
FAIL - $ vinagre test106.test.redhat.com:5901

Very strange, perhaps a bug in vinagre?
Comment 6 James Laska 2011-03-30 09:22:56 EDT
(In reply to comment #4)
> What command did you use to try to connect?

Interesting ...
$ strace -fe connect vncviewer test106.test.redhat.com:1

connect(3, {sa_family=AF_FILE, path=@"/tmp/.X11-unix/X0"}, 20) = 0
connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.11.255.156")}, 16) = 0
connect(4, {sa_family=AF_INET, sin_port=htons(5901), sin_addr=inet_addr("10.10.8.106")}, 16) = 0


$ strace -fe connect vinagre test106.test.redhat.com:1

connect(18, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.11.255.156")}, 16) = 0
connect(18, {sa_family=AF_INET, sin_port=htons(5901), sin_addr=inet_addr("10.10.8.106")}, 16) = -1 EINPROGRESS (Operation now in progress)


What's with the "operation now in progress"?
Comment 7 Chris Lumens 2011-03-30 10:49:43 EDT
From vncserver.log:

Wed Mar 30 10:19:15 2011
 Connections: accepted: 10.66.12.141::54796
 SConnection: Client needs protocol version 3.8
 SConnection: Client requests security type VeNCrypt(19)
 Connections: closed: 10.66.12.141::54796 (Clean disconnection)
Comment 8 James Laska 2011-04-04 09:58:27 EDT
Adding to Beta nice-to-have list, adding berrange to the cc list per suggestion from Bastien.
Comment 9 Daniel Berrange 2011-04-04 10:01:53 EDT
Please capture a full strace log of vinagre, not simply the "connect" syscall. Also run vinagre with '--gtk-vnc-debug' flag set and capture that log.


> connect(18, {sa_family=AF_INET, sin_port=htons(5901),
> <sin_addr=inet_addr("10.10.8.106")}, 16) = -1 EINPROGRESS (Operation now in
> progress)
>
> What's with the "operation now in progress"?

That is perfectly normal when the socket FD has O_NONBLOCK enabled.
Comment 10 James Laska 2011-04-04 10:07:41 EDT
Created attachment 489779 [details]
vinagre strace output

(In reply to comment #9)
> Please capture a full strace log of vinagre, not simply the "connect" syscall.

See attached
Comment 11 James Laska 2011-04-04 10:08:29 EDT
Created attachment 489781 [details]
vinagre --gtk-vnc-debug

(In reply to comment #9)
> Also run vinagre with '--gtk-vnc-debug' flag set and capture that log.

See attached
Comment 12 Daniel Berrange 2011-04-04 10:20:59 EDT
The VNC server is advertizing two authentication types

> (vinagre:6529): gvnc-DEBUG: vncconnection.c Possible auth 19
> (vinagre:6529): gvnc-DEBUG: vncconnection.c Possible auth 1

19 == VENCRYPT,  1 == VNC

GTK-VNC is choosing VENCRYPT since that's a stronger auth type. The legacy vncviewer will be choosing VNC auth.

Once VENCRYPT is chosen, we ask for sub-auth types to which the server replies

(vinagre:6529): gvnc-DEBUG: vncconnection.c Possible auth 1

This is *not* a valid sub-auth type for VENCRYPT - we're expecting one of 


typedef enum {
	VNC_CONNECTION_AUTH_VENCRYPT_PLAIN = 256,
	VNC_CONNECTION_AUTH_VENCRYPT_TLSNONE = 257,
	VNC_CONNECTION_AUTH_VENCRYPT_TLSVNC = 258,
	VNC_CONNECTION_AUTH_VENCRYPT_TLSPLAIN = 259,
	VNC_CONNECTION_AUTH_VENCRYPT_X509NONE = 260,
	VNC_CONNECTION_AUTH_VENCRYPT_X509VNC = 261,
	VNC_CONNECTION_AUTH_VENCRYPT_X509PLAIN = 262,
	VNC_CONNECTION_AUTH_VENCRYPT_X509SASL = 263,
	VNC_CONNECTION_AUTH_VENCRYPT_TLSSASL = 264,
} VncConnectionAuthVencrypt;

And thus GTK-VNC kills the connection:

(vinagre:6529): gtk-vnc-DEBUG: vncdisplay.c No preferred auth subtype found

So I believe this is a VNC server bug.

What is the VNC server being connected to ?
Comment 13 James Laska 2011-04-04 10:59:50 EDT
(In reply to comment #12)
> So I believe this is a VNC server bug.
> 
> What is the VNC server being connected to ?

Looking at /usr/bin/Xvnc from inside the installer environment, it looks like tigerVNC?

# Xvnc --help 2>&1 | grep -i tiger
Xvnc TigerVNC 1.0.90 - built Mar 23 2011 12:06:08
See http://www.tigervnc.org for information on TigerVNC.
Comment 14 Daniel Berrange 2011-04-04 11:40:39 EDT
Ok, I've found the bug in TigerVNC. In common/rfb/Security.cxx

There are two methods, one for getting the primary auth type, and one for the secondary auth type:

const std::list<rdr::U8> Security::GetEnabledSecTypes(void)
{
  list<rdr::U8> result;
  list<U32>::iterator i;

  result.push_back(secTypeVeNCrypt);
  for (i = enabledSecTypes.begin(); i != enabledSecTypes.end(); i++)
    if (*i < 0x100)
      result.push_back(*i);

  return result;
}

const std::list<rdr::U32> Security::GetEnabledExtSecTypes(void)
{
  list<rdr::U32> result;
  list<U32>::iterator i;

  for (i = enabledSecTypes.begin(); i != enabledSecTypes.end(); i++)
    if (*i != secTypeVeNCrypt) /* Do not include VeNCrypt type to avoid loops */
      result.push_back(*i);

  return result;
}


As can be seen here, the data returned for the secondary auth type is just a copy of the primary auth type, but with VeNCrypt skipped.

This is broken, and not compliant with the VeNCrypt protocol 0.2 which TigerVNC claims to support.

[quote src=http://berrange.com/~dan/vencrypt.txt]
The server sends a U8 listing the number of sub-types supported.  If
this is zero, the connection terminates, otherwise it is followed by the
sub-types it supports/permits as U32s:
No. of bytes    Type    [Value]         Description
1               U8	[n]             Number of supported sub-types
4*n             U32 array               Supported sub-types

The sub-types are as follows:
0: Failure
256: Plain
257: TLSNone
258: TLSVnc
259: TLSPlain
260: X509None
261: X509Vnc
262: X509Plain
[/quote]

TigerVNC is just sending back an auth type==1, or subtype==2, which is out of spec, and doesn't even make sense because there's no way for the client to know if the server wanted anonymous of x509 TLS credentials.
Comment 15 Adam Tkac 2011-04-06 07:24:39 EDT
(In reply to comment #14)
> This is broken, and not compliant with the VeNCrypt protocol 0.2 which TigerVNC
> claims to support.
> 
> [quote src=http://berrange.com/~dan/vencrypt.txt]
> The server sends a U8 listing the number of sub-types supported.  If
> this is zero, the connection terminates, otherwise it is followed by the
> sub-types it supports/permits as U32s:
> No. of bytes    Type    [Value]         Description
> 1               U8 [n]             Number of supported sub-types
> 4*n             U32 array               Supported sub-types
> 
> The sub-types are as follows:
> 0: Failure
> 256: Plain
> 257: TLSNone
> 258: TLSVnc
> 259: TLSPlain
> 260: X509None
> 261: X509Vnc
> 262: X509Plain
> [/quote]
> 
> TigerVNC is just sending back an auth type==1, or subtype==2, which is out of
> spec, and doesn't even make sense because there's no way for the client to know
> if the server wanted anonymous of x509 TLS credentials.

It is quite tricky to claim "standard" VNC types (i.e. None or VncAuth) aren't VeNCrypt subtypes because original VeNCrypt authors haven't written any specification, yet. Btw original VeNCrypt designer, Martin Koegler, wrote [1] that standard types should be also considered as valid VeNCrypt subtypes.

However I understand current Xvnc in F15 can break older vinagre clients (RHEL, older Fedoras) so I will patch Xvnc to not advertise standard VNC types as VeNCrypt subtypes.

[1] http://www.mail-archive.com/tigervnc-devel@lists.sourceforge.net/msg00746.html
Comment 16 Tao Wu 2011-04-07 06:25:58 EDT
Through the analysis above, the bug is focused on TigerVNC which sends back an unexpect result to the VNC client.
But if it is only the reason of VNC server, why vinagre fails but vncviewer successes? Shall we check the vinagre too? 
Thanks a lot!
Comment 17 Adam Tkac 2011-04-07 07:30:06 EDT
(In reply to comment #16)
> Through the analysis above, the bug is focused on TigerVNC which sends back an
> unexpect result to the VNC client.
> But if it is only the reason of VNC server, why vinagre fails but vncviewer
> successes? Shall we check the vinagre too? 

Problem is that both vinagre and TigerVNC developers (note that vncviewer is the TigerVNC viewer) have different explanation of the original VeNCrypt specification. Upstreams must discuss and decide what is "correct" and then either TigerVNC or vinagre should be fixed.

For now I will make TigerVNC server compatible with vinagre.
Comment 18 Daniel Berrange 2011-04-07 07:50:20 EDT
Actually, Martin Koegler did not design VeNCrypt - he provided something called 'TLS for VNC' which was not RFB compatible.  Stuart Becker took the concepts from this work and produced the RFB compatible extension, VeNCrypt.

  http://sibecker.co.uk/vencrypt.html

When implementing VeNCrypt in GTK-VNC/QEMU, Stuart wrote up a specification for it, which I reproduce here for convenience:

  http://berrange.com/~dan/vencrypt.txt

This lists the valid auth sub-types (NB they are different in v0.1 and v0.2). The range 0-255 was reserved, for possible future versions of the protocol, hence valid subtypes start at 256.
Comment 19 He Rui 2011-04-08 02:36:43 EDT
Reproduced in anaconda 15.27
Comment 20 Tao Wu 2011-04-08 06:09:04 EDT
(In reply to comment #17)
> (In reply to comment #16)
> > Through the analysis above, the bug is focused on TigerVNC which sends back an
> > unexpect result to the VNC client.
> > But if it is only the reason of VNC server, why vinagre fails but vncviewer
> > successes? Shall we check the vinagre too? 
> 
> Problem is that both vinagre and TigerVNC developers (note that vncviewer is
> the TigerVNC viewer) have different explanation of the original VeNCrypt
> specification. Upstreams must discuss and decide what is "correct" and then
> either TigerVNC or vinagre should be fixed.
> 
> For now I will make TigerVNC server compatible with vinagre.

So we do need take some time to discuss the original VeNCrypt specification in
the near future, but for now we'd better do something to make the vinagre works
well. Thanks Adam!
Comment 21 Fedora Update System 2011-04-08 09:29:59 EDT
tigervnc-1.0.90-2.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/tigervnc-1.0.90-2.fc15
Comment 22 James Laska 2011-04-08 15:07:28 EDT
VERIFIED using custom built boot.iso (64b) built with tigervnc-1.0.90-2.fc15 and anaconda-15.27-1.  boot.iso available from http://jlaska.fedorapeople.org/boot-x86_64.iso
Comment 23 James Laska 2011-04-08 15:16:26 EDT
Discussed during the 2011-04-08 blocker review meeting [1].  AcceptedNTH for Beta, will take into RC2 if a VERIFIED fix is available

[1] http://meetbot.fedoraproject.org/fedora-bugzappers/2011-04-08/f-15-beta-blocker-review.2011-04-08-17.00.html
Comment 24 Fedora Update System 2011-04-08 22:13:45 EDT
tigervnc-1.0.90-2.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 25 Adam Williamson 2011-04-14 21:02:04 EDT
dropping commonbugs keyword as we took a fix for this.