Bug 692048

Created attachment 488731 [details]
/tmp/vncserver.log

Reproduced and attached logs.

Created attachment 488732 [details]
/tmp/anaconda.log

Created attachment 488733 [details]
/tmp/program.log

What command did you use to try to connect?

(In reply to comment #4)
> What command did you use to try to connect?

PASS - $ vncviewer test106.test.redhat.com:1
PASS - $ vncviewer test106.test.redhat.com:5901
FAIL - $ vinagre test106.test.redhat.com:1
FAIL - $ vinagre test106.test.redhat.com:5901

Very strange, perhaps a bug in vinagre?

(In reply to comment #4)
> What command did you use to try to connect?

Interesting ...
$ strace -fe connect vncviewer test106.test.redhat.com:1

connect(3, {sa_family=AF_FILE, path=@"/tmp/.X11-unix/X0"}, 20) = 0
connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.11.255.156")}, 16) = 0
connect(4, {sa_family=AF_INET, sin_port=htons(5901), sin_addr=inet_addr("10.10.8.106")}, 16) = 0


$ strace -fe connect vinagre test106.test.redhat.com:1

connect(18, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.11.255.156")}, 16) = 0
connect(18, {sa_family=AF_INET, sin_port=htons(5901), sin_addr=inet_addr("10.10.8.106")}, 16) = -1 EINPROGRESS (Operation now in progress)


What's with the "operation now in progress"?

From vncserver.log:

Wed Mar 30 10:19:15 2011
 Connections: accepted: 10.66.12.141::54796
 SConnection: Client needs protocol version 3.8
 SConnection: Client requests security type VeNCrypt(19)
 Connections: closed: 10.66.12.141::54796 (Clean disconnection)

Adding to Beta nice-to-have list, adding berrange to the cc list per suggestion from Bastien.

Please capture a full strace log of vinagre, not simply the "connect" syscall. Also run vinagre with '--gtk-vnc-debug' flag set and capture that log.


> connect(18, {sa_family=AF_INET, sin_port=htons(5901),
> <sin_addr=inet_addr("10.10.8.106")}, 16) = -1 EINPROGRESS (Operation now in
> progress)
>
> What's with the "operation now in progress"?

That is perfectly normal when the socket FD has O_NONBLOCK enabled.

Created attachment 489779 [details]
vinagre strace output

(In reply to comment #9)
> Please capture a full strace log of vinagre, not simply the "connect" syscall.

See attached

Created attachment 489781 [details]
vinagre --gtk-vnc-debug

(In reply to comment #9)
> Also run vinagre with '--gtk-vnc-debug' flag set and capture that log.

See attached

The VNC server is advertizing two authentication types

> (vinagre:6529): gvnc-DEBUG: vncconnection.c Possible auth 19
> (vinagre:6529): gvnc-DEBUG: vncconnection.c Possible auth 1

19 == VENCRYPT,  1 == VNC

GTK-VNC is choosing VENCRYPT since that's a stronger auth type. The legacy vncviewer will be choosing VNC auth.

Once VENCRYPT is chosen, we ask for sub-auth types to which the server replies

(vinagre:6529): gvnc-DEBUG: vncconnection.c Possible auth 1

This is *not* a valid sub-auth type for VENCRYPT - we're expecting one of 


typedef enum {
	VNC_CONNECTION_AUTH_VENCRYPT_PLAIN = 256,
	VNC_CONNECTION_AUTH_VENCRYPT_TLSNONE = 257,
	VNC_CONNECTION_AUTH_VENCRYPT_TLSVNC = 258,
	VNC_CONNECTION_AUTH_VENCRYPT_TLSPLAIN = 259,
	VNC_CONNECTION_AUTH_VENCRYPT_X509NONE = 260,
	VNC_CONNECTION_AUTH_VENCRYPT_X509VNC = 261,
	VNC_CONNECTION_AUTH_VENCRYPT_X509PLAIN = 262,
	VNC_CONNECTION_AUTH_VENCRYPT_X509SASL = 263,
	VNC_CONNECTION_AUTH_VENCRYPT_TLSSASL = 264,
} VncConnectionAuthVencrypt;

And thus GTK-VNC kills the connection:

(vinagre:6529): gtk-vnc-DEBUG: vncdisplay.c No preferred auth subtype found

So I believe this is a VNC server bug.

What is the VNC server being connected to ?

(In reply to comment #12)
> So I believe this is a VNC server bug.
> 
> What is the VNC server being connected to ?

Looking at /usr/bin/Xvnc from inside the installer environment, it looks like tigerVNC?

# Xvnc --help 2>&1 | grep -i tiger
Xvnc TigerVNC 1.0.90 - built Mar 23 2011 12:06:08
See http://www.tigervnc.org for information on TigerVNC.

Ok, I've found the bug in TigerVNC. In common/rfb/Security.cxx

There are two methods, one for getting the primary auth type, and one for the secondary auth type:

const std::list<rdr::U8> Security::GetEnabledSecTypes(void)
{
  list<rdr::U8> result;
  list<U32>::iterator i;

  result.push_back(secTypeVeNCrypt);
  for (i = enabledSecTypes.begin(); i != enabledSecTypes.end(); i++)
    if (*i < 0x100)
      result.push_back(*i);

  return result;
}

const std::list<rdr::U32> Security::GetEnabledExtSecTypes(void)
{
  list<rdr::U32> result;
  list<U32>::iterator i;

  for (i = enabledSecTypes.begin(); i != enabledSecTypes.end(); i++)
    if (*i != secTypeVeNCrypt) /* Do not include VeNCrypt type to avoid loops */
      result.push_back(*i);

  return result;
}


As can be seen here, the data returned for the secondary auth type is just a copy of the primary auth type, but with VeNCrypt skipped.

This is broken, and not compliant with the VeNCrypt protocol 0.2 which TigerVNC claims to support.

[quote src=http://berrange.com/~dan/vencrypt.txt]
The server sends a U8 listing the number of sub-types supported.  If
this is zero, the connection terminates, otherwise it is followed by the
sub-types it supports/permits as U32s:
No. of bytes    Type    [Value]         Description
1               U8	[n]             Number of supported sub-types
4*n             U32 array               Supported sub-types

The sub-types are as follows:
0: Failure
256: Plain
257: TLSNone
258: TLSVnc
259: TLSPlain
260: X509None
261: X509Vnc
262: X509Plain
[/quote]

TigerVNC is just sending back an auth type==1, or subtype==2, which is out of spec, and doesn't even make sense because there's no way for the client to know if the server wanted anonymous of x509 TLS credentials.

(In reply to comment #14)
> This is broken, and not compliant with the VeNCrypt protocol 0.2 which TigerVNC
> claims to support.
> 
> [quote src=http://berrange.com/~dan/vencrypt.txt]
> The server sends a U8 listing the number of sub-types supported.  If
> this is zero, the connection terminates, otherwise it is followed by the
> sub-types it supports/permits as U32s:
> No. of bytes    Type    [Value]         Description
> 1               U8 [n]             Number of supported sub-types
> 4*n             U32 array               Supported sub-types
> 
> The sub-types are as follows:
> 0: Failure
> 256: Plain
> 257: TLSNone
> 258: TLSVnc
> 259: TLSPlain
> 260: X509None
> 261: X509Vnc
> 262: X509Plain
> [/quote]
> 
> TigerVNC is just sending back an auth type==1, or subtype==2, which is out of
> spec, and doesn't even make sense because there's no way for the client to know
> if the server wanted anonymous of x509 TLS credentials.

It is quite tricky to claim "standard" VNC types (i.e. None or VncAuth) aren't VeNCrypt subtypes because original VeNCrypt authors haven't written any specification, yet. Btw original VeNCrypt designer, Martin Koegler, wrote [1] that standard types should be also considered as valid VeNCrypt subtypes.

However I understand current Xvnc in F15 can break older vinagre clients (RHEL, older Fedoras) so I will patch Xvnc to not advertise standard VNC types as VeNCrypt subtypes.

[1] http://www.mail-archive.com/tigervnc-devel@lists.sourceforge.net/msg00746.html

Through the analysis above, the bug is focused on TigerVNC which sends back an unexpect result to the VNC client.
But if it is only the reason of VNC server, why vinagre fails but vncviewer successes? Shall we check the vinagre too? 
Thanks a lot!

(In reply to comment #16)
> Through the analysis above, the bug is focused on TigerVNC which sends back an
> unexpect result to the VNC client.
> But if it is only the reason of VNC server, why vinagre fails but vncviewer
> successes? Shall we check the vinagre too? 

Problem is that both vinagre and TigerVNC developers (note that vncviewer is the TigerVNC viewer) have different explanation of the original VeNCrypt specification. Upstreams must discuss and decide what is "correct" and then either TigerVNC or vinagre should be fixed.

For now I will make TigerVNC server compatible with vinagre.

Actually, Martin Koegler did not design VeNCrypt - he provided something called 'TLS for VNC' which was not RFB compatible.  Stuart Becker took the concepts from this work and produced the RFB compatible extension, VeNCrypt.

  http://sibecker.co.uk/vencrypt.html

When implementing VeNCrypt in GTK-VNC/QEMU, Stuart wrote up a specification for it, which I reproduce here for convenience:

  http://berrange.com/~dan/vencrypt.txt

This lists the valid auth sub-types (NB they are different in v0.1 and v0.2). The range 0-255 was reserved, for possible future versions of the protocol, hence valid subtypes start at 256.

Reproduced in anaconda 15.27

(In reply to comment #17)
> (In reply to comment #16)
> > Through the analysis above, the bug is focused on TigerVNC which sends back an
> > unexpect result to the VNC client.
> > But if it is only the reason of VNC server, why vinagre fails but vncviewer
> > successes? Shall we check the vinagre too? 
> 
> Problem is that both vinagre and TigerVNC developers (note that vncviewer is
> the TigerVNC viewer) have different explanation of the original VeNCrypt
> specification. Upstreams must discuss and decide what is "correct" and then
> either TigerVNC or vinagre should be fixed.
> 
> For now I will make TigerVNC server compatible with vinagre.

So we do need take some time to discuss the original VeNCrypt specification in
the near future, but for now we'd better do something to make the vinagre works
well. Thanks Adam!

tigervnc-1.0.90-2.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/tigervnc-1.0.90-2.fc15

VERIFIED using custom built boot.iso (64b) built with tigervnc-1.0.90-2.fc15 and anaconda-15.27-1.  boot.iso available from http://jlaska.fedorapeople.org/boot-x86_64.iso

Discussed during the 2011-04-08 blocker review meeting [1].  AcceptedNTH for Beta, will take into RC2 if a VERIFIED fix is available

[1] http://meetbot.fedoraproject.org/fedora-bugzappers/2011-04-08/f-15-beta-blocker-review.2011-04-08-17.00.html

tigervnc-1.0.90-2.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

dropping commonbugs keyword as we took a fix for this.