This issue affects the versions of the vino package, as shipped
with Red Hat Enterprise Linux 4, 5, and 6.

This issue affects the versions of the vino package, as shipped
with Fedora release of 13 and 14.

--

This issue did NOT affect the version of the libvncserver package,
as shipped with Red Hat Enterprise Linux 6.

This issue did NOT affect the version of the libvncserver package,
as present within EPEL-5 repository.

This issue did NOT affect the versions of the libvncserver package,
as shipped with Fedora release of 13 and 14.

Note: See also comment c#17 of this bug report for further information
      / explanation of conclusions from the above paragraph.

--

This issue affects the versions of the kdenetwork package,
as shipped with Red Hat Enterprise Linux 4 and 5.

This issue did NOT affect the version of the kdenetwork package,
as shipped with Red Hat Enterprise Linux 6.

This issue did NOT affect the version of the kdenetwork package,
as shipped with Fedora release of 13 and 14.

Analysis from Petr Pisar (Red Hat libvncserver package maintainer) why various
libvncserver package versions, as shipped within different products are not
vulnerable to this flaw:


The libvncserver warning is produced in
rfbserver.c:rfbProcessClientNormalMessage() when converting rectangle
coordinates just after processing client packet of rfbFramebufferUpdateRequest
type:

    case rfbFramebufferUpdateRequest:
        [...]
        if ((n = rfbReadExact(cl, ((char *)&msg) + 1,
                           sz_rfbFramebufferUpdateRequestMsg-1)) <= 0) {
            [...]
        }
        [...]
        /* The values come in based on the scaled screen, we need to convert
         * them to values based on the main screen's coordinate system
         */
       
if(!rectSwapIfLEAndClip(&msg.fur.x,&msg.fur.y,&msg.fur.w,&msg.fur.h,cl))
        {
                rfbLog("Warning, ignoring rfbFramebufferUpdateRequest: "
                    "%dXx%dY-%dWx%dH\n",
                    msg.fur.x, msg.fur.y, msg.fur.w, msg.fur.h);
                return;
        }
        [...]

Not only the rectSwapIfLEAndClip() function converts values from network-order
and scales the coordinates, it also clips requested rectangle (msg.fur members)
to fit into server screen size:

rfbBool rectSwapIfLEAndClip(uint16_t* x,uint16_t* y,uint16_t* w,uint16_t* h,
                rfbClientPtr cl)
{
        /* Convert */
        int x1=Swap16IfLE(*x);
        int y1=Swap16IfLE(*y);
        int w1=Swap16IfLE(*w);
        int h1=Swap16IfLE(*h);

        /* Scale */
        rfbScaledCorrection(cl->scaledScreen, cl->screen, &x1, &y1, &w1, &h1,
            "rectSwapIfLEAndClip");
        *x = x1;
        *y = y1;
        *w = w1;
        *h = h1;

        /* Clip
        rfbLog("Clipping: region(%uXx%uY-%uWx%uH) to screen(%dWx%dH)\n",
                *x, *y, *w, *h,
                cl->screen->width, cl->screen->height);
        */
        if(*w>cl->screen->width-*x)
                *w=cl->screen->width-*x;
        /* possible underflow */
        if(*w>cl->screen->width-*x)
                return FALSE;
        if(*h>cl->screen->height-*y)
                *h=cl->screen->height-*y;
        if(*h>cl->screen->height-*y)
                return FALSE;

        return TRUE;
}

And see the comment /* possible underflow */ (other comments added by me). It
checks whether subtraction result fits into uint16_t. Otherwise it refuse the
conversion and warning is printed and client request is refused. This is the
output I get on server side with the Clipping log line:

Clipping: region(29696Xx32768Y-1Wx30975H) to screen(400Wx300H)
Warning, ignoring rfbFramebufferUpdateRequest: 29696Xx32768Y-36240Wx30975H

The Cliping values correspond to the run_00108.input3 packet content:
$ hexdump -C run_00108.input3
00000000  03 00 74 00 80 00 00 01  78 ff                    |..t.....x.|

03 00 is the request type probably
74 00 (29696) is X coordinate
80 00 (32768) is Y cooridnate
00 01 (1) is Width
78 ff (30975) is Height

Let's look on the clipping code closer and remember the *x and *w are uint16_t:
if(*w>cl->screen->width-*x)
        *w=cl->screen->width-*x;
/* possible underflow */
if(*w>cl->screen->width-*x)
        return FALSE;

If X fits into screen width, then substraction result is not negative and
request width is clamped properly.

If X is out of screen width, then subtraction underruns and *W will be
garbled (as can be seen in my output, 1W → 36240W), but still non-negative
(because unsigned type). However subsequent underrun test will compare garbled
non-negative value to negative subtraction result (the same as former one)
resulting is true condition and causing `return FALSE'.

The same applies to vertical clipping.

Thus I conclude LIBVNCSERVER IS NOT VULNERABLE because it performs the range
check on this place. One can see it's the same equation as in current patch:

+   /* Validate the rectangle given by the update packet. */
+   if (w + x > cl->scaledScreen->width || h + y > cl->scaledScreen->height)
+       goto updateFailed;


-- Petr

Created vino tracking bugs for this issue

Affects: fedora-all [bug 701712]

Statement:

(none)

This issue has been addressed in following products:

  Red Hat Enterprise Linux 6

Via RHSA-2013:0169 https://rhn.redhat.com/errata/RHSA-2013-0169.html