An integer overflow vulnerability in gtk+2.0 and gtk+3.0 in image handling was found when supplying large pixbufs to gdk_cairo_set_source_pixbuf, gdkcairo tries to allocate an immense number of bytes.
Created gtk2 tracking bugs for this issue:
Affects: fedora-all [bug 1306682]
Created gtk3 tracking bugs for this issue:
Affects: fedora-all [bug 1306683]
This is basically a DoS flaw when opening a crafted image. When g_malloc() is asked to malloc huge amount of memory, and it is unable to do it, it crashes with a g_error() amount similar to:
(eog:18754): GLib-ERROR **: /build/buildd/glib2.0-2.36.0/./glib/gmem.c:165: failed to allocate 18446744071914584320 bytes
Also it looks like gtk3 is not affected by this flaw, since the code there is completely different.
In gdk_cairo_set_source_pixbuf() rather than doing:
cairo_pixels = g_malloc (height * cairo_stride); and creating a "surface" after that by using cairo_image_surface_create_for_data() what gtk3 does is that it uses the cairo function cairo_surface_create_similar_image() to create the surface.
cairo_surface_create_similar_image() uses pixman_image_create() which calculates the size of the buffer needed to hold the image.
pixman_image_create() handles it correctly by using the following code:
pixels = FbPixelsCreate (width, height, format->depth);
if (pixels == NULL)