A heap overflow and a DoS in the gdk-pixbuf implementation triggered by the scaling of a malformed bmp was found.
Quoting the reporter:
The issue happens when a program is trying to parse and scale a crafted bmp using gdk-pixbuf (for instance, using get_scaled_pixbuf). It will result at least in an DoS aborting the program with a SIGTRAP. It is also possible to perform a heap overflow if you select the suitable width and height in a malicious bmp according to the scaled width and height. The cause of the heap overflow is this integer overflow located here:
and the insufficient checks performed in the gdk_pixbuf_new function.
Interestingly enough, in a recent version of gdk-pixbuf (2.31 or newer) somebody replaced some old code that checks for a overflow with a g_try_malloc_n that is supposed to check for overflow, but it doesn't in x86_64 (you can see the old and new code here: https://github.com/GNOME/gdk-pixbuf/commit/deb78d971c4bcb9e3ccbb71e7925bc6baa707188#diff-cde3af8b5b1c0789407148d53a75ae22R448)
This issud did not affect the versions of gdk-pixbuf as shipped with Red Hat Enterprise Linux 5 since they did not include the vulnerable code.
*** This bug has been marked as a duplicate of bug 1252290 ***