Created kernel tracking bugs for this issue:

Affects: fedora-all [bug 1599169]

Notes:

the flaw is not triggerable in the Red Hat products.

1) an attacker must be able to do ioctl() to /dev/fb0 and most probably he can not due to the permissions. namespaces or containers won't help attacker with this.

$ ls -l /dev/fb0
crw-rw---- 1 root video 29, 0 Jul 28 19:35 /dev/fb0

2) for the overflow to happen size_t must be 32-bit, as cmap->len is __u32:

[include/uapi/linux/fb.h]
struct fb_cmap {
        __u32 start;                    /* First entry  */
        __u32 len;                      /* Number of entries */

and kmalloc() accepts size_t and type of sizeof() is size_t too:

void *kmalloc(size_t size, gfp_t flags)

[drivers/video/fbdev/uvesafb.c]
entries = kmalloc(sizeof(*entries) * cmap->len, GFP_KERNEL);

if size_t is 64-bit then multiplication is also 64-bit and there is no overflow. this means that only 32-bit systems are vulnerable.

3) even with a 32-bit system there is a check in fb_set_user_cmap():

[drivers/video/fbdev/core/fbcmap.c]
int fb_set_user_cmap(struct fb_cmap_user *cmap, struct fb_info *info)
{
        int rc, size = cmap->len * sizeof(u16);
        if (size < 0 || size < cmap->len)
                return -E2BIG;

this especially limits cmap->len to be less-or-equal to UINT_MAX/4 i.e. 0x3fffffff. so later when multipying it to sizeof(*entries) == 4 it can be as maximum as:

3FFFFFFF * 4 = FFFFFFFC i.e. no 32-bit overflow.

4) the check above is from the upstream commit 1e7c7804884fc which is present since v2.6.37-rc4 and so is present in all versions of RHEL-7 and later but not in RHEL-6.

still, fb_set_user_cmap() calls fb_alloc_cmap():

[drivers/video/fbcmap.c]
int fb_set_user_cmap(struct fb_cmap_user *cmap, struct fb_info *info)
{
        int rc, size = cmap->len * sizeof(u16);
        rc = fb_alloc_cmap(&umap, cmap->len, cmap->transp != NULL);

int fb_alloc_cmap(struct fb_cmap *cmap, int len, int transp)
{
    int size = len*sizeof(u16);
    if (!(cmap->red = kmalloc(size, GFP_ATOMIC)))
        goto fail;

and kmalloc() will fail for the sizes > 32 Mb (theoretically) and > 2^22 = 4Mb (in practice):

[include/linux/slab.h]
/* The largest kmalloc size supported by the slab allocators is
   32 megabyte (2^25) or the maximum allocatable page order if that is
   less than 32 MB. */
#define KMALLOC_SHIFT_HIGH ((MAX_ORDER + PAGE_SHIFT - 1) <= 25 ? \
                           (MAX_ORDER + PAGE_SHIFT - 1) : 25)
#define KMALLOC_MAX_SIZE (1UL << KMALLOC_SHIFT_HIGH)

[include/linux/mmzone.h]
#ifndef CONFIG_FORCE_MAX_ZONEORDER
#define MAX_ORDER 11
#else
#define MAX_ORDER CONFIG_FORCE_MAX_ZONEORDER
#endif

$ grep CONFIG_FORCE_MAX_ZONEORDER *
config-2.6.32-754.el6.ppc64:CONFIG_FORCE_MAX_ZONEORDER=9
config-2.6.32-754.el6.s390x:CONFIG_FORCE_MAX_ZONEORDER=9

Note that for Fedora the long analysis is not needed, Fedora is not vulnerable because the uvesafb driver is not enabled:

[hans@shalem ~]$ grep CONFIG_FB_UVESA /boot/config-4.18.0-0.rc7.git1.1.fc29.x86_64
# CONFIG_FB_UVESA is not set