Bug 1208094

Created attachment 1009626 [details]
Possible patch to prevent SIGFPE in ProcPutImage()

Description of problem:

Using nxplayer from NoMachine sometimes crashes Xorg with 'Floating point exception'

Version-Release number of selected component (if applicable):

xorg-x11-server-Xorg-1.15.0-25.1.el6

How reproducible:

About 25 to 50% of the time

Steps to Reproduce:
1. Install latest nxplayer from NoMachine
2. Run nxplayer to run Firefox from a NoMachine Terminal Server
3. Browse to various sites - e.g. Youtube

Actual results:

Xorg (on the host running the nxplayer) can crash - /var/log/Xorg.0.log.old will report:

(EE) 
(EE) Backtrace:
(EE) 0: /usr/bin/Xorg (xorg_backtrace+0x51) [0x5aade1]
(EE) 1: /usr/bin/Xorg (0x400000+0x1af1d9) [0x5af1d9]
(EE) 2: /lib64/libpthread.so.0 (0x3837000000+0xf710) [0x383700f710]
(EE) 3: /usr/bin/Xorg (0x400000+0x34fe5) [0x434fe5]
(EE) 4: /usr/bin/Xorg (0x400000+0x38e86) [0x438e86]
(EE) 5: /usr/bin/Xorg (0x400000+0x3c8ea) [0x43c8ea]
(EE) 6: /lib64/libc.so.6 (__libc_start_main+0xfd) [0x3836c1ed5d]
(EE) 7: /usr/bin/Xorg (0x400000+0x269c9) [0x4269c9]
(EE) 
(EE) Floating point exception at address 0x434fe5
(EE) 
Fatal server error:
(EE) Caught signal 8 (Floating point exception). Server aborting
(EE) 


Expected results:

Xorg doesn't crash :-)

Additional info:

Enabling core dumps and using gdb reports:

(gdb) where
#0  0x0000003adc832625 in raise () from /lib64/libc.so.6
#1  0x0000003adc833e05 in abort () from /lib64/libc.so.6
#2  0x00000000005b006e in OsAbort () at utils.c:1330
#3  0x000000000047c497 in ddxGiveUp (error=EXIT_ERR_ABORT) at xf86Init.c:1098
#4  0x00000000005b7082 in AbortServer () at log.c:770
#5  0x00000000005b7d34 in FatalError (
    f=0x5e0ec0 "Caught signal %d (%s). Server aborting\n") at log.c:911
#6  0x00000000005af22e in OsSigHandler (signo=8, sip=<value optimized out>,
    unused=<value optimized out>) at osinit.c:147
#7  <signal handler called>
#8  0x0000000000434fe5 in ProcPutImage (client=0x27a6730) at dispatch.c:1960
#9  0x0000000000438e86 in Dispatch () at dispatch.c:434
#10 0x000000000043c8ea in dix_main (argc=9, argv=<value optimized out>,
    envp=<value optimized out>) at main.c:294
#11 0x0000003adc81ed5d in __libc_start_main () from /lib64/libc.so.6
#12 0x00000000004269c9 in _start ()


The crash is at:

#8  0x0000000000434fe5 in ProcPutImage (client=0x27a6730) at dispatch.c:1960
1960        if (lengthProto >= (INT32_MAX / stuff->height))
(gdb) print stuff->height
$1 = 0 

It looks like this code was added as part of a fix to CVE-2014-8092 'dix: integer overflow in ProcPutImage()' - but there is no check to make sure 'stuff->height' is not zero ...

I guess stuff->height being zero is probably not valid - but it does look like it can happen - and I suspect in this case, ProcPutImage() will return with an error later on after further checks ?

I've attached a possible patch that appears to fix the Xorg crash