Description of problem:

When browsing a given PDF file, evince exits with a "Bogus memory allocation size" error

Version-Release number of selected component (if applicable):

poppler-0.5.4-4.4.el5.11

How reproducible:

Always

Steps to Reproduce:
1. Open the PDF file in evince
2. Page down to page 13 / 15
  
Actual results:

evince exits with "Bogus memory allocation size"

Expected results:

evince does not exit

Additional info:

The problem does not occur in recent version of Evince. There is also an error displayed "Dictionary key must be a name object" but that's unrelated to the bogus memory allocation.

The message "Bogus memory allocation size" can come from different part of the code.

So breaking on _exit() in gdb, we can get a clearer idea of the code path that leads to the error:

Breakpoint 6, _exit (status=1) at ../sysdeps/unix/sysv/linux/_exit.c:29
29 {
(gdb) bt
#0  _exit (status=1) at ../sysdeps/unix/sysv/linux/_exit.c:29
#1  0x0000003ec44efb1a in gmallocn3 (a=<value optimized out>, b=<value optimized out>, c=<value optimized out>) at gmem.c:189
#2  0x0000003ec44b46b3 in JPXStream::readTilePart (this=0x2aaab01403d0) at JPXStream.cc:1750
#3  0x0000003ec44b5cbe in JPXStream::readCodestream (this=0x2aaab01403d0, len=<value optimized out>) at JPXStream.cc:1252
#4  0x0000003ec44b74f2 in JPXStream::readBoxes (this=0x2aaab01403d0) at JPXStream.cc:642
#5  0x0000003ec44b7816 in JPXStream::reset (this=0x2aaab01403d0) at JPXStream.cc:276
#6  0x0000003ec4810118 in CairoOutputDev::drawImage (this=0x2aaab002df90, state=<value optimized out>, ref=<value optimized out>, str=0x2aaab01403d0,
   width=370, height=259, colorMap=0x2aaab0140510, maskColors=0x0, inlineImg=0) at CairoOutputDev.cc:809
#7  0x0000003ec447cbec in Gfx::doImage (this=0x2aaab00008c0, ref=0x409ffcb0, str=0x2aaab01403d0, inlineImg=0) at Gfx.cc:3223
#8  0x0000003ec447fad1 in Gfx::opXObject (this=0x2aaab00008c0, args=0x409ffd80, numArgs=<value optimized out>) at Gfx.cc:2902
#9  0x0000003ec447a075 in Gfx::go (this=0x2aaab00008c0, topLevel=1) at Gfx.cc:580
#10 0x0000003ec447a4b0 in Gfx::display (this=0x2aaab00008c0, obj=0x409ffef0, topLevel=1) at Gfx.cc:543
#11 0x0000003ec44c0c89 in Page::displaySlice (this=0x2aaab0049340, out=0x2aaab002df90, hDPI=<value optimized out>, vDPI=<value optimized out>,
   rotate=<value optimized out>, useMediaBox=<value optimized out>, crop=<value optimized out>, sliceX=<value optimized out>, sliceY=0, sliceW=433,
   sliceH=554, links=0x0, catalog=0x2aaab002cba0, abortCheckCbk=0, abortCheckCbkData=0x0, annotDisplayDecideCbk=0, annotDisplayDecideCbkData=0x0)
   at Page.cc:375
#12 0x0000003ec480d4a3 in poppler_page_render_to_pixbuf (page=0x915b40, src_x=0, src_y=0, src_width=433, src_height=554, scale=0.70710700750350952,
   rotation=0, pixbuf=0x2aaab0029ca0) at poppler-page.cc:366
#13 0x000000000044c8a1 in ?? ()
#14 0x0000000000419209 in gtk_toggle_button_set_active () at gtktogglebutton.c:287
#15 0x0000000000417c52 in gtk_toggle_button_set_active () at gtktogglebutton.c:287
#16 0x00000000004181af in gtk_toggle_button_set_active () at gtktogglebutton.c:287
#17 0x0000003f35048e04 in g_thread_create_proxy (data=0x6c6b80) at gthread.c:553
#18 0x0000003f2f40673d in start_thread (arg=<value optimized out>) at pthread_create.c:301
#19 0x0000003f2e8d3d1d in clone () from /lib64/libc.so.6
(gdb) 

The values of a, b and c are optimized out, so gdb won't be of much help here.

Instrumenting poppler lib, we can get thr values actually passed to gmallocn3(), it's :

   a=1, b=0, c=72

So the program exits because b=0 

In poppler/JPXStream.cc, current upstream uses:

   subband->cbs = (JPXCodeBlock *)gmallocn(subband->nXCBs *
                                           subband->nYCBs,
                                           sizeof(JPXCodeBlock));

Instead of

   subband->cbs = (JPXCodeBlock *)gmallocn3( subband->nXCBs,
                                             subband->nYCBs,
                                             sizeof(JPXCodeBlock));

So that if subband->nYCBs == 0, subband->nXCBs * subband->nYCBs == 0 and the problem does not occur.

Patch for poppler 0.5.4 attached.