Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 5 product line. The current stable release is 5.10. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 598945

Summary: Bogus memory allocation size when browsing a PDF in Evince
Product: Red Hat Enterprise Linux 5 Reporter: Olivier Fourdan <ofourdan>
Component: popplerAssignee: Marek Kašík <mkasik>
Status: CLOSED ERRATA QA Contact: Desktop QE <desktop-qa-list>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 5.5CC: cmeadors, fleite, jwest, kem, mclasen, rkhadgar, tao, tpelka
Target Milestone: rcKeywords: Patch, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: poppler-0.5.4-15.el5 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-02-21 06:11:33 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 698595    
Attachments:
Description Flags
Proposed patch none

Description Olivier Fourdan 2010-06-02 11:51:33 UTC
Description of problem:

When browsing a given PDF file, evince exits with a "Bogus memory allocation size" error

Version-Release number of selected component (if applicable):

poppler-0.5.4-4.4.el5.11

How reproducible:

Always

Steps to Reproduce:
1. Open the PDF file in evince
2. Page down to page 13 / 15
  
Actual results:

evince exits with "Bogus memory allocation size"

Expected results:

evince does not exit

Additional info:

The problem does not occur in recent version of Evince. There is also an error displayed "Dictionary key must be a name object" but that's unrelated to the bogus memory allocation.

The message "Bogus memory allocation size" can come from different part of the code.

So breaking on _exit() in gdb, we can get a clearer idea of the code path that leads to the error:

Breakpoint 6, _exit (status=1) at ../sysdeps/unix/sysv/linux/_exit.c:29
29 {
(gdb) bt
#0  _exit (status=1) at ../sysdeps/unix/sysv/linux/_exit.c:29
#1  0x0000003ec44efb1a in gmallocn3 (a=<value optimized out>, b=<value optimized out>, c=<value optimized out>) at gmem.c:189
#2  0x0000003ec44b46b3 in JPXStream::readTilePart (this=0x2aaab01403d0) at JPXStream.cc:1750
#3  0x0000003ec44b5cbe in JPXStream::readCodestream (this=0x2aaab01403d0, len=<value optimized out>) at JPXStream.cc:1252
#4  0x0000003ec44b74f2 in JPXStream::readBoxes (this=0x2aaab01403d0) at JPXStream.cc:642
#5  0x0000003ec44b7816 in JPXStream::reset (this=0x2aaab01403d0) at JPXStream.cc:276
#6  0x0000003ec4810118 in CairoOutputDev::drawImage (this=0x2aaab002df90, state=<value optimized out>, ref=<value optimized out>, str=0x2aaab01403d0,
   width=370, height=259, colorMap=0x2aaab0140510, maskColors=0x0, inlineImg=0) at CairoOutputDev.cc:809
#7  0x0000003ec447cbec in Gfx::doImage (this=0x2aaab00008c0, ref=0x409ffcb0, str=0x2aaab01403d0, inlineImg=0) at Gfx.cc:3223
#8  0x0000003ec447fad1 in Gfx::opXObject (this=0x2aaab00008c0, args=0x409ffd80, numArgs=<value optimized out>) at Gfx.cc:2902
#9  0x0000003ec447a075 in Gfx::go (this=0x2aaab00008c0, topLevel=1) at Gfx.cc:580
#10 0x0000003ec447a4b0 in Gfx::display (this=0x2aaab00008c0, obj=0x409ffef0, topLevel=1) at Gfx.cc:543
#11 0x0000003ec44c0c89 in Page::displaySlice (this=0x2aaab0049340, out=0x2aaab002df90, hDPI=<value optimized out>, vDPI=<value optimized out>,
   rotate=<value optimized out>, useMediaBox=<value optimized out>, crop=<value optimized out>, sliceX=<value optimized out>, sliceY=0, sliceW=433,
   sliceH=554, links=0x0, catalog=0x2aaab002cba0, abortCheckCbk=0, abortCheckCbkData=0x0, annotDisplayDecideCbk=0, annotDisplayDecideCbkData=0x0)
   at Page.cc:375
#12 0x0000003ec480d4a3 in poppler_page_render_to_pixbuf (page=0x915b40, src_x=0, src_y=0, src_width=433, src_height=554, scale=0.70710700750350952,
   rotation=0, pixbuf=0x2aaab0029ca0) at poppler-page.cc:366
#13 0x000000000044c8a1 in ?? ()
#14 0x0000000000419209 in gtk_toggle_button_set_active () at gtktogglebutton.c:287
#15 0x0000000000417c52 in gtk_toggle_button_set_active () at gtktogglebutton.c:287
#16 0x00000000004181af in gtk_toggle_button_set_active () at gtktogglebutton.c:287
#17 0x0000003f35048e04 in g_thread_create_proxy (data=0x6c6b80) at gthread.c:553
#18 0x0000003f2f40673d in start_thread (arg=<value optimized out>) at pthread_create.c:301
#19 0x0000003f2e8d3d1d in clone () from /lib64/libc.so.6
(gdb) 

The values of a, b and c are optimized out, so gdb won't be of much help here.

Instrumenting poppler lib, we can get thr values actually passed to gmallocn3(), it's :

   a=1, b=0, c=72

So the program exits because b=0 

In poppler/JPXStream.cc, current upstream uses:

   subband->cbs = (JPXCodeBlock *)gmallocn(subband->nXCBs *
                                           subband->nYCBs,
                                           sizeof(JPXCodeBlock));

Instead of

   subband->cbs = (JPXCodeBlock *)gmallocn3( subband->nXCBs,
                                             subband->nYCBs,
                                             sizeof(JPXCodeBlock));

So that if subband->nYCBs == 0, subband->nXCBs * subband->nYCBs == 0 and the problem does not occur.

Patch for poppler 0.5.4 attached.

Comment 1 Olivier Fourdan 2010-06-02 12:04:22 UTC
Created attachment 419004 [details]
Proposed patch

Comment 4 RHEL Program Management 2010-08-09 19:51:18 UTC
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.

Comment 17 errata-xmlrpc 2012-02-21 06:11:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0236.html