Bug 1425969

Summary: Exception: unable to recover stream data on problematic PDFs
Product: Red Hat Enterprise Linux 7 Reporter: Orion Poplawski <orion>
Component: qpdfAssignee: Zdenek Dohnal <zdohnal>
Status: CLOSED ERRATA QA Contact: Petr Dancak <pdancak>
Severity: high Docs Contact:
Priority: low    
Version: 7.6CC: orion, pdancak, thozza
Target Milestone: rcKeywords: Patch, TestCaseProvided, Triaged
Target Release: ---Flags: thozza: mirror+
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: qpdf-5.0.1-4.el7 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-09-29 19:18:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1709724, 1757052, 1780577    
Attachments:
Description Flags
Backported patch with upstream test none

Description Orion Poplawski 2017-02-22 22:02:05 UTC
Description of problem:

Paycheck PDFs generated by ADP's web portal are not able to be printed with cups due to the following error in pdftopdf which uses libqpdf:

WARNING: document.pdf (object 3 0, file position 22059): stream keyword not
followed by proper line terminator
WARNING: document.pdf (object 11 0, file position 20260): stream keyword not
followed by proper line terminator
WARNING: document.pdf (object 11 0, file position 20260): attempting to
recover stream length
ERROR: Exception: document.pdf (object 11 0, file position 20260): unable to
recover stream data

The PDFs are not compliant, but ghostscript is able to clean it up so that it can be printed with:

gs -sOutputFile=output.pdf -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dNOPAUSE -dBATCH input.pdf

which complains:

   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.
   **** Warning: stream operator isn't terminated by valid EOL.

   **** This file had errors that were repaired or ignored.
   **** The file was produced by:
   **** >>>> PDFOUT v3.8v by Xenos, inc. <<<<
   **** Please notify the author of the software that produced this
   **** file that it does not conform to Adobe's published PDF
   **** specification.

So, while this is clearly an issue with ADP and the PDFOUT/Xenos software, it would be great if qpdf/pdftopdf could be made more robust.

Unfortunately, since these are paycheck information, I'm not sure I can find a sample that I can attach.

The pdf has "stream" followed by a space and then a new-line - from od:

0030440   >  sp   s   t   r   e   a   m  sp  nl   q  sp   1   9   0   .
           203e    7473    6572    6d61    0a20    2071    3931    2e30

From ghostscript-9.20/Resource/Init/pdf_base.ps:

% PDF files are inconsistent about what may fall between the 'stream' keyword
% and the actual stream data, and it appears that no one algorithm can
% detect this reliably.  We used to try to guess whether the file included
% extraneous \r and/or \n characters, but we no longer attempt to do so,
% especially since the PDF 1.2 specification states flatly that the only
% legal terminators following the 'stream' keyword are \n or \r\n, both of
% which are properly skipped and discarded by the token operator.
% Unfortunately, this doesn't account for other whitespace characters that
% may have preceded the EOL, such as spaces or tabs. Thus we back up one
% character and scan until we find the \n terminator.

So I believe that qpdf should do the same thing and just keep scanning until the \n terminator is found.

Upstream issue: https://github.com/qpdf/qpdf/issues/104

Version-Release number of selected component (if applicable):
5.0.1-3.el7

Comment 2 Orion Poplawski 2017-07-31 17:45:33 UTC
Looks like upstream has checked in a fix to master, and hopefully will release 7.0 with it soon.

Comment 4 Zdenek Dohnal 2018-01-18 14:57:35 UTC
Hi Orion,

thank you for reporting this issue and I'm sorry for delay. Patch seems to be small, but I'm not sure if I will be able to reproduce this issue right now without any affected pdf file.
I'm setting CondNACk: reproducer.

Comment 7 Zdenek Dohnal 2018-10-05 10:22:24 UTC
Orion,

would you mind contact the support team with the issue? It will help us with prioritizing it.


For information on how to contact the Red Hat production support team, please visit:
    https://www.redhat.com/support/process/production/#howto

Comment 8 Orion Poplawski 2018-10-05 14:30:09 UTC
I don't have a support contract.

Comment 9 Tomáš Hozza 2018-11-19 15:39:47 UTC
Hi.

Would you be willing to provide an example PDF to reproduce the issue?

Comment 10 Orion Poplawski 2018-11-19 15:59:46 UTC
Reproducer sent privately.  Still a problem on 7.6.

Comment 14 Zdenek Dohnal 2020-01-28 17:53:16 UTC
I was able to reproduce the issue with Orion's pdf.

Comment 15 Zdenek Dohnal 2020-01-28 17:55:10 UTC
Created attachment 1656061 [details]
Backported patch with upstream test

Backporting was more difficult than expected, but it works now. It took several patches to backport too.

Comment 17 Orion Poplawski 2020-01-28 18:00:01 UTC
Thanks for working on this!

Comment 23 errata-xmlrpc 2020-09-29 19:18:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (qpdf bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3857