Bug 1806696 - Cups can't detect file type when file type is UTF-8 Unicode text
Summary: Cups can't detect file type when file type is UTF-8 Unicode text
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: paps
Version: 30
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Akira TAGOH
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-24 19:01 UTC by Greta Watson
Modified: 2020-03-13 23:50 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2020-03-11 04:23:54 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
File that does not print (103 bytes, text/plain)
2020-02-25 14:36 UTC, Greta Watson
no flags Details
File that prints, similar to non-working file except for first two characters. (105 bytes, text/plain)
2020-02-25 14:38 UTC, Greta Watson
no flags Details
file from /etc/cups/ppd (42.96 KB, text/plain)
2020-02-25 14:39 UTC, Greta Watson
no flags Details
cups job log (7.09 KB, text/plain)
2020-02-25 14:51 UTC, Greta Watson
no flags Details
texttopaps output (14.34 KB, application/postscript)
2020-03-05 19:15 UTC, Greta Watson
no flags Details

Description Greta Watson 2020-02-24 19:01:04 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Greta Watson 2020-02-24 19:15:41 UTC
Somehow, this bug got created before I could do the description, etc.  So, I'm doing it as comments:

Description of problem:  file type of UTF-8 Unicode text contains at least one text character, such as ¢, æ, °, or other character whose value lies between 128 and 255.  When one goes to print it, the error message in /var/log/messages says:  Feb 24 10:19:02 sparky cupsd[1010]: Can\'t detect file type
                         Feb 24 10:19:02 sparky cupsd[1010]: temp file: can\'t find PDF header
                           ...
Version of Fedora has ranged from 28 to 30.  Currently, I'm using 5.4.19-100.fc30.x86_64.
It can be reproduced by trying to print the UTF-8 Unicode text file from the command line.
Nothing is printed, and the /var/log/messages entry is added.  One has to cancel printing of the file to allow printing of other files.

There is a work-around:  if one places a double slash at the start of the file, it prints.  Apparently, it then thinks it is a C or C++ file, even though the file type is still UTF-8 Unicode text.

Comment 2 Zdenek Dohnal 2020-02-25 07:43:15 UTC
Hi Greta,

thank you for reporting the issue!

Would you mind attaching the file which are you trying to print?

Would you mind telling me from which app you trying to print?

Would you mind sending me ppd file of your print queue? (from /etc/cups/ppd)

Would you mind attaching debug log as a file when job is processed (see https://fedoraproject.org/wiki/How_to_debug_printing_problems)

Comment 3 Greta Watson 2020-02-25 14:36:23 UTC
Created attachment 1665654 [details]
File that does not print

Comment 4 Greta Watson 2020-02-25 14:38:24 UTC
Created attachment 1665655 [details]
File that prints, similar to non-working file except for first two characters.

This file differs from the non-working file in that the two characters // are at the start of this file.

Comment 5 Greta Watson 2020-02-25 14:39:29 UTC
Created attachment 1665656 [details]
file from /etc/cups/ppd

Comment 6 Greta Watson 2020-02-25 14:51:20 UTC
Created attachment 1665657 [details]
cups job log

The cups job log was created after I tried to print the non-working file using the HP printer via wifi from the xfce4-terminal using the command lp.

Comment 7 Zdenek Dohnal 2020-02-25 15:29:49 UTC
From the log:

Feb 25 06:46:19 sparky cupsd[7777]: Started filter /usr/lib/cups/filter/texttopaps (PID 7795)
Feb 25 06:46:19 sparky cupsd[7777]: Started filter /usr/lib/cups/filter/gstopdf (PID 7796)
Feb 25 06:46:19 sparky cupsd[7777]: Started filter /usr/lib/cups/filter/pdftopdf (PID 7797)
Feb 25 06:46:19 sparky cupsd[7777]: Started filter /usr/lib/cups/filter/gstoraster (PID 7798)
Feb 25 06:46:19 sparky cupsd[7777]: Started filter /usr/lib/cups/filter/hpcups (PID 7799)
Feb 25 06:46:19 sparky cupsd[7777]: Started backend /usr/lib/cups/backend/hp (PID 7800)
Feb 25 06:46:19 sparky cupsd[7777]: gstopdf argv[5] = 187 greta notworking.txt 1 finishings=3 number-up=1 job-uuid=urn:uuid:daea883b-765d-3cf1-6624-1306516eae74 job-originating-host-name=localhost date-time-at-creation= date-time-at-processing= time-at-creation=1582641979 time-at-processing=1582641979 document-name-supplied=notworking.txt
Feb 25 06:46:19 sparky cupsd[7777]: PPD: /etc/cups/ppd/HP_Printer.ppd
Feb 25 06:46:19 sparky cupsd[7777]: OUTFORMAT=\"(null)\", so output format will be CUPS/PWG Raster
Feb 25 06:46:19 sparky cupsd[7777]: STATE: -marker-supply-low-warning
Feb 25 06:46:19 sparky cupsd[7777]: pdftopdf: Last filter determined by the PPD: hpcups; FINAL_CONTENT_TYPE: application/vnd.cups-raster => pdftopdf will not log pages in page_log.
Feb 25 06:46:19 sparky cupsd[7777]: OUTFORMAT=\"PDF\", so output format will be PDF
Feb 25 06:46:19 sparky cupsd[7777]: HP_Printer: Error while converting strings.
Feb 25 06:46:19 sparky cupsd[7777]: PID 7795 (/usr/lib/cups/filter/texttopaps) stopped with status 1.

The error happens in texttopaps filter, which is part of paps package, I'll reassign the bug.

Comment 8 Zdenek Dohnal 2020-02-25 16:11:17 UTC
I'm not able to reproduce the issue on up-to-date F30 using your uploaded file and ppd.

Wasn't a different file uploaded? Is your system up-to-date?

Comment 9 Greta Watson 2020-02-25 18:52:18 UTC
My system is up to date, with the exception of google-chrome-stable, which does not have anything to do with cups.  I updated yesterday.  I uploaded several files:  the non-working file does not have the // characters at the start of the file.  the working file does have the // characters at the start of the file, so the filter program apparently thinks it may be a C or C++ file, so it prints.

Comment 10 Akira TAGOH 2020-03-05 09:57:20 UTC
So where is the file can be reproduced? I think you can test it directly invoking /usr/lib/cups/texttopaps. that may gives you more detailed logs.

Comment 11 Greta Watson 2020-03-05 19:15:59 UTC
Created attachment 1667887 [details]
texttopaps output

Comment 12 Akira TAGOH 2020-03-06 06:32:51 UTC
Are you able to reproduce your error?

In general, CUPS filters are brought up through PIPE. thus, the output should contains PostScript code only. What I can say from that result is PostScript file you attached looks valid.

Did you see any errors/warnings when you generated it? how about the exit status?

Comment 13 Greta Watson 2020-03-06 14:49:21 UTC
I can reproduce the error 100% of the time when I use the notworkong.txt file.  It gives a filter error.  On my machine, texttopaps is not in /usr/lib/cups.  It is in /usr/lib/cups/filter.  I neglected to mention that when I ran the texttopaps binary.  When I couldn't find it in /usr/lib/cups, I used the find command to find it and run it.  When I sent the postscript file to the printer, it printed okay.

Comment 14 Akira TAGOH 2020-03-09 06:50:28 UTC
Ouch, my bad. yes, it should be in /usr/lib/cups/filter/

Comment 15 Akira TAGOH 2020-03-09 10:08:17 UTC
I can't still reproduce this. can you check the log with "LogLevel debug" and see argv lines and envp lines. so you can see how texttopaps was brought up with. I assume that you can reproduce this even when running texttopaps with those parameters directly.

Unfortunately there seems no errors logs in cups why texttopaps was failed. so we need to find out a way to reproduce this with more simpler way.

Comment 16 Greta Watson 2020-03-10 15:39:50 UTC
I would like to do what you ask, but need detailed instructions.  This bug appears only when I use the lp command from the command line.  If I look at the file with kwrite, and then print from within kwrite, it works.  If I bring up the text file in my browser and try to print, it works okay.  So please give me exactly what I should type into the xfce4 terminal (or gnome terminal, or konsole terminal to do what you need me to do.  Thank-you.  And sorry for having to ask, but I really do not understand the printing system.

Comment 17 Zdenek Dohnal 2020-03-10 15:52:35 UTC
The file which goes from 'lp' to cupsd scheduler and which is going into texttopaps filter is in /var/spool/cups. It begins with 'd', has job id in its filename and ends with number of files in the job.

f.e. file for job id 1, which has one file to print, is d00001-001.

Comment 18 Greta Watson 2020-03-10 16:33:31 UTC
Found the file, and the corresponding file that starts with the letter c.  In the latter file, the error message reads that there is no reason for the failure.  Is this what you wanted?  I'm confused.

Comment 19 Greta Watson 2020-03-10 18:07:02 UTC
I may have a way for you to reproduce the bug.  It may have something to do with one of the KDE environment variables.  So, if you are running KDE, set the LANG environment variable to C (export LANG=C).  That should enable you to reproduce the bug.  When I unset LANG, the problem did not occur.

Comment 20 Akira TAGOH 2020-03-11 04:23:54 UTC
Well, that isn't a bug. the encoding under C locale isn't UTF-8. if you want to use UTF-8 in C locale, you should set LANG=C.UTF-8.

Comment 21 Greta Watson 2020-03-11 14:57:02 UTC
Filter still fails with LANG=C.UTF-8.

Comment 22 Akira TAGOH 2020-03-12 05:36:48 UTC
Where did you set that locale? how about your system locale? if you have it as system locale, did you restart cups service after changing it?

Comment 23 Greta Watson 2020-03-12 13:17:28 UTC
I have the locale set in my .bashrc file (~/.bashrc).  I did not change that.  I prefer the collating sequence provided when LANG=C or LANG=C.UTF-8.

What I did was to tweak the value on the command line or through the env command:

This worked, printing the file:
   1.  unset LANG
   2.  lp notworking.txt
   3.  export LANG=C
This also worked:
   1. env -u LANG /usr/bin/lp notworking.txt

This did not work:
   0.  (LANG=C set in my .bashrc)
   1.  lp notworking.txt

This also did not work:
   1.  export LANG=C.UTF-8
   2.  lp notworking.txt

Could KDE itself be the culprit?  I know KDE system settings printers subsection now makes it impossible for non-root users to add printers (KDE bug 408512).  It makes me wonder what else KDE has messed with....  That thought is what led me to the LANG issue.

Comment 24 Akira TAGOH 2020-03-13 05:10:52 UTC
Dunno. it may be more related to CUPS perhaps, because texttopaps itself accurately recognize C.UTF-8 as one of utf-8 locale. I assume that CUPS is calling it as is though, if CUPS modifies locale to set for sub-processes, it won't work indeed. I suppose such information should be provided in the debugging log when you run cupsd with "LogLevel debug" in cupsd.conf. you can find them out like:
envp[0]="CUPS_CACHEDIR=/var/cache/cups"
envp[1]="CUPS_DATADIR=/usr/share/cups"
...
envp[18]="CHARSET=utf-8"
envp[19]="LANG=ja_JP.UTF-8"
...

Comment 25 Zdenek Dohnal 2020-03-13 06:25:17 UTC
CUPS takes system locale (from /etc/locale.conf) and LANG variable too.

I was able to reproduce with steps:

1) localectl set-local LANG=C
2) export LANG=C
3) /usr/lib/cups/filter/texttopaps 1 root '' 1 '' <Greta's file>

When I change system locale and env variable LANG to C.UTF-8, the file goes through the filter without error.

Comment 26 Zdenek Dohnal 2020-03-13 09:46:52 UTC
The same setup works for printing via lp - after setting with localectl and export LANG to utf-8 enconding printing works. Setting to non-utf-8 it fails with error in texttopaps.

Comment 27 Greta Watson 2020-03-13 23:50:31 UTC
Thank-you for all the information.

By leaving /etc/locale.conf to its default (LANG-en_US.UTF-8) and setting LC_COLLATE to C.UTF-8, I get the collating sequence I want without having a printing problem.


Note You need to log in before you can comment on or make changes to this bug.