Bug 923346 - Symbols incorrectly displayed
Summary: Symbols incorrectly displayed
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: google-croscore-fonts
Version: rawhide
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Parag Nemade
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 901858
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-03-19 15:57 UTC by Jan Včelák
Modified: 2015-01-31 16:50 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 901858
Environment:
Last Closed: 2015-01-31 16:46:46 UTC
Type: Bug


Attachments (Terms of Use)
diff-screenshot from okular (135.52 KB, image/png)
2013-03-20 16:48 UTC, Jan Včelák
no flags Details

Description Jan Včelák 2013-03-19 15:57:40 UTC
I'm seeing the same problem as described in bug #901858 with google-croscore-symbolneu-fonts-1.21.0-4.fc18.noarch package. Removing the package solves the problem and math symbols are displayed correctly.

google-croscore-symbolneu-fonts-1.21.0-4.fc18.noarch
okular-4.10.1-1.fc18.x86_64

+++ This bug was initially created as a clone of Bug #901858 +++

Created attachment 683157 [details]
microchip pdf showing the fault

Description of problem:

When reading technical pdfs, the wrong symbols are displayed for some math characters. e.g. pi is displayed as not equal, ohms as a vertical bar, and micro (mu) is displayed as infinity.
   
Version-Release number of selected component (if applicable):

okular-4.9.5.-1.fc18

How reproducible:

Displaying a technical pdf such as 
http://ww1.microchip.com/downloads/en/AppNotes/00954A.pdf
shows the wrong symbols.
E.G. In figure 1 - the value for C2 is displayed as 470 (inf) F instead of 470 (mu) F

Steps to Reproduce:
1. Display technical pdf
2. 
3.
  
Actual results:
pi is displayed as not equals
ohms as a vertical bar
micro(mu) as infinity

Expected results:
The correct symbols should be displayed

Additional info:
Evince also has this problem, but emacs displays everything correctly.
This happens across a range of pdfs and I think is a serious bug than needs fixing as soon as possible,. It reflects badly on fedora if we cannot display technical specifications properly.

--- Additional comment from Kevin Kofler on 2013-01-19 12:47:09 EST ---

> Evince also has this problem

Reassigning to poppler, which is the common library used by both.


What I see is that this PDF does not embed its fonts, which means you get whatever fonts are installed on the system. In this case, Symbol is mapped to wine-symbol-fonts. So I'm not sure whether poppler or the font is doing the wrong thing there.

And by the way, the PDF is bad because it uses a font outside of the PostScript standard (Symbol) and doesn't embed it. A portable PDF MUST embed all the non-PostScript fonts it uses.

--- Additional comment from Richard Kennedy on 2013-02-28 07:57:45 EST ---

Just a quick update.

The new embedded pdf reader in Firefox 19 displays these files correctly too.

Well, the pdf may not be correct but it was created by adobe software & there are lots of them out in the wild, so we should do something to fix it.
Can I change the default font mapping for popper to something else ? If so how do I go about that?

--- Additional comment from Kevin Kofler on 2013-02-28 19:00:01 EST ---

As I said, it's a bug in either poppler or wine-symbol-fonts (or both). Unfortunately, the poppler maintainer hasn't answered so far.

--- Additional comment from Marek Kašík on 2013-03-01 07:05:38 EST ---

Hi,

this is a bug in wine-symbol-fonts package. The symbol.ttf font has wrong mapping.
For additional info see upstream bug http://bugs.winehq.org/show_bug.cgi?id=24099.
I'm reassigning this to wine.

Regards

Marek

--- Additional comment from Jason Tibbitts on 2013-03-01 14:45:34 EST ---

Wow, it's great news that this the cause is known, because I've been struggling with this for some time.  Unfortunately upstream bug report has been there for quite some time with no progress, so I'm not sure what chance we have of getting this fixed.  What would break by removing this font?  Can it be removed from the dependency list of the wine-fonts metapackage?

--- Additional comment from Kevin Kofler on 2013-03-01 17:53:16 EST ---

Would that really fix the issue? I don't think the symbols can be properly displayed if the font is entirely missing. I suspect there's no way around fixing the font.

--- Additional comment from Richard Kennedy on 2013-03-02 11:50:34 EST ---

I've just fixed the font following the steps in the wine bug 24099 and it works.
I only run one wine app, ltspice, and it still works after the fix, so it's all looking good. 

Both firefox & emacs can display these characters properly so they must be using some other font, but I don't know how to find out which one. But if we could find out can we swap the mapping to that?

--- Additional comment from Richard Kennedy on 2013-03-02 13:29:50 EST ---

Having thought about this some more, I think that is more likely that the bug really is that poppler isn't handling unicode characters properly.

I cut & pasted the failing mu character from evince into emacs, and it says it's unicode 0x3BC. So it seems that poppler tries to use the symbol font and 0x3BC
 is something different in there because it's broken too.

emacs and firefox somehow resolve the unicode character correctly and don't end up in the symbol font, but I haven't found out which font they do use.

I didn't realise all of this stuff was so involved and complex. At least fixing up the font is a temporary fix.

--- Additional comment from Kevin Kofler on 2013-03-02 17:58:39 EST ---

Maybe poppler shouldn't be looking up "Symbol" in fontconfig at all, but handle the Symbol font specially? The Window$ Symbol TTF font which WINE's font tries to emulate is a hack which predates Unicode and maps Greek letters and other symbol characters to plain 8-bit characters (0-255). Somebody should probably test what happens when you use the original M$ Symbol.ttf. If it has the same issue, it's not WINE's fault.

--- Additional comment from Richard Kennedy on 2013-03-04 12:44:39 EST ---

Just a couple of notes :-

* The poppler version here is 0.20.2 which was released on Tue July 10, 2012, and
the latest is 0.22.1 which contains quite a number of bug fixes.

* I've reset my machine to the default state and if I run pdftohtml on this file 
the correct characters are generated in the html. (pdftohtml is a poppler tool).

* Libre Office also can display these files correctly.

Is there any way to get poppler to use the opensymobol font to resolve these symbols? AFAICT opensymbol contains all of the correct unicode so might make a good replacement and it's already installed.

--- Additional comment from Marek Kašík on 2013-03-05 07:26:52 EST ---

Created attachment 705385 [details]
blacklist wine font Symbol

(In reply to comment #10)
> Just a couple of notes :-
> 
> * The poppler version here is 0.20.2 which was released on Tue July 10,
> 2012, and
> the latest is 0.22.1 which contains quite a number of bug fixes.

The poppler-0.22.1 doesn't fix this.


> * I've reset my machine to the default state and if I run pdftohtml on this
> file 
> the correct characters are generated in the html. (pdftohtml is a poppler
> tool).

It generates correct html because the codes for those characters are correct in the pdf.


> * Libre Office also can display these files correctly.

Do you mean the html or the pdf? Libre Office displays me the same characters as poppler when importing the pdf.


> Is there any way to get poppler to use the opensymobol font to resolve these
> symbols? AFAICT opensymbol contains all of the correct unicode so might make
> a good replacement and it's already installed.

Place the attached file to ~/.config/fontconfig/fonts.conf. This will blacklist the font for all applications which use fontconfig (useful when the wine-symbol-fonts is a dependency of a package which you don't want to remove).

I've tried to view a file with the character for the micro sign (UTF-8 code 0xC2B5) in notepad which is part of wine-common. After switching the font to "Symbol", it showed me the incorrect symbol. I guess that there is really something wrong with the font.

Regards

Marek

--- Additional comment from Kevin Kofler on 2013-03-05 10:00:50 EST ---

The problem is that the Symbol font was designed in a way that the 'm' character is displayed as 'µ', and 'µ' is not mapped to anything. In fact, Symbol was the way to get those special characters in old software which didn't support Unicode. I don't know whether it's possible to fix this issue without breaking that old software when people try to run it under WINE. I hope it's possible, but I'm not sure. The WINE upstream bug report has a proposed fix, but I don't know whether that fix doesn't break the non-Unicode apps.

--- Additional comment from Andreas Bierfert on 2013-03-05 14:55:02 EST ---

Well as a workaround we could confine wine symbol to wine for now (like with tahoma). This would reduce the problem to wine applications. 

Before resorting to this I will investigate a bit...

Comment 1 Parag Nemade 2013-03-20 03:37:41 UTC
Quick reply:
 I just downloaded the pdf you have given. Checked my F18 for Google Croscore SymbolNeu font and found its not present. Used evince and took a screenshot of figure 1.
 Then installed SymbolNeu font and open the pdf in evince and took a screenshot and conclusion both the screenshots are identical.

Comment 2 Marek Kašík 2013-03-20 13:23:44 UTC
(In reply to comment #1)
> Quick reply:
>  I just downloaded the pdf you have given. Checked my F18 for Google
> Croscore SymbolNeu font and found its not present. Used evince and took a
> screenshot of figure 1.
>  Then installed SymbolNeu font and open the pdf in evince and took a
> screenshot and conclusion both the screenshots are identical.

I've just tried to open the file with and without the font and it differs depending on whether the font is installed or not (in both evince and okular on F18).

Comment 3 Parag Nemade 2013-03-20 14:16:23 UTC
I uninstalled google-croscore-symbolneu-fonts package and reopened pdf using evince and still see the C2 value in figure 1 as 

"470 μF"

Oops! while I pasted above string from evince the inf changed to mu above. In my testing evince keep showing inf irrespective of google croscore symbolneu is installed or not. Both cases when above string copied from evince here it is always showing mu.

Comment 4 Parag Nemade 2013-03-20 14:22:16 UTC
Same with Okular. Display in pdf always showing me inf but when copied to clipboard it changes to mu.

Comment 5 Jan Včelák 2013-03-20 16:48:47 UTC
Created attachment 713339 [details]
diff-screenshot from okular

I took a screenshot of the page and highlighted the differences.

On the left side you can see the output without google-crosscore-sybolneu-fonts package installed. Okular shows in properties, that '/usr/share/fonts/default/Type1/s050000l.pfb' is used to display Symbol font.

On the right side, the package is installed and '/usr/share/fonts/google-croscore/SymbolNeu.ttf' is used instead.

Comment 6 Nicolas Mailhot 2013-03-20 19:26:59 UTC
Since the problem only occurs in a pdf that was built using a font known for not following the unicode standard, it should be analysed by pdf people, unless someone can point a specific encoding mistake in Symbol Neu

That it "works" with a pfb font that dates from the same broken encoding era could be the result of two bugs (one in the font, the other in the pdf reader) that neutralise themselves, not any specific Symbol Nu problem.

Please reproduce the "working" result with a modern unicode font before claiming Symbol Nu is broken.

The target should be to work by default with good modern fonts, not to break them all to reproduce the behaviour of legacy fonts

Comment 7 Nicolas Mailhot 2013-03-20 19:34:22 UTC
(modern unicode font should be either dejavu, since the project is quite anal in unicode compliance, or STIX 1.1, since they've driven the normalization or numerous math symbols. opensymbol is supposed to have been fixed encoding-wise nowadays, but I haven't checked it myself)

Comment 8 Nicolas Mailhot 2013-03-20 19:40:56 UTC
If it is confirmed poppler is applying an encoding workaround that is not necessary with proper unicode fonts, a bug should be filled against the provider of /usr/share/fonts/default/Type1/s050000l.pfb to have the encoding of this font fixed or the file dropped from the distro

Comment 9 Nicolas Mailhot 2013-03-20 20:10:11 UTC
Anyway, it seems my defense of Google was a mistake. It seems they had the bright idea to 'solve' interoperability problems by releasing a font that reproduces the exact brokenness of symbol

This is too broken for thought and is going to break multiple unix apps that rely on proper font encoding (in fact it is so broken our font packaging guidelines didn't even envision it had to be protected against)

The font should be dropped from the distro. All the symbols it provides are already present in our default fonts, its only purpose is to reproduce a Microsoft encoding mistake and to confuse apps.

Comment 10 Nicolas Mailhot 2013-03-20 20:11:05 UTC
and poppler should add it to its broken font list next to Microsoft symbol — all the same workarounds apply)

Comment 11 Nicolas Mailhot 2013-03-20 20:17:39 UTC
lastly, gucharmap refuses to use this font to render mu even when it is selected in the font dropdown. So unlike poppler it detects its a font best avoided

Comment 12 Kevin Kofler 2013-03-20 22:15:16 UTC
Please note that google-croscore-symbolneu is not the only font in Fedora affected by this issue, there's also the WINE Symbol font, see the original bug this bug is a clone of (bug #901858).

Comment 13 Fedora End Of Life 2013-12-21 12:17:48 UTC
This message is a reminder that Fedora 18 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 18. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '18'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 18's end of life.

Thank you for reporting this issue and we are sorry that we may not be 
able to fix it before Fedora 18 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior to Fedora 18's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 14 Parag Nemade 2015-01-31 16:46:46 UTC
Note we have already dropped google-croscore-symbolneu-fonts subpackage in 1.23.0-5 build as a fix to bug 1037882. This fix is available in f20, f21 and in rawhide.

Comment 15 Parag Nemade 2015-01-31 16:50:59 UTC
(In reply to Parag from comment #14)
> Note we have already dropped google-croscore-symbolneu-fonts subpackage in
> 1.23.0-5 build as a fix to bug 1037882. This fix is available in f20, f21
> and in rawhide.

ah! sorry just realized after clicking on submit button. We have not removed subpackage but the fontconfig file only.


Note You need to log in before you can comment on or make changes to this bug.