Bug 1534206 - StandardSymbolsPS.otf font has broken charmaps
Summary: StandardSymbolsPS.otf font has broken charmaps
Keywords:
Status: CLOSED CANTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: urw-base35-fonts
Version: 27
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
Assignee: David Kaspar [Dee'Kej]
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1537736 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-01-14 07:19 UTC by Mattias Ellert
Modified: 2018-07-30 12:46 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-07-30 12:46:45 UTC
Type: Bug


Attachments (Terms of Use)
Test program illustrating the problem (1.14 KB, text/x-csrc)
2018-01-14 07:19 UTC, Mattias Ellert
no flags Details
Example file greek.ps (17.52 KB, application/postscript)
2018-01-17 07:49 UTC, Mattias Ellert
no flags Details
How it displays in evince without the otf version installed (51.85 KB, image/png)
2018-01-17 07:50 UTC, Mattias Ellert
no flags Details
How it displays in evince with the otf version installed (43.05 KB, image/png)
2018-01-17 07:51 UTC, Mattias Ellert
no flags Details
Screenshot of greek.ps example file in Rawhide (60.76 KB, image/png)
2018-02-19 18:07 UTC, David Kaspar [Dee'Kej]
no flags Details

Description Mattias Ellert 2018-01-14 07:19:43 UTC
Created attachment 1380898 [details]
Test program illustrating the problem

Description of problem:

After installing the urw-base35-fonts update containing the .otf versions of the fonts, applications can no longer find glyphs for the symbol font. With the update installed, fontconfig prefers the .otf version over the .t1 version. But since the charmaps are broken in the .otf version, applications that worked before the update, now no longer work.

Version-Release number of selected component (if applicable):

urw-base35-standard-symbols-ps-fonts-20170801-4.fc27.noarch
urw-base35-standard-symbols-ps-fonts-20170801-4.fc28.noarch

How reproducible:

Always

Steps to Reproduce:
1. Download the attachment
2. Compile: g++ -I/usr/include/freetype2 -o symbol-charmap-test symbol-charmap-test.cxx -lfreetype
3. Run: ./symbol-charmap-test

Actual results:

$ ./symbol-charmap-test 
File: /usr/share/fonts/urw-base35/StandardSymbolsPS.t1
Found unicode charmap
Index for alpha (0x03b1): 65
Found adode charmap
Index for alpha (0x61): 65
File: /usr/share/fonts/urw-base35/StandardSymbolsPS.otf
Found unicode charmap
Index for alpha (0x03b1): 0
Found adode charmap
Index for alpha (0x61): 0

Expected results:

$ ./symbol-charmap-test 
File: /usr/share/fonts/urw-base35/StandardSymbolsPS.t1
Found unicode charmap
Index for alpha (0x03b1): 65
Found adode charmap
Index for alpha (0x61): 65
File: /usr/share/fonts/urw-base35/StandardSymbolsPS.otf
Found unicode charmap
Index for alpha (0x03b1): <non-zero number>
Found adode charmap
Index for alpha (0x61): <same non-zero number>

Additional info:

The expected charmaps are found, but do not provide the expected mappings.

Comment 1 David Kaspar [Dee'Kej] 2018-01-15 13:05:12 UTC
Thank you for the report, Mattias.

Unfortunately, this is not something I or upstream could fix. This has to be taken directly to (URW)++ company. But they are not very responsive, so it might take some time to get this resolved... :-/

I'm sorry for the inconvenience.

Comment 2 David Kaspar [Dee'Kej] 2018-01-16 13:54:41 UTC
Upstream has been informed about this, and they will look into it. However, they are moving into a new office, so it might take some time.

Comment 3 Mattias Ellert 2018-01-17 07:49:55 UTC
Created attachment 1382254 [details]
Example file greek.ps

Comment 4 Mattias Ellert 2018-01-17 07:50:58 UTC
Created attachment 1382255 [details]
How it displays in evince without the otf version installed

Comment 5 Mattias Ellert 2018-01-17 07:51:55 UTC
Created attachment 1382257 [details]
How it displays in evince with the otf version installed

Comment 6 David Kaspar [Dee'Kej] 2018-01-17 12:50:10 UTC
Thanks for additional info, Mattias! :) Upstream is aware of this BZ, so that information should reach them... ;)

Comment 7 Jason Tibbitts 2018-02-14 19:24:08 UTC
*** Bug 1537736 has been marked as a duplicate of this bug. ***

Comment 8 Jason Tibbitts 2018-02-14 19:26:54 UTC
So could we not just stop shipping StandardSymbolsPS.otf in Fedora for now?  Not being able to view postscript files containing mathematics is kind of problematic.

Comment 9 David Kaspar [Dee'Kej] 2018-02-15 09:46:43 UTC
Hello Jason!

(In reply to Jason Tibbitts from comment #8)
> So could we not just stop shipping StandardSymbolsPS.otf in Fedora for now?
I'd like to avoid such a solution for now. But I'll do it at some point if I don't hear from upstream for a while. I have pinged upstream about this issue today, so lets hope they'll respond soon...

If you're facing this problem, please remove the StandardSymbolsPS.otf manually:
> sudo rm -f /usr/share/fonts/urw-base35/StandardSymbolsPS.otf

The Ghostscript itself was updated in Rawhide (F28) to do a lookup for *.t1 versions first, but I can't backport these changes into F27. It could break a lot of things.

So for now we have to wait for upstream to respond, and use the workaround.

Best regards,

 -- Dee'Kej --

Comment 10 Jason Tibbitts 2018-02-15 14:51:39 UTC
Well, that's disappointing.  It's bad to regress within a stable release and the fix is so trivial.

For anyone else who like me has a large deployment but doesn't want to have to manually apply the workaround everywhere and deal with the fact that their machines will now fail RPM verification, you can fix this by adding this single line:

%exclude %{_fontdir}/StandardSymbolsPS.otf \

to the end of the definition of the %fontfamily_subpkg macro and rebuilding the package.  That gives you all of the otf files except for the broken one.

Comment 11 Kamil Dudka 2018-02-15 15:10:25 UTC
Agreed.  This is a major breakage.  David, could you please apply the suggested workaround until the proper fix is available?

Comment 12 David Kaspar [Dee'Kej] 2018-02-15 16:54:28 UTC
(In reply to Jason Tibbitts from comment #10)
> Well, that's disappointing.  It's bad to regress within a stable release and
> the fix is so trivial.

It's not the fix, that's the problem with it. It's only a workaround...
 
> For anyone else who like me has a large deployment but doesn't want to have
> to manually apply the workaround everywhere...

You could have mentioned it before. :) I can't know your setup - I can only assume.

Anyway, here you go - give it a try:
https://bodhi.fedoraproject.org/updates/urw-base35-fonts-20170801-4.fc27.1

(In reply to Kamil Dudka from comment #11)
> Agreed.  This is a major breakage.  David, could you please apply the
> suggested workaround until the proper fix is available?

Done. However, regarding these fonts specifically, the major breakage would be different (e.g. messed up fontconfig files affecting every Fedora user).

Comment 13 Mattias Ellert 2018-02-16 09:38:37 UTC
Could you fix it temporary in rawhide too?

Manually removing the file does not work e.g. in koji when the presence of the file causes problems when building packages.

Comment 14 David Kaspar [Dee'Kej] 2018-02-16 11:22:51 UTC
(In reply to Mattias Ellert from comment #13)
> Could you fix it temporary in rawhide too?
> 
> Manually removing the file does not work e.g. in koji when the presence of
> the file causes problems when building packages.

Could you please provide more information about the issue in koji, please?

In Rawhide, the Ghostscript is set to do use Type1 versions by default (not OTF), so unless you're using the urw-base35-fonts directly, this shouldn't be happening.

For which package does it cause the build to fail?

Comment 15 Fedora Update System 2018-02-16 16:27:30 UTC
urw-base35-fonts-20170801-4.fc27.1 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-96f579f6ec

Comment 16 Mattias Ellert 2018-02-19 10:54:35 UTC
You are aware that of all the packages in Fedora, ghostscript is not the one and only package that reads font files? Adding a hack to ghostscript to work around the bug does not magically annihilate the bug for all other users of the font. This bug is a serious regression. I think it should be nominated as a Blocker Bug for the release.

Comment 17 David Kaspar [Dee'Kej] 2018-02-19 13:36:52 UTC
(In reply to Mattias Ellert from comment #16)
> You are aware that of all the packages in Fedora, ghostscript is not the one
> and only package that reads font files?
It might surprise you, but I do.

> Adding a hack to ghostscript to work
> around the bug does not magically annihilate the bug for all other users of
> the font.

First of all - it's not a hack, it's how it should have been packaged long-time ago according to upstream.

Secondly, you say it breaks things, but I haven't seen a specific example/reproducer from you yet, even though I have asked for it. By example/reproducer I mean something that clearly shows it is breaking something, not just an artifical example.

Currently, I'm aware of only the 'gv' not working because of this (BZ #1537736), which is built on top of Ghostcript. In F27, this is fixed by excluding the *.otf file - as requested - becaue we do not have the updated Ghostscript in F27. That's why I haven't released the temporary workaound for F28, because the underlying Ghostscript should resolve this. If not, then it will at least show us, there's another problem with gv and new Ghostscript.

Right now, I don't have a real reason to release this workaround in Rawhide/F28, especially when Ghostscript's upstream raised their own doubts about the artifical reproducer you have provided.

So, as I said - unless you provide me with something that proves your claims of e.g. "in koji the presence of the file causes problems when building packages", like link to this failing koji build, etc. then I don't see a reason why should I "blindly" fix this.

> This bug is a serious regression. I think it should be nominated
> as a Blocker Bug for the release.

I'm OK with that. Feel free to nominate the BZ for it. But you will still have to show how it is actually negatively affecting our users or build process or so. :)

Once I'd had this info, then I would be OK to release the workaround.

Comment 18 David Kaspar [Dee'Kej] 2018-02-19 13:40:47 UTC
I forgot to mention, that I'm currently waiting for more info from (URW)++ as well. The upstream developer/maintainer of these fonts said he will look into this BZ, but he's currently travelling around the World. He does not have any ETA for us now.

Comment 19 Kamil Dudka 2018-02-19 17:04:37 UTC
As I understand it, a font file that recently started to be installed on Fedora is just broken.  What is the actual benefit of installing the broken font file?

What exactly could break if you (temporarily) remove it from the package?

Comment 20 David Kaspar [Dee'Kej] 2018-02-19 18:07:17 UTC
Created attachment 1397974 [details]
Screenshot of greek.ps example file in Rawhide

As you can see in the screenshot, the Greek alphabet is now correctly displayed in Rawhide with Evince.

If I'm supposed to release the workaround for Rawhide as well, then I need to first know what else is broken, so I can have a look at it. I can't just go around and release workarounds for something that *might* be broken. This can potentionally obfuscate some other real issue(s) that might return later in the future.

I agree with you that the OTF version of Standard Symbols PS has some broken glyphs, and I'm generally OK with releasing the workaround. But I just want to know, what else exactly is broken as well because of it - Mattias was mentioning some failing builds in Koji... If this is a case, I'd like to have a look at it.

Comment 21 Fedora Update System 2018-02-20 17:16:25 UTC
urw-base35-fonts-20170801-4.fc27.1 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 22 Hans de Goede 2018-03-01 21:43:30 UTC
I've been working on fixing bug 1523624, which is somewhat related to this. For now I'm falling back to the Symbols font from the xorg-x11-fonts-100dpi package as old X11 apps cannot use the new .t1 files.

While working on this I noticed that not only StandardSymbolsPS.otf is broken in this manner, Z003-MediumItalic.otf and D050000L.otf are broken in the same manner.

From a xfig pov esp. D050000L.otf being broken is a problem since there is no alternative font offering the Dingbats symbols.

Comment 23 David Kaspar [Dee'Kej] 2018-03-02 10:04:35 UTC
Thank you Hans for letting us know. This is something way beyond my skills & knowledge, so for now we have to wait for upstream's fix, and use the workaround if necessary.

So lets keep this BZ opened for now.

Comment 24 Hans de Goede 2018-03-03 15:23:28 UTC
Correction, I'm not sure if this applies to D050000L.otf and Z003-MediumItalic.otf too:

[hans@shalem Download]$ ./symbol-charmap-test 
File: /usr/share/fonts/urw-base35/D050000L.t1
Found unicode charmap
Index for alpha (0x03b1): 0
Found adode charmap
Index for alpha (0x61): 65
File: /usr/share/fonts/urw-base35/D050000L.otf
Found unicode charmap
Index for alpha (0x03b1): 0
Found adode charmap
Index for alpha (0x61): 66

[hans@shalem Download]$ ./symbol-charmap-test 
File: /usr/share/fonts/urw-base35/Z003-MediumItalic.t1
Found unicode charmap
Index for alpha (0x03b1): 555
File: /usr/share/fonts/urw-base35/Z003-MediumItalic.otf
Found unicode charmap
Index for alpha (0x03b1): 556

Since D050000L is the Dingbats fonts and those do not have unicode mappings I think both are actually correct and I was confusing bug 1551219 for this one.

Comment 25 David Kaspar [Dee'Kej] 2018-03-23 11:09:26 UTC
I have received an updated version of StandardSymbolsPS.otf from upstream. I will prepare a testing archive next week, to see if it fixes the issue or not.

In case I forget it somehow, feel free to "bombard me" with messages. :)

Comment 26 David Kaspar [Dee'Kej] 2018-04-06 10:59:04 UTC
(In reply to David Kaspar [Dee'Kej] from comment #25)
> I have received an updated version of StandardSymbolsPS.otf from upstream. I
> will prepare a testing archive next week, to see if it fixes the issue or
> not.
> 
> In case I forget it somehow, feel free to "bombard me" with messages. :)

I have created a scratch-build with the new StandardSymbolsPS.otf inside it:

https://dkaspar.fedorapeople.org/share/scratch-build/fedora/

Please test the packages, and let me know the results. Thanks!

Comment 27 Mattias Ellert 2018-04-16 10:15:29 UTC
The adobe charmap is correct now. The unicode charmap is different from the one in the T1 version. The application I package rely on the adobe charmap, so this version will work for that, and it also seems to work when displaying pdf files in evince. I don't think it makes sense for the unicode charmap to be different in the T1 and otf versions.

$ ./symbol-charmap-test 
File: usr/share/fonts/urw-base35/StandardSymbolsPS.t1
Found unicode charmap
Index for alpha (0x03b1): 65
Found adobe charmap
Index for alpha (0x61): 65
File: usr/share/fonts/urw-base35/StandardSymbolsPS.otf
Found unicode charmap
Index for alpha (0x03b1): 0
Found adobe charmap

Comment 28 Mattias Ellert 2018-04-16 11:19:24 UTC
I small cut and past error in the last comment, the last line was missing - showing that the adobe charmap was fixed:

Index for alpha (0x61): 66

The symbol font file from the old urw fonts (i.e. s050000l.pfb) for which this is supposed to be a drop in replacement, has a unicode charmap that corresponds to the one used in the T1 version:

File: ./s050000l.pfb
Found unicode charmap
Index for alpha (0x03b1): 65
Found adobe charmap
Index for alpha (0x61): 65

Comment 29 David Kaspar [Dee'Kej] 2018-04-16 13:16:11 UTC
I'm not sure I understand... Is it fixed for you or is still something missing?

By the way, this was the message I have received with the testing version of the font:
> I have looked into the issue and found the following reason:
> - In OTF you have the encoding information twice, in the cmap-table and in the
> CFF encoding vector.
>  Obviously your program refers to the CFF encoding vector and not to the cmap
> information. The cmap is correct, but the CFF encoding vector was missing.
>
> I have attached a version which includes the encoding correctly. Please let me
> know whether this fixes your problems.

Comment 30 Mattias Ellert 2018-04-16 13:58:21 UTC
It is partially fixed.

Requesting the Adobe charmap (platform 7, encoding 2) now gives the right result. Before this just gave a bunch of zeroes that could no be used for anything. So anything that requires this charmap is now OK.

Requesting the Unicode charmap (platform 3, encoding 1) gives a copy of the adobe charmap instead of a unicode charmap. If I want the "small Greek alpha, α" in the unicode char map I should request the glyph index for code point 0x03b1, which returns the right result in the T1 version and in the old s050000l.pfb version, i.e. the same index as requesting the glyph index for code point 0x61 in the adobe charmap.

But the OTF version return 0, which is not expected. However, if I request the glyph index for code point 0x61 (small Latin a) in the OTF version's unicode charmap it returns the index for the small alpha, which it should not. The T1 version and the old s050000l.pfb version both return 0 as expected since there is no Latin a in the symbol font.

Comment 31 David Kaspar [Dee'Kej] 2018-04-16 14:09:12 UTC
Thanks for the clarification. I will let (URW)++ know. :)

Comment 32 David Kaspar [Dee'Kej] 2018-07-30 12:46:45 UTC
Hello Mattias,

I'm sorry to disappoint, but upstream won't be fixing those I guess. Here's the reply I got:

> The symbol font does not have correct Unicodes. It was constructed the same
> way as done by Windows, using fake Unicodes based in the latin 1 area between
> x0020 and x00ff. This was necessary in case you have unicode based documents
> and do a font change. In this case the result would be strange if you have a
> font with correct unicodes.

We are now waiting for new usptream release of the fonts with the latest updates. Once we have it, I'll put it into Fedora. However, there's not much more I can do here anymore I guess.


Note You need to log in before you can comment on or make changes to this bug.