Bug 1513343

Created attachment 1352448 [details]
Screenshot of policytool

Created attachment 1352449 [details]
Simplified Chinese characters

Is this regression? How it looked in older versions? was it correct? Both cases? Both wrongly rendered character and the not rendered characters? Can you write simple java pogram printing those three characters? The correct one, the incorrect one, and the not rendered one?
When I copy pasted "添" to source code or properties file, it  seemed correct to me...

There was one patch to fonts recently: https://bugzilla.redhat.com/show_bug.cgi?id=1484079  But I doubt this was an consequentness. 

Openjdk is using system fonts - so maybe there was an change? What about f26 or openjdk9? 

Sorry for asking many information, but debugging Chinese fonts isnot easy task fot Latin alphabets users. TY!!

Okay, I will try to write some simple java program.

Actually I think this problem is caused by Chinese fonts changes in Fedora.

We changed the default Chinese fonts from TrueType to OpenType/CFF.

In the past, we use UMing and WQY TrueType fonts.
Now we use Adobe Source Han fonts, which is OpenType/CFF fonts.

Could you tell me where I can find the properties file for translation in policytool?
Maybe I can check if it is related in translation.

Thanks!

(In reply to Peng Wu from comment #4)
> Could you tell me where I can find the properties file for translation in
> policytool?
> Maybe I can check if it is related in translation.

jdk/src/share/classes/sun/security/tools/policytool/Resources_zh_CN.java should be it:
http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/07011844584f/src/share/classes/sun/security/tools/policytool/Resources_zh_CN.java

This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Re-targeting to F27. The JDK is the same there.

Created attachment 1356356 [details]
AWT example

Created attachment 1356357 [details]
Swing example

I copied the string from translation and uploaded simple java program for both AWT and Swing.

Please notice the top stroke of the "添" character.

In Simplified Chinese, the top stroke is straight.

But in the java program, the top stroke is a curve.
It seems from Traditional Chinese characters.

Adobe Source Han fonts contains both Simplified Chinese
and Traditional Chinese characters.

(In reply to Peng Wu from comment #4)
> Okay, I will try to write some simple java program.
> 
> Actually I think this problem is caused by Chinese fonts changes in Fedora.
> 
> We changed the default Chinese fonts from TrueType to OpenType/CFF.

When this change happened in Fedora? Before the latest JDK update OpenJDK was not picking up CFF fonts at all, only True Type and Type 1. If there was a change in the font file in Fedora it means that before the update OpenJDK would only pick a True Type version of the Chinese font, but after the CFF may be the one selected, depending on what fc-match feels like returning.

Unfortunately the result depends only of the font you have installed, but it may narrow down to a broken/incorrect font, we should be checking this first (identify what font is actually used and then debug the font to see if it's correct).

> In the past, we use UMing and WQY TrueType fonts.
> Now we use Adobe Source Han fonts, which is OpenType/CFF fonts.

Can you please link the output of this:

fc-match -f "sans:regular:roman" -sv

Also, can you please run the test application with this flag:

-Dsun.java2d.debugfonts=true

And attach the whole output too?
 
> Could you tell me where I can find the properties file for translation in
> policytool?
> Maybe I can check if it is related in translation.

This seems unlikely if it was working before, but if you can check this out it would helps us a lot.

Cheers,
Mario

Created attachment 1358483 [details]
fc-match output

Created attachment 1358484 [details]
JButtonExample output

Created attachment 1358485 [details]
ButtonExample output

Thanks, I uploaded the output files.

The translation is correct I think.

My bad, I forgot to add, can you please run the same command with -Dsun.java2d.debugfonts=true but before delete your $HOME/.java/fonts directory each time?

The JDK caches the fonts to use, so I want to be sure the JDK is using a fully new font set at each run.

Thanks!

Cheers,
Mario

Created attachment 1358577 [details]
JButtonExample output

Created attachment 1358580 [details]
ButtonExample output

No problem, I uploaded new output.

Created attachment 1358647 [details]
Screenshot of policytool with new font cache

After remove $HOME/.java/fonts directory,
policytool can render all characters now.

But it still renders some wrong glyphs like "添".

I did some more debugging, so far I think the issue is that it's not using Source Han Sans, but instead either NanumGothic or VL Gothic.

If you force your sample application to use Source Han, it displays the correct font:

Font font = Font.createFont(Font.TRUETYPE_FONT, new File("/usr/share/fonts/adobe-source-han-sans-cn/SourceHanSansCN-Normal.otf"));
b.setFont(font.deriveFont(40.0f));

I need to debug a bit more to understand why the font is not loaded by default, that maybe fontconfig returning a different match, or something else entirely.

I think I got it, debugging was quite interesting since the toString used by the debugger to see the actual font had a side effect thus hiding the point where the font gets somehow discarded and replaced.

Anyway, the font contains a list of family names and full names:

family:
"Source Han Sans CN"(s)
"思源黑体 CN"(s)
"Source Han Sans CN Regular"(s)
"思源黑体 CN Regular"(s)

fullname:
"Source Han Sans CN Regular"(s)
"思源黑体 CN Regular"(s)

But the JDK doesn't see those, it doesn't seem like FontConfig return all the information, it only returns:

family: Source Han Sans CN
fullname: Source Han Sans CN Regular

There's some code that checks that those matches, but they don't of course (Source Han Sans CN != Source Han Sans CN Regular). It looks to me like a bit bogus check to be honest and I'm not entirely sure what is the proper intention (check the validity of the font, yes, but why comparing those two fields? Doesn't seem correct to me). This font (as well as the TW counterpart) seems the only one that has a multitude of names of all the ones I tested, so they are the only ones to fail the validation. I'll need to check out with upstream why this check is there or if it's correct at all, but the code is quite delicate so even if this is fixed I'm not sure we will be able to back port the fix in the existing releases.

An option could be to scan the file if the match fails to double check if there is another name that identifies the font.

BTW, There's a discrepancy between what FcPatternPrint(fontPattern) does and what querying the pattern directly returns, i.e.:

FcPatternGetString(fontPattern, FC_FILE, 0, &file[i]);
FcPatternGetString(fontPattern, FC_FAMILY, 0, &family[i]);

I wonder if I can make FontConfig return all the names in the first place, then I can match them during initialisation, which is the fastest and safest, but I hadn't find a way yet.

Cheers,
Mario

You can get all of values in certain property by:
for (i = 0; ; i++) {
  if (FcPatternGetString(fontPattern, FC_FAMILY, i, &file) != FcResultMatch)
    break;
  ...
}

I think to get the translated strings.

Maybe need to change FcObjectSetBuild function call to include FC_FAMILYLANG.

Like: https://git.gnome.org/browse/gnome-font-viewer/commit/?id=e512b0bd53636ebad6ce2428c44bfb02816ed7f2

Sorry, maybe also need to include FC_FULLNAMELANG.

(In reply to Akira TAGOH from comment #23)
> You can get all of values in certain property by:
> for (i = 0; ; i++) {
>   if (FcPatternGetString(fontPattern, FC_FAMILY, i, &file) != FcResultMatch)
>     break;
>   ...
> }

Thanks, this is very useful!

Cheers,
Mario

This message is a reminder that Fedora 27 is nearing its end of life.
On 2018-Nov-30  Fedora will stop maintaining and issuing updates for
Fedora 27. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora  'version' of '27'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 27 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

This bug still exists in Fedora 29.

Changed version to Fedora 29.

There should be some way. but depends how they use fontconfig API in the code and that should be fixed there. so I don't recommend and no guarantees because that may breaks a lot but it still can be done in the config like this. just for deminstration purpose:

<fontconfig>
  <match target="font">
    <test name="prgname" compare="eq">
      <string>java</string>
    </test>
    <edit name="family" mode="assign">
      <name target="font">fullname</name>
    </edit>
  </match>
</fontconfig>

This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.