Bug 475743 - Japanese desktop defaulting to Chinese fonts
Japanese desktop defaulting to Chinese fonts
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: fontconfig (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Behdad Esfahbod
Fedora Extras Quality Assurance
: i18n, Regression, Reopened
Depends On:
Blocks: F11Blocker/F11FinalBlocker F11Beta/F11BetaBlocker
  Show dependency treegraph
 
Reported: 2008-12-10 03:34 EST by Mamoru TASAKA
Modified: 2013-01-09 23:58 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 485562 (view as bug list)
Environment:
Last Closed: 2009-03-16 18:09:20 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
screenshot with VLGothic 20081203 (251.57 KB, image/png)
2008-12-10 03:36 EST, Mamoru TASAKA
no flags Details
screenshot with VLGothic 20081029 (241.39 KB, image/png)
2008-12-10 03:37 EST, Mamoru TASAKA
no flags Details
glyphs of arabic numerals (112.60 KB, image/png)
2008-12-12 01:54 EST, Mamoru TASAKA
no flags Details
screenshot again (200.34 KB, image/png)
2009-01-30 08:50 EST, Mamoru TASAKA
no flags Details
another screenshot (127.29 KB, image/png)
2009-01-31 01:22 EST, Jens Petersen
no flags Details
screenshot with old fontconfig (152.58 KB, image/png)
2009-02-17 02:57 EST, Mamoru TASAKA
no flags Details
screenshot with new fontconfig (157.30 KB, image/png)
2009-02-17 02:58 EST, Mamoru TASAKA
no flags Details

  None (edit)
Description Mamoru TASAKA 2008-12-10 03:34:25 EST
Description of problem:
Since VLGothic-fonts is upgraded to 20081203, I see many 
chinese glyphs on Japanese environment.
Downloading to 20081029 works good.

Version-Release number of selected component (if applicable):
VLGothic-fonts-20081203-2.fc11

(perhaps?)
cjkunifonts-ukai-0.2.20080216.1-10.fc11.noarch.rpm
cjkunifonts-uming-0.2.20080216.1-10.fc11.noarch.rpm


How reproducible:
100%

Steps to Reproduce:
1. For example, the following URL:
   http://cvs.fedoraproject.org/viewvc/policycoreutils/po/ja.po?root=elvis&r1=1.24&r2=1.25
 
Actual results:
See attached
Comment 1 Mamoru TASAKA 2008-12-10 03:36:20 EST
Created attachment 326463 [details]
screenshot with VLGothic 20081203
Comment 2 Mamoru TASAKA 2008-12-10 03:37:06 EST
Created attachment 326465 [details]
screenshot with VLGothic 20081029
Comment 3 Mamoru TASAKA 2008-12-10 03:39:27 EST
Also with 20081203 the glyphs of arabic numerals frequently changes.
Comment 4 Akira TAGOH 2008-12-10 05:59:25 EST
my rawhide box is broken now so I can't do any testing... so just guessing, is this issue gone if you do remove /etc/fonts/conf.d/64-ttf-arphic-uming.conf or add testing for Chinese like:

<match>
  <test name="lang" compare="contains">
    <string>zh</string>
  </test>
  <test name="family">
    <string>sans-serif</string>
  </test>
  <edit name="family" mode="prepend" binding="same">
    <string>AR PL UMing HK</string>
    <string>AR PL UMing CN</string>
  </edit>
</match>

instead of preference alias?
Comment 5 Mamoru TASAKA 2008-12-10 06:37:34 EST
(In reply to comment #4)
> is
> this issue gone if you do remove /etc/fonts/conf.d/64-ttf-arphic-uming.conf 

It seems this works

> or
> add testing for Chinese like:
> <snip>
> 
> instead of preference alias?

Would you tell me to what file?
Comment 6 Nicolas Mailhot 2008-12-10 15:00:50 EST
Behdad recently wanted to experiment with new syntax to deal with fonts that need locale-specific ordering. You should try to ping him to check if he hasn't a better fontconfig recipe.
Comment 7 Akira TAGOH 2008-12-10 20:13:50 EST
(In reply to comment #5)
> Would you tell me to what file?

Try to replace:

        <alias>
                <family>sans-serif</family>
                <prefer>
                        <family>AR PL UMing HK</family>
                        <family>AR PL UMing CN</family>
                </prefer>
        </alias>

in 64-ttf-arphic-uming.conf with the above for sans-serif.
Comment 8 Akira TAGOH 2008-12-10 20:19:55 EST
(In reply to comment #6)
> Behdad recently wanted to experiment with new syntax to deal with fonts that
> need locale-specific ordering. You should try to ping him to check if he hasn't
> a better fontconfig recipe.

Aha. Cc'ing him.

Behdad, do you have any idea to resolve a kind of locale-specific ordering issue in fontconfig?
Comment 9 Mamoru TASAKA 2008-12-12 01:43:41 EST
(In reply to comment #7)
> (In reply to comment #5)
> > Would you tell me to what file?
> 
> Try to replace:
> 
>         <alias>
>                 <family>sans-serif</family>
>                 <prefer>
>                         <family>AR PL UMing HK</family>
>                         <family>AR PL UMing CN</family>
>                 </prefer>
>         </alias>
> 
> in 64-ttf-arphic-uming.conf with the above for sans-serif.

Thanks. This seems to work for Japanese locale (I cannot test
for Chinese locale because I don't use it and I don't know
where to check...)
Comment 10 Mamoru TASAKA 2008-12-12 01:54:38 EST
Created attachment 326711 [details]
glyphs of arabic numerals

By the way is it a different issue that the glyphs (especially
the widths) of arabic numerals change according to contexts?
Comment 11 Behdad Esfahbod 2008-12-17 02:00:44 EST
I had an idea but I tested quickly and it didn't work.  I'll think about it again.  I have some other ideas about how to fix the CJK issues in fontconfig.  I'm currently working on fontconfig.  I'll see if I get to those.
Comment 12 Caius Chance 2009-01-29 19:30:38 EST
I could lower the ranking of uming (Chinese fonts) by increasing the number of the .conf file.

Which side would you prefer to make the changes? VL Gothic, Uming, or fontconfig?
Comment 13 Jens Petersen 2009-01-29 20:32:39 EST
The Japanese problem seems to be with Sans: eg if one switches the desktop
application font to Monospace then VLGothic is used correctly.
Comment 14 Jens Petersen 2009-01-29 21:37:50 EST
Reverting to fontconfig-2.6.0-3.fc10 also fixes the problem, so this really does look like a fontconfig regression or change of behaviour.
Comment 16 Jens Petersen 2009-01-30 02:55:01 EST
Thanks fixes the Japanese desktop problem for me. :)

I still don't understand the discrepancy between monospace and sans... Behdad, any idea?
Comment 17 Caius Chance 2009-01-30 04:09:59 EST
If it didn't break some other things, then I'm going to build to rawhide.
Comment 18 Akira TAGOH 2009-01-30 06:42:18 EST
(In reply to comment #12)
> I could lower the ranking of uming (Chinese fonts) by increasing the number of
> the .conf file.
> 
> Which side would you prefer to make the changes? VL Gothic, Uming, or
> fontconfig?

No options for VLGothic-fonts at least because this issue doesn't happen without cjkuni-fonts.
Comment 19 Mamoru TASAKA 2009-01-30 08:50:25 EST
Created attachment 330464 [details]
screenshot again

Still seeing many Chinese glyphs.

Note that this screenshot is taken with fontconfig-2.6.95-1.git.66.gb162bfb.fc11
With reverting to fontconfig-2.6.0-3.fc10 it goes _much_ better,
however it does not remove this problem entirely.
Comment 20 Mamoru TASAKA 2009-01-30 09:21:32 EST
For record:

# rpm -qf /etc/fonts/conf.d/* | sort | uniq
VLGothic-fonts-20081203-2.fc11.noarch
VLGothic-fonts-proportional-20081203-2.fc11.noarch
baekmuk-ttf-batang-fonts-2.2-17.fc11.noarch
baekmuk-ttf-dotum-fonts-2.2-17.fc11.noarch
baekmuk-ttf-gulim-fonts-2.2-17.fc11.noarch
baekmuk-ttf-hline-fonts-2.2-17.fc11.noarch
cjkuni-uming-fonts-0.2.20080216.1-18.fc11.noarch
dejavu-lgc-sans-fonts-2.28-3.fc11.noarch
dejavu-lgc-sans-mono-fonts-2.28-3.fc11.noarch
dejavu-lgc-serif-fonts-2.28-3.fc11.noarch
dejavu-sans-fonts-2.28-3.fc11.noarch
dejavu-sans-mono-fonts-2.28-3.fc11.noarch
dejavu-serif-fonts-2.28-3.fc11.noarch
fontconfig-2.6.95-1.git.66.gb162bfb.fc11.i386
sazanami-gothic-fonts-0.20040629-6.20061016.fc11.noarch
sazanami-mincho-fonts-0.20040629-6.20061016.fc11.noarch
Comment 21 Jens Petersen 2009-01-30 23:53:48 EST
(In reply to comment #19)
> Created an attachment (id=330464) [details]
> screenshot again
> Still seeing many Chinese glyphs.

What is the URL for that page?

I could be wrong but it could be a Unihan issue with firefox not knowing what language the page is in?
Comment 22 Mamoru TASAKA 2009-01-31 00:01:36 EST
(In reply to comment #21)
> (In reply to comment #19)
> > Created an attachment (id=330464) [details] [details]
> > screenshot again
> > Still seeing many Chinese glyphs.
> 
> What is the URL for that page?

Note that the glyph is wrong also on the other place
(please look at gnome-panel in the picture: especially the glyph
 of "場所" is apparently not Japanese)

The URL is the same as the one in my comment 0.
Comment 23 Jens Petersen 2009-01-31 01:22:12 EST
Created attachment 330518 [details]
another screenshot

This is what it currently looks like for me, but I will try with a fresh rawhide install soon. :)
Comment 24 Jens Petersen 2009-01-31 02:24:10 EST
I just tried on a fresh rawhide install and it still looks good to me.

Tasaka-san, perhaps could you try again?

(Anyway unfortunately this fix/workaround won't be in F11Alpha anyway...)
Comment 25 Mamoru TASAKA 2009-01-31 03:23:32 EST
Well, after some investigation, it seems that
when I hide (i.e. add 'dot' to the file name)
65-baekmuk-ttf-dotum.conf (in baekmuk-ttf-dotum-fonts-2.2-17.fc11.noarch)
it looks good now (even with fontconfig-2.6.95-1.git.66.gb162bfb.fc11)
Comment 26 Jens Petersen 2009-01-31 05:14:30 EST
Ah I see, thank you.  Yes, baekmuk dotum is no longer installed by default, but sounds like a similar workaround is needed there too.
Comment 27 Jens Petersen 2009-01-31 05:15:03 EST
would still appreciate some comment from Behdad...
Comment 28 Caius Chance 2009-01-31 07:07:16 EST
(In reply to comment #24)
> (Anyway unfortunately this fix/workaround won't be in F11Alpha anyway...)

I will push to F11 immediately when it is released.

(In reply to comment #25)
> Well, after some investigation, it seems that
> when I hide (i.e. add 'dot' to the file name)
> 65-baekmuk-ttf-dotum.conf (in baekmuk-ttf-dotum-fonts-2.2-17.fc11.noarch)
> it looks good now (even with fontconfig-2.6.95-1.git.66.gb162bfb.fc11)

Hmm, then I should also work on baekmuk then.
Comment 29 Mamoru TASAKA 2009-02-03 05:20:22 EST
It seems that with baekmuk-ttf-XXX-2.2-19.fc11,
cjkuni-XXXX-0.2.20080216.1-20.fc11 Japanese glyphs are shown
correctly.
Comment 30 Caius Chance 2009-02-03 19:51:38 EST
There are .conf templates from package fontpackages. I applied from that.

cjkuni-fonts is patched:
http://koji.fedoraproject.org/koji/buildinfo?buildID=81269

baekmuk-ttf-fonts is patched:
http://koji.fedoraproject.org/koji/buildinfo?buildID=81270
Comment 31 Jens Petersen 2009-02-03 22:52:37 EST
No comment from Behdad so can probably close this for now.

Dunno if it is worth discussing the change of behaviour upstream...
Comment 32 Ryo Dairiki 2009-02-14 04:22:14 EST
On Fedora-10, the same bug appears after usual update.

VLGothic fonts:
vlgothic-p-fonts-20090204-2.fc10.noarch
vlgothic-fonts-20090204-2.fc10.noarch
vlgothic-fonts-common-20090204-2.fc10.noarch

Baekmuk fonts:
baekmuk-ttf-fonts-gulim-2.2-9.fc10.noarch
baekmuk-ttf-fonts-dotum-2.2-9.fc10.noarch
baekmuk-ttf-fonts-batang-2.2-9.fc10.noarch
baekmuk-ttf-fonts-common-2.2-9.fc10.noarch
baekmuk-ttf-fonts-hline-2.2-9.fc10.noarch
baekmuk-bdf-fonts-2.2-5.fc9.noarch

Is this also related to this report?
Comment 33 Jens Petersen 2009-02-15 21:51:36 EST
(In reply to comment #32)
> On Fedora-10, the same bug appears after usual update.
> baekmuk-ttf-fonts-gulim-2.2-9.fc10.noarch
> baekmuk-ttf-fonts-dotum-2.2-9.fc10.noarch
> baekmuk-ttf-fonts-batang-2.2-9.fc10.noarch
> baekmuk-ttf-fonts-common-2.2-9.fc10.noarch
> baekmuk-ttf-fonts-hline-2.2-9.fc10.noarch
> baekmuk-bdf-fonts-2.2-5.fc9.noarch

It sounds like it could be.
What if you remove the baekmuk fonts?
(They are not default for Hangul anymore anyway.)
Comment 34 Mamoru TASAKA 2009-02-16 09:52:55 EST
fontconfig-2.6.97-3.g945d6a4.fc11 again reproduces this problem......
Comment 35 Caius Chance 2009-02-16 10:54:14 EST
I have only built on rawhide. Let me backport that.
Comment 36 Jens Petersen 2009-02-16 22:43:47 EST
(In reply to comment #35)
> I have only built on rawhide. Let me backport that.

May be better to test first on f10.
Comment 37 Jens Petersen 2009-02-16 23:05:25 EST
Behdad, could you please update on this important issue?
Comment 38 Mamoru TASAKA 2009-02-17 02:57:31 EST
Created attachment 332187 [details]
screenshot with old fontconfig

(In reply to comment #34)
> fontconfig-2.6.97-3.g945d6a4.fc11 again reproduces this problem......

To clarify this, screenshot again.
This one with (the old) fontconfig-2.6.95-1.git.66.gb162bfb.fc11.i386 .
Looks fine.
Comment 39 Mamoru TASAKA 2009-02-17 02:58:48 EST
Created attachment 332188 [details]
screenshot with new fontconfig

And screenshot with new fontconfig fontconfig-2.6.97-3.g945d6a4.fc11
Japanese font glyphs are not used.
Comment 40 Jens Petersen 2009-02-17 03:19:03 EST
Yes, I see un-core-fonts being used on the ja desktop - I will try to get the fontconfig files there updated soon.
Comment 41 Ryo Dairiki 2009-02-20 12:15:40 EST
At last, I've found that cjkunifonts-uming package is the cause of this problem in my environment.

It seems like gryphs from that package are chosen in some cases, even in Japanese environments.
Japanese period followed by an alphabet is the one example.

After uninstalling that package, the problem goes away.
And I've confirmed that reinstalling this package caused the problem again.

I'll send you a detailed report later.
Comment 42 Ryo Dairiki 2009-02-23 23:59:14 EST
Problem:
- Chinese gryphs are shown when you use Monospace even in Japanese locale.
- Some characters seems different in the same context, as different gryphs are selected according to the following characters.
(For example, the width of the space character followed by "A (alphabet)" has different width with the one followed by "a (hiragana)".)

Reason:

Fontconfig choose proper gryph for virtual fonts, according to the current locale, and the context.
If you write Japanese in Japanese locale, fontconfig seeks to fonts which has Japanese gryphs.
It seems like some Chinese fonts also has Japanese gryphs by some reason.
Maybe, they are in CJ unified regions, so we cannot fix this problem simply by eliminating them.
Talking about the second problem, both Japanese fonts and DejaVu fonts have space characters, 
so fontconfig sometimes choose gryphs from Japanese fonts, and sometimes from DejaVu fonts.

Solution:

Change the gryph selection order of Fontconfig.
In Japanese environment, gryphs from Japanese gryphs should be chosen first.
In Chinese environment, gryphs from Chinese gryphs should be chosen first.
Talking about DejaVu fonts, either locale specific gryphs or DejaVu gryphs should be chosen 
independently with the context.
This change maybe require huge change in fonts.conf and might need some patches in the source of fontconfig too.
Comment 43 Behdad Esfahbod 2009-03-11 11:51:10 EDT
I'm still confused.  From what I understand, an alias I added in recent fontconfig updates is causing this.  If that's true, which alias, and if you can move it further down to not cause the bug, what's the patch?  Thanks.
Comment 44 Jens Petersen 2009-03-11 20:32:28 EDT
Ok - I think we have to wait for an installable rawhide or F11 Beta
for further testing.  With various font .conf files added to CJK fonts
I think things are a bit better now but there may still be some
more finer adjusts needed for F11?
Comment 45 Jesse Keating 2009-03-16 14:02:59 EDT
Rawhide should be installable, what I need is a clear idea if this is really a beta blocker issue or not.
Comment 46 Mamoru TASAKA 2009-03-16 14:14:57 EDT
(In reply to comment #45)
> Rawhide should be installable, what I need is a clear idea if this is really a
> beta blocker issue or not.  

Well, I tried the current fontconfig-2.6.99.behdad-3.fc11 and it seems
be working for me. So tagging this fontconfig as f11-beta may be
a good idea.
Comment 47 Jesse Keating 2009-03-16 18:09:20 EDT
We just tagged so I'll close this rawhide.

Note You need to log in before you can comment on or make changes to this bug.