This service will be undergoing maintenance at 00:00 UTC, 2016-09-28. It is expected to last about 1 hours
Bug 228804 - [CJK] different fonts used for Latin and Common chars
[CJK] different fonts used for Latin and Common chars
Status: CLOSED UPSTREAM
Product: Fedora
Classification: Fedora
Component: pango (Show other bugs)
16
All Linux
medium Severity medium
: ---
: ---
Assigned To: Behdad Esfahbod
bzcl34nup
: Desktop, i18n, Reopened, Triaged
: 529594 (view as bug list)
Depends On: 220885
Blocks:
  Show dependency treegraph
 
Reported: 2007-02-15 00:51 EST by Peng Huang
Modified: 2011-08-18 02:54 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-08-18 02:54:34 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
Screensohts from firefox & gedit (211.91 KB, application/x-bzip2)
2007-02-15 22:14 EST, Peng Huang
no flags Details
Let pango process common script as Latin (1.20 KB, text/x-patch)
2007-02-25 22:26 EST, Peng Huang
no flags Details
Patch for fontconfig comcharset (20.22 KB, patch)
2010-03-03 05:13 EST, fujiwara
no flags Details | Diff
Patch for pango compcharset (8.32 KB, patch)
2010-03-03 05:17 EST, fujiwara
no flags Details | Diff


External Trackers
Tracker ID Priority Status Summary Last Updated
GNOME Desktop 481210 None None None Never

  None (edit)
Description Peng Huang 2007-02-15 00:51:07 EST
+++ This bug was initially created as a clone of Bug #220885 +++

Description of problem:
The face of the number is changing when char is entered after number. Must be
logged in with any locale other than en_US locale. For Example : input may be :
1111aaaa or 1111<any other language char>.

Version-Release number of selected component (if applicable):
firefox-1.5.0.9-3.el5
firefox-1.5.0.9-4.el5

How reproducible:
Always

Steps to Reproduce:
1. Login into the system with any locale except en_US locale
2. Open firefox with any CJKI locale/
3. Open batman.brisbane.redhat.com/bugzilla -- Login with your Account
4. Enter a New Bug for this test
5. In the text box, type 1111aaaa or 1111<any other language char>
6. Oserve the face of 1111 is changing just any char is entered after the number
  
Actual results:
face of the number is changing.

Expected results:
Face should not be changed

Additional info:
Screenshot attached. In the scrshot, there is a line where 1111 is written
without any locale reference. Thats the actual face of the number before any
char typed after that. It is red marked.

-- Additional comment from smaitra@redhat.com on 2006-12-28 06:33 EST --
Created an attachment (id=144459)
Firefox Face change screenshot for all locales


-- Additional comment from phuang@redhat.com on 2007-02-09 01:38 EST --
I found CJK locales use font monospace for textarea, but western locals use
Courier. If I change the fonts for CJK in about:config. This bug will disappear.


-- Additional comment from phuang@redhat.com on 2007-02-09 01:41 EST --
Created an attachment (id=147728)
Change font config for chinese.


-- Additional comment from mcepl@redhat.com on 2007-02-09 05:41 EST --
Could you try to switch all monospace fonts to monospace? Does problem disappear
as well? I think that having Courier in Western fonts is not good either --
serif/sans-serif/monospace should be the standard fonts.

-- Additional comment from phuang@redhat.com on 2007-02-09 07:29 EST --
Created an attachment (id=147760)
A simple test code

I think it's a bug of pango or fontconfig. I reproduced it in gedit and in a
simple pango test code. Run the simple test program with zh_CN.UTF8 locations,
it will use different fonts to render two lines.

-- Additional comment from mcepl@redhat.com on 2007-02-09 09:39 EST --
Changing component accordingly.

-- Additional comment from phuang@redhat.com on 2007-02-11 21:29 EST --
I found pango treats numbers and punctuations as common script and almost fonts
support common script characters. So pango will render common script using
different fonts. If common script is among of other script, pango will render it
with the other script's font, If common script is alone, it will be render with
locale font. So the varied strategy will cause this bug, and the text will be
looked weirdly. 
So I think pango should not use different fonts to render common script
characters according to characters before or after it.


-- Additional comment from phuang@redhat.com on 2007-02-11 21:47 EST --
Created an attachment (id=147872)
Using Latin font to render numbers and punctuations.


-- Additional comment from besfahbo@redhat.com on 2007-02-12 14:16 EST --
(In reply to comment #7)
> I found pango treats numbers and punctuations as common script and almost fonts
> support common script characters. So pango will render common script using
> different fonts. If common script is among of other script, pango will render it
> with the other script's font, If common script is alone, it will be render with
> locale font. So the varied strategy will cause this bug, and the text will be
> looked weirdly. 
> So I think pango should not use different fonts to render common script
> characters according to characters before or after it.
> 

That's not going to happen, and this is not a bug.  If you run firefox under a
CJK locale, pango chooses fonts for that locale.  Why do you expect it to choose
digits from a Latin font?

-- Additional comment from besfahbo@redhat.com on 2007-02-12 14:22 EST --
This is NOTABUG IMHO.

-- Additional comment from phuang@redhat.com on 2007-02-12 20:50 EST --
(In reply to comment #9)
> 
> That's not going to happen, and this is not a bug.  If you run firefox under a
> CJK locale, pango chooses fonts for that locale.  Why do you expect it to choose
> digits from a Latin font?

yeah, you are right. But when common characters are among of Latin charaters,
pango will render it with Latin fonts. In a document, some numbers are rendered
with CJK fonts, and another numbers are rendered with Latin fonts. It's really
weird, and It is not be acceptable for CJK users. Maybe other locale users do
not accept it too.
Except my patch in Comment #8, there is another solutions. It's rendering all
common and Latin characters with locale fonts. Almost all local fonts support
common and Latin characters.
Do you think it reasonable?

Thanks
Shawn

-- Additional comment from besfahbo@redhat.com on 2007-02-13 13:43 EST --
If you really want to get the same common chars, just remove those from your
locale's font.

-- Additional comment from mcepl@redhat.com on 2007-02-14 04:13 EST --
Is Chinese supported in DejaVu fonts or is fonts-chinese still necessary?

-- Additional comment from besfahbo@redhat.com on 2007-02-14 13:24 EST --
fonts-chinese is necessary.  Chinese is not supported in DejaVu fonts, and note
that we ship dejavu-lgc-fonts in core, which only contains the
Latin/Greek/Cyrrillic subsets of dejavu.

-- Additional comment from petersen@redhat.com on 2007-02-15 00:44 EST --
But why are some numbers rendered in one font and others in different one?
Comment 1 Peng Huang 2007-02-15 22:14:10 EST
Created attachment 148168 [details]
Screensohts from firefox & gedit

The attached files are some screen shots from firefox and gedit. In 1.png and
2.png, some digits and punctuations are rendered with different fonts. It
reduces pango's good rendering effect, And sometime it will cause some problem
when application calculates width from variable common characters. I think it
is necessary to fix it. And I think removing common and Latin glyphs from
locale's fonts are not easy. There are many fonts need fixing and some of
locale's fonts in RHEL are commercial copies, we must ask font companies to
modify them. It's very hard.:(
I think fixing it in pango is easier. Do you have some ideas to fix it in pango
and make the text output perfect? Please consider it. Thanks.
BTW, In 3.png, pango renders all common characters with Latin font, it's very
good.
In 4.png, pango renders all common and Latin characters with Chinese font, it
is also acceptable.

Thanks
Shawn Huang
Comment 2 Peng Huang 2007-02-25 22:26:07 EST
Created attachment 148776 [details]
Let pango process common script as Latin

Hi Behdad

I created a patch that processes common script as Latin. Could you review it?
Any comments are welcome.

Thanks
Shawn
Comment 3 Behdad Esfahbod 2007-02-27 13:30:31 EST
(In reply to comment #2)
> Created an attachment (id=148776) [edit]
> Let pango process common script as Latin
> 
> Hi Behdad
> 
> I created a patch that processes common script as Latin. Could you review it?
> Any comments are welcome.

What's the rationale for such a change?  Common characters are common to all
scripts.  They are not Latin.  If I write in Persian, I want common characters
chosen from my Persian font, not Latin font.

> Thanks
> Shawn

Comment 4 Peng Huang 2007-02-27 21:17:23 EST
(In reply to comment #3)
> (In reply to comment #2)
> What's the rationale for such a change?  Common characters are common to all
> scripts.  They are not Latin.  If I write in Persian, I want common characters
> chosen from my Persian font, not Latin font.

The patch will give a good result for Chinese users. Maybe it is not suitable
for all locales.:( As you said, you want common characters chosen from Persian
font, but when a common character is among of Latin characters, it is no chance
to be rendered by Persian, although it is a Persian article. Do you think it is
reasonable? I believe that different scripts need different processing logic for
common script. Do you agree with me? If not, do you have some ideas to resolve
this problem in Chinese locale? 

Thanks
Shawn
Comment 5 Liang Zhang 2007-09-28 04:54:08 EDT
The upstream bug:
http://bugzilla.gnome.org/show_bug.cgi?id=481210
Comment 6 Bug Zapper 2008-04-03 15:09:32 EDT
Based on the date this bug was created, it appears to have been reported
against rawhide during the development of a Fedora release that is no
longer maintained. In order to refocus our efforts as a project we are
flagging all of the open bugs for releases which are no longer
maintained. If this bug remains in NEEDINFO thirty (30) days from now,
we will automatically close it.

If you can reproduce this bug in a maintained Fedora version (7, 8, or
rawhide), please change this bug to the respective version and change
the status to ASSIGNED. (If you're unable to change the bug's version
or status, add a comment to the bug and someone will change it for you.)

Thanks for your help, and we apologize again that we haven't handled
these issues to this point.

The process we're following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.
Comment 7 Bug Zapper 2008-05-06 21:11:32 EDT
This bug has been in NEEDINFO for more than 30 days since feedback was
first requested. As a result we are closing it.

If you can reproduce this bug in the future against a maintained Fedora
version please feel free to reopen it against that version.

The process we're following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp
Comment 8 Jens Petersen 2008-05-06 21:30:43 EDT
This still seems to happen with firefox3 in F9.

Naively to users it still seems unnatural that already input text should
change font sometimes if more text is added.
Comment 9 Bug Zapper 2008-05-13 22:37:17 EDT
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 10 Tony Fu 2008-09-09 23:11:50 EDT
requested by Jens Petersen (#27995)
Comment 11 Bug Zapper 2009-06-09 18:27:35 EDT
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 12 A S Alam 2009-06-10 06:39:41 EDT
Bug Exist in Fedora 11 with following Pacakge
pango-1.24.1-1.fc11.x86_64
firefox-3.5-0.20.beta4.fc11.x86_64
Moving to Fedora 11
Comment 13 Jens Petersen 2009-10-23 01:23:22 EDT
firefox-3.5 in current f12 rawhide looks ok to me.

But the general pango issue is still unchanged I think.

(In reply to comment #3)
> What's the rationale for such a change?  Common characters are common to all
> scripts.  They are not Latin.  If I write in Persian, I want common characters
> chosen from my Persian font, not Latin font.

It is different for CJK - they have their own wide punctuation
and number glyphs.  For CJK COMMON and LATIN glyphs should be
rendered in the same font.
Comment 14 Peng Huang 2009-10-28 01:59:09 EDT
I can reproduce this problem in rawhide.
Please try:
1. LANG=ja_JP.UTF-8 firefox http://start.fedoraproject.org/
2. input 111 or 111a in search entry

111 and 111a will be rendered with different fonts
Comment 15 Bug Zapper 2009-11-16 02:54:32 EST
This bug appears to have been reported against 'rawhide' during the Fedora 12 development cycle.
Changing version to '12'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 16 Jens Petersen 2010-03-03 04:14:36 EST
*** Bug 529594 has been marked as a duplicate of this bug. ***
Comment 17 fujiwara 2010-03-03 05:13:26 EST
Created attachment 397520 [details]
Patch for fontconfig comcharset

Sigh, opening duplicated bugs.

Copying the description since it's under the discussion.

This is a feature to customize pango common script in fontconfig.

 <match target="font">
  <test name="family" compare="contains">
   <string>AR PL</string>
  </test> 
  <edit name="compcharset" mode="assign">
   <compcharset>
   <complang>zh</complang>
   <compmode>weak</compmode>
   <string>0x00-0x7F</string>
   </compcharset>
  </edit>
 </match>

If the config file has the lines, fc-match shows the composite char flags.
The idea is, if compmode is weak, the code point is weak in the specified font
and get the char from other fonts when "monospace", "sans" or "sans-serif" is
referred.
Comment 18 fujiwara 2010-03-03 05:17:13 EST
Created attachment 397522 [details]
Patch for pango compcharset

The patch is the pango part with attachment #397520 [details] .
If there is no specific comments, I'll file the patches in upstream bugs. 

Now this pango patch also could support that 'compmode' is strong.

 <match target="font">
  <test name="family" compare="contains">
   <string>VL Gothic</string>
  </test> 
  <edit name="compcharset" mode="assign">
   <compcharset>
   <complang>ja</complang>
   <compmode>strong</compmode>
   <string>all</string>
   </compcharset>
  </edit>
 </match>

Then pango latin, common, hira/kana, greek, etc could use ja fonts on ja
locale.
Comment 19 fujiwara 2010-03-03 05:18:16 EST
Filed the upstreamed bug in pango side at the moment.
https://bugzilla.gnome.org/show_bug.cgi?id=604149
Comment 21 Bug Zapper 2010-11-04 08:12:26 EDT
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 22 fujiwara 2010-11-04 21:56:49 EDT
Probably I need to be back again.
Comment 23 A S Alam 2011-08-18 02:54:34 EDT
bug still exist in fedora 16 with following packages:
pango-1.29.3-2.fc16.x86_64
firefox-6.0-1.fc16.x86_64
xulrunner-6.0-1.fc16.x86_64
qt-4.8.0-0.6.beta1.fc16.x86_64

Note You need to log in before you can comment on or make changes to this bug.