Bug 211660

Is this a dupe of bug 149991?

Yes, but 149991 was for FC.

This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

This works for me on fresh install environment. what I tried is:

case a)
1. run thunderbird with LANG=ja_JP.UTF-8.
2. open a composer
3. make sure a character encoding is ISO-2022-JP
4. try examples above.

case b)
1. run thunderbird with LANG=en_US.UTF-8 (make sure there are no .thunderbird to
not get confused)
2. open a composer
3. change a character encoding to ISO-2022-JP
4. try examples above.

It sounds like we need more info from you, Warren. e.g. screenshot, the font
package list installed and your env was upgrade or fresh install etc.

Ok, I found the way to reproduce this. the steps are:

1. run thunderbird (I could confirm this with LANG=ja_JP.UTF-8 and en_US.UTF-8)
2. open a composer
3. change character encoding to ISO-2022-JP (Options->Character Encoding->Japanese)
4. type '1' 4 times
5. type 'a' 4 times
6. move back the cursor

Actual Results:
after type 'a', type face is changed and you will see the cursor doesn't point
to  the proper position during moving back to the beginning of line.

This is because firefox/thunderbird are going to get the widths of characters
with the strings between the beginning of line and the current cursor position.
i.e. when "123a" is there in the text area and the cursor is at the end of line,
and press left key to move back, nsFontMetricsPango::GetWidth is called with
"123". however, as you realized on the above steps to reproduce this, Pango uses
different font on "123" and "123a" as long as the kind of alias fonts are sets.
so this problem seems to happens.

FWIW this behavior depends on fonts that used for only numeral characters and
others. if they has the same width, it just looks like no problem of course, but
anyway.

So possible solution for this issue are:

a) if there are any ways to know which faces Pango currently uses but not any
alias names, we could set the real face to PangoLayout temporarily and get the
correct width.

b) to modify GetWidth to be invoked with a string of current line and an offset
of current cursor position. this way should works even if alias font are sets as
current font.

reassigning this bug to firefox so that this also happens on firefox too.
both code base is the same and firefox is tier-1 application for us. after
getting this fix in firefox, we can apply it to thunderbird later I think.

Bug still exist in Snapshot #3.
---
Please refer to Release Criteria Secion 8.F.3

8.I18N-> F.Firefox-> 3.Rendering of supported languages (pango enabled)

<http://intranet.corp.redhat.com/ic/intranet/RHEL500ReleaseCriteria#i18n>

Created attachment 143508 [details]
proposed patch

I've worked out a patch on firefox based on Behdad's new pango printing patch,
which was posted to upstream bugzilla.

Update: Behdad is testing the patch, says it looks incomplete and it is hard to
fix the problem completely before FF3

Ok, please let me know if there are any testcases that still has a problem. it
just works for me at least with the above testcases and the mixture of ASCII and
Japanese say.

What I've tried aside from the above testcases is:

Testcase a)
 1. type 123
 2. press ctrl+space to enable scim.
 3. type nihongo and hit space then enter. should looks like 123日本語

Testcase b)
 1. type 123
 2. press ctrl+space to enable scim.
 3. type nihongo and hit space then enter.
 4. type 123

Testcase c)
 1. type 123
 2. press ctrl+space to enable scim.
 3. type nihongo and hit space then enter.
 4. type 123
 5. type abc

Setting dev ack, since we have a patch

Patch tested.  I think we can pick it up and it fixes the problem in this bug.

Behdad, Chris had problems applying the patch. Can you redo it against firefox
as in pkgcvs ?

I've built the patch into firefox and thunderbird in RHEL5.  A minor issue that
I just noticed is that the error underline is still at the wrong position. 
Attaching shot.

Created attachment 144058 [details]
screenshot showing the remaining problem

As you see in the shot, the error underline is positioned incorrectly.	The fix
should be as simple as the committed patch, using GetRangeWidth in one more
place.

Created attachment 144076 [details]
another patch to fix underline position for spellchecking

Thanks for catching that up. attached patch should fixes that issue.

Thanks Akira.  Tested and built ff and tb.

Tested the bug for both thunderbird and firefox.

The cursor is not mispositioning and it does point
to the proper position during moving back to the beginning of line BUT the face
or shape of numbers are changing for en_US and Indic locales. i followed the
steps mentioned in comments #4, #5, #11.  For ja, ch the face of the numbers are
not changing, but for en, indic, its changing, once u type aaa or any indic char
after 11111 or 123456789 or any number. 

Component Versions Tested :
For Thunderbird : 
   thunderbird-1.5.0.9-4.el5 
   htmlview-4.0.0-1.el5
   launchmail-4.0.0-1.el5 
For firefox :
   firefox-1.5.0.9-3.el5
   devhelp-0.12-9.el5
   yelp-2.16.0-13.el5

Packages available in tree :
http://porkchop.redhat.com/nightly/RHEL5-Client-20061221.nightly/tree-i386/Client/

So, if the face of char changing is not the correct behavior, I am moving it to
ASSIGNED for now. Please inform if face change is a correct behavior, then I
will make it VERIFIED and CLOSE it.

Additional Info : Screenshots of numbers with and without char for both packages
 are attached, pls hv a look.

Created attachment 144257 [details]
Thunderbird-screenshot-for-numbers

Created attachment 144258 [details]
Firefox-screenshot-for-numbers

In general, you can easily determine that comparing to the previous version if
that happened on it too or that's an regression on this fix. if this fix
introduces an regression, it should be corrected though, otherwise it shouldn't
prevent the resolution. instead should file an another bug.  In adition, as I
described the behavior on comment #5, I can see it was happened originally. in
any case it's not related to this directly. back to MODIFIED.

I don't think this is a regression.  We have a bug filed about that specific
behavior you are describing (fonts changing).

Tested this Bug. This issue is solved now. Not reproducable to my Test Machine.

Component Version Tested :
firefox-1.5.0.9-4.el5

Moving it to VERIFIED->RESOLVED for now.

Created attachment 144888 [details]
screenshots

The problem was still found in RHEL5-Client-20061226.nightly:
rpm -q pango cairo freetype firefox
pango-1.14.9-3.el5
cairo-1.2.4-1.fc6
freetype-2.2.1-16.el5
firefox-1.5.0.9-4.el5

Steps to Reproduce:
1. $ LANG=zh_CN.UTF-8 firefox.
2. Try typing some words which contains number and punctuation into an edit
box.
3. Try deleting chars by BackSpace.
  
Actual results:
2. The glyphs change size.
   The last char you typed cannot be rendered normally (no char displayed or
just half of char displayed).
	The cursor became strange.
3. The words were not deleted immediately (need to refresh the screen by
dragging the scrollbar and sometimes the words were not cleaned totally).

Yep, I still see this in our bugzilla pages for example for ja_JP.UTF-8 firefox.

Akira, can you take another look at this?

Ok, I got this problem. space's width seems wrong here.

After some investigation on that, I see the rendering regions miscalculated
caused by the font change behavior on pango. we may need the kind of
GetRangeWidth() for GetTextDimensions() too, like GetRangeTextDimensions() say
though, I saw another rendering breakage on just hacking
nsTextFrame::MeasureText with overwriting width with GetRangeWidth(). so that
way may be wrong or need more fix on others...
FWIW actually calculating the cursor position in firefox was no problem, but it
was just limited by the rendering rectangle regions:

In nsTextFrame::GetPointFromOffset
  if (width > mRect.width)
    outPoint->x = mRect.width;
  else
    outPoint->x = width;

We could file an another bug for that since it appears regardless of the sort of
fixes on this bug. and that is close to an rendering issue rather than just the
cursor positioning issue. or we may want to just fix the font change behavior
since it triggers that.

Akira, can you attach your working patch?
So, the problem is that the width after deleting a character becomes smaller
than the width for the same range before the deletion?  And that the difference
is not cleared?

Created attachment 145770 [details]
work-in-progress patch

Sure. well, this issue just appears when you type a space between numeric and
alphabet say.

Steps to reproduce:
1. type 123 and hit a space
2. type a

Actual result:
after type 'a', the font is changed and render them within old dimension.
therefore half of character are displayed. and since the cursor doesn't points
to the proper position by the above reason, characters isn't cleared properly
with the deletion.

Current state of firefox is in much better position than it was before. Hence, I
agree with Tagoh-san to resolve this bug and open a new bug for the remaining
issue highlighted in Comment #31 (numbers + space + letters), and include the
new patch in the next firefox update.

This bug seems to have caused bug 223156, so I'm not sure I'd define the
behavior as "better".  This only affected a few locales with pango, and the fix
seems to have broke behavior on all locales, regardless of pango or not...

Still found the problem in ko_KR locale by inputting punctuations and letters
(No need to type numbers) on RHEL5-Client-20070111.0 Snap #7. 
1. ./;/'/:/, + letters -> has problem
2. ./;/'/:/, + numbers ->ok
3. numbers + letters ->ok 

Akira, do you have a finished patch?

This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Created attachment 271071 [details]
screenshot got from RHEL5.1

rpm -q pango cairo freetype firefox
pango-1.14.9-3.el5
cairo-1.2.4-2.el5
freetype-2.2.1-19.el5
firefox-1.5.0.12-3.el5

I did not find the problems which were found in Comment #25 in
RHEL5.1-Client-20071017.0 (zh_CN locale) except when I type letter after symbol
the font was changed automatically. Please refer to the attached screenshot. I
remember there was another bug for my findings but did not search it out.

Thanks ndai.

Since this bugzilla report was filed, we have seriously upgraded Gecko-related
packages, which may have resolved this issue. Users who have experienced this
problem are encouraged to upgrade their system to the latest version of their
distribution available.

Please, confirm to us that this bug is reproducible on the latest upgrade of the
supported distribution (that's RHEL, or Fedora 7, 8, and Rawhide).

Setting the bug to NEEDINFO. If I won't get confirmation of reproducability in
30 days, the bug will be closed as INSUFFICIENT_DATA.

[This is mass-changing of bugs which seem to be too old and irrelevant anymore;
we are sorry, if this bug should not be incldued.]

I have confirmed that this is still broken in Fedora 8.  I don't have ready
access to RHEL5 to test it there.  Can somebody else retest?

I am pretty sure it is broken in all firefox 2 versions.

I did some testing(given in comment#5 and comment#11) on RHEL5.2 nightly as well
as rawhide with Firefox 3 and see that problem described here is fixed now. But
can anyone also cross verify whether this is fixed or not?

There may still be some changes of fonts in some cases, but I believe cursor
movement is fixed in Firefox 3, which is the crucial part.

This request was previously evaluated by Red Hat Product Management
for inclusion in the current Red Hat Enterprise Linux release, but
Red Hat was unable to resolve it in time.  This request will be
reviewed for a future Red Hat Enterprise Linux release.

I believe this is fixed with Firefox 3 and later which is now in RHEL5.x.