Bug 872786 - [sr@latin] 'Serbian (Latin)' not listed in languages menu; 'Serbian (Serbia)' listed twice
[sr@latin] 'Serbian (Latin)' not listed in languages menu; 'Serbian (Serbia)'...
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: anaconda (Show other bugs)
18
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Vratislav Podzimek
Fedora Extras Quality Assurance
: i18n
Depends On:
Blocks: F18Blocker/F18FinalBlocker
  Show dependency treegraph
 
Reported: 2012-11-03 00:35 EDT by Steve Tyler
Modified: 2012-11-22 14:29 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-11-22 14:29:34 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
screenshot showing 'Serbian (Serbia)' listed twice (115.84 KB, image/png)
2012-11-03 00:35 EDT, Steve Tyler
no flags Details
patch to list 'Serbian (Latin, Serbia)' in language menus (781 bytes, patch)
2012-11-04 23:55 EST, Steve Tyler
no flags Details | Diff
screenshot showing 'Serbian (Latin, Serbia)' in language menu (119.67 KB, image/png)
2012-11-04 23:58 EST, Steve Tyler
no flags Details
patch to list 'Serbian (Latin, Serbia)' in language menus with { "sr@latin": "sr_Latn_RS" } (775 bytes, patch)
2012-11-05 00:51 EST, Steve Tyler
no flags Details | Diff
localization-test-report-18_24_1_EXP_1.txt with { "sr@latin": "sr_Latn_RS" } patch (7.47 KB, text/plain)
2012-11-05 08:36 EST, Steve Tyler
no flags Details

  None (edit)
Description Steve Tyler 2012-11-03 00:35:55 EDT
Created attachment 637382 [details]
screenshot showing 'Serbian (Serbia)' listed twice

Description of problem:
'Serbian (Latin)' is not listed in the languages menu, and 'Serbian (Serbia)' is listed twice. Users cannot tell whether the menu entries correspond to the same locale or to different locales. And, if the latter, users cannot tell which locale will be configured by which entry.

Test installs confirm that the entries configure different locales:
1. sr_RS.UTF-8
2. sr_RS.UTF-8@latin

Version-Release number of selected component (if applicable):
anaconda-18.24-1

How reproducible:
Always.

Steps to Reproduce:
1. Start installer.
2. Scroll to 'Serbian (Serbia)'.
  
Actual results:
'Serbian (Serbia)' is listed twice.
See attached screenshot.

Expected results:
'Serbian (Serbia)' is listed once.
'Serbian (Latin)' is listed once.

Additional info:
Bug 866730 - invalid locales configured for some languages
Bug 858591 - anaconda setting invalid system locale xx.UTF-8 not xx_YY.UTF-8
Comment 1 Steve Tyler 2012-11-03 00:49:31 EDT
[Copied from Bug 866730, Comment 39]
Vratislav Podzimek 2012-11-02 22:15:58 UTC

(In reply to comment #38)
> Created attachment 637237 [details]
> localization-test-report-18_23_1_EXP_2.txt
> 
> The locale is 'sr.UTF-8@latin', not 'sr_RS.UTF-8@latin'
The mangleMap update is needed. And the same goes for:
> $ LANG='sr.UTF-8@latin' locale

> The english_name is 'Serbian', not 'Serbian (Latin)'.
This really doesn't have a reasonable (without additional mapping) solution. But it really should be resolved in babel and not in Anaconda. Also "Serbian (Latin)" doesn't fit in the rest of the list with the "language (territory)" pattern.

I'm posting your patch to the anaconda-patches. If it gets approved for F18, I will push it. Otherwise I'd like to push my patch (with mangleMap update) for F18 and hope that it will fulfill the requirements of the NTH flag. Having "Serbian (Serbia)" twice is not such a big deal.

One thing that I don't understand -- I thought that one could see which one is latin by trying to select one or the other, but both appear to have the same translations (both using cyrilic not latin).
Comment 2 Steve Tyler 2012-11-03 01:54:52 EDT
The name 'Serbian (Latin)' is used at Transifex[1] and is used in the anaconda-17.29-1 lang-table[2].

python-babel also returns that name:[3]

>>> print babel.Locale.parse('sr_Latn').english_name
Serbian (Latin)
>>> print babel.Locale.parse('sr_Latn_RS').english_name
Serbian (Latin, Serbia)

python-babel returns Latin or Cyrillic display names depending on the locale name:

>>> print babel.Locale.parse('sr_Latn').display_name
Srpski (Latinica)
>>> print babel.Locale.parse('sr_Latn_RS').display_name
Srpski (Latinica, Srbija)
>>> print babel.Locale.parse('sr_Cyrl').display_name
Српски (Ћирилица)
>>> print babel.Locale.parse('sr_Cyrl_RS').display_name
Српски (Ћирилица, Србија)

[1] https://fedora.transifex.com/projects/p/fedora/language/sr@latin/

[2] http://git.fedorahosted.org/cgit/anaconda.git/tree/data/lang-table?id=anaconda-17.29-1
    Serbian        sr       True sr_RS.UTF-8       sr-cy    Europe/Belgrade
    Serbian(Latin) sr@latin True sr_RS.UTF-8@latin sr-latin Europe/Belgrade

[3] For a full list of python-babel locale names matching 'sr_*':
$ rpm -ql python-babel | grep localedata | grep 'sr_'
Comment 3 Steve Tyler 2012-11-03 17:05:14 EDT
(In reply to comment #1)
> [Copied from Bug 866730, Comment 39]
> Vratislav Podzimek 2012-11-02 22:15:58 UTC
...
> One thing that I don't understand -- I thought that one could see which one
> is latin by trying to select one or the other, but both appear to have the
> same translations (both using cyrilic not latin).

A search at Transifex for "what language" shows that the translation uses Latin glyphs:[1]

Source string
"What language would you like to use during the installation process?"

Translation
"Koji jezik biste želeli da koristite tokom procesa instalacije?"

The Latin string is also in anaconda-18.24-1:

[liveuser@localhost xfr]$ strings /usr/share/locale/sr@latin/LC_MESSAGES/anaconda.mo | egrep 'Koji jezik|koristite tokom'
Koji jezik biste 
eleli da koristite tokom procesa instalacije?
[liveuser@localhost xfr]$ rpm -q anaconda
anaconda-18.24-1.fc18.x86_64

[1] Anaconda  /  Resources (sr@latin)  /  Master  / Serbian (Latin) (sr@latin)
https://fedora.transifex.com/projects/p/anaconda/resource/master/l/sr@latin/view/
Comment 4 Steve Tyler 2012-11-04 23:55:55 EST
Created attachment 638319 [details]
patch to list 'Serbian (Latin, Serbia)' in language menus

With this patch, 'Serbian (Latin, Serbia)' is listed in the language menus. Test installs verify that the locale strings are as expected:

'Serbian (Serbia)'          sr_RS.UTF-8
'Serbian (Latin, Serbia)'   sr_RS.UTF-8@latin

The translation is not complete enough to verify that Serbian (Latin) translations are actually displayed in the installer. I could find only one string -- in the Language spoke.

The string 'sr_Latn_RS' is the name of a Babel locale:

>>> 'sr_Latn_RS' in babel.localedata.list()
True
>>> babel.Locale.parse('sr_Latn_RS').__dict__
{'_Locale__data': None, 'territory': 'RS', 'variant': None, 'language': 'sr', 'script': 'Latn'}
>>> print babel.Locale.parse('sr_Latn_RS').english_name
Serbian (Latin, Serbia)
>>> print babel.Locale.parse('sr_Latn_RS').display_name
Srpski (Latinica, Srbija)

A more general solution would be welcome ... :-)
Comment 5 Steve Tyler 2012-11-04 23:58:39 EST
Created attachment 638320 [details]
screenshot showing 'Serbian (Latin, Serbia)' in language menu
Comment 6 Steve Tyler 2012-11-05 00:51:06 EST
Created attachment 638335 [details]
patch to list 'Serbian (Latin, Serbia)' in language menus with { "sr@latin": "sr_Latn_RS" }

Your previous patches already make this simplification possible:[1]

{ "sr@latin": "sr_Latn_RS" }

Test installs verify that the i18n locale strings are as expected:

'Serbian (Serbia)'          sr_RS.UTF-8
'Serbian (Latin, Serbia)'   sr_RS.UTF-8@latin

Now, the mangleMap can be viewed as a mapping between translation directory names[2] and Babel locale names[3].

[1] Obsoleting localization-sr_Latn_RS_at_latin-1.patch.
    Attached patch is against anaconda-18.24-1.
[2] $ find /usr/share/locale/ -name anaconda.mo | sort
[3] $ rpm -ql python-babel | grep localedata
Comment 7 Vratislav Podzimek 2012-11-05 04:43:36 EST
(In reply to comment #6)
> Created attachment 638335 [details]
> patch to list 'Serbian (Latin, Serbia)' in language menus with { "sr@latin":
> "sr_Latn_RS" }
> 
> Your previous patches already make this simplification possible:[1]
> 
> { "sr@latin": "sr_Latn_RS" }
I haven't tested this change yet, but doesn't it result in "sr_Latn_RS" set as system language/locale?

$ LANG=sr_Latn_RS locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=sr_Latn_RS

$ LANG='sr_RS@latin' locale
LANG=sr_RS@latin
LC_CTYPE="sr_RS@latin"
LC_NUMERIC="sr_RS@latin"
Comment 8 Steve Tyler 2012-11-05 08:36:40 EST
Created attachment 638618 [details]
localization-test-report-18_24_1_EXP_1.txt with { "sr@latin": "sr_Latn_RS" } patch

With localization-sr_Latn_RS_at_latin-2.patch, the locale validation test passes:

L:  62: sr_RS.UTF-8         ...     'Serbian (Serbia)'            
L:  63: sr_RS.UTF-8@latin   ...     'Serbian (Latin, Serbia)'     

Full report attached.
Comment 9 Vratislav Podzimek 2012-11-06 05:37:06 EST
Yeah, I understand that test passes. But the patch you are proposing is something I've tested when I was writing the patch for bug 866730 and did not included because it causes traceback. "sr_Latn_RS" is not a valid locale and setting it in Anaconda as value of $LANG (which is the result of the patch you propose) results in traceback(s).
Comment 10 Steve Tyler 2012-11-06 15:12:06 EST
(In reply to comment #9)
> Yeah, I understand that test passes. But the patch you are proposing is
> something I've tested when I was writing the patch for bug 866730 and did
> not included because it causes traceback. "sr_Latn_RS" is not a valid locale
> and setting it in Anaconda as value of $LANG (which is the result of the
> patch you propose) results in traceback(s).

'sr_Latn_RS' is not a valid _i81n_ locale. 'sr_Latn_RS' is a _Unicode_ locale.[1] As I reported in Comment 6:

"""
Test installs verify that the i18n locale strings are as expected:

'Serbian (Serbia)'          sr_RS.UTF-8
'Serbian (Latin, Serbia)'   sr_RS.UTF-8@latin
"""

The testing was with anaconda-18.24-1. If there had been any tracebacks, I certainly would not be proposing a patch that caused them ... :-)

How did you get tracebacks?

[1] http://cldr.unicode.org/
Comment 11 Steve Tyler 2012-11-06 16:00:54 EST
I don't get any tracebacks with a patched anaconda-18.25-1 and Fedora-18-Beta-TC7-x86_64-Live-Desktop.iso. Test installs complete normally, and the i18n locale names are as expected.

Test with:
$ qemu-kvm -m 2048 -hda f18-test-2.img -cdrom ~/xfr/fedora/F18/F18-Beta/TC7/Fedora-18-Beta-TC7-x86_64-Live-Desktop.iso -usb -vga qxl -boot menu=on -usbdevice mouse
Comment 12 Vratislav Podzimek 2012-11-07 06:54:48 EST
Now neither I get any tracebacks. I probably used "sr_RS_Latn" instead of "sr_Latn_RS" in the previous cases. Sorry for the noise.

So, we could use this patch to have "Serbian (Latin, Serbia)" correctly listed and I'm posting the patch to anaconda-patches list.

But to get it to Fedora 18 this bug would have to be NTH. Otherwise this could go to Fedora 19, but we would really like to completely remove the mangleMap for the future releases.
Comment 13 Steve Tyler 2012-11-07 10:45:48 EST
Proposing as an F18 Final Blocker:

1. Cannot be fixed with an update and anaconda is a Critical Path package.

2. Under F18 Final Release Requirement 17:

"17. Menu sanity ... No application may unintentionally appear twice in the menus. ..."

The installer languages menu is not an applications menu, but the problem is the same -- duplicate entries are ambiguous as to what each does. See Comment 0.

Fedora 18 Final Release Criteria
https://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria
Comment 14 Vratislav Podzimek 2012-11-08 05:07:30 EST
Patch pushed to master (Rawhide). If this gets approved for F18 I will cherry-pick it to fedora-18-beta-branch.
Comment 15 Fedora Update System 2012-11-09 19:57:29 EST
anaconda-18.28-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/FEDORA-2012-17823/anaconda-18.28-1.fc18
Comment 16 Fedora Update System 2012-11-10 14:40:24 EST
Package anaconda-18.28-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing anaconda-18.28-1.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-17823/anaconda-18.28-1.fc18
then log in and leave karma (feedback).
Comment 17 Adam Williamson 2012-11-22 14:29:34 EST
18.28 went stable, I verified this is fixed in RC1. closing.

Note You need to log in before you can comment on or make changes to this bug.