Created attachment 1536365 [details] 官话 and 简体中文（新加坡） are not available on zanata Description of problem: During the F30-rawhide installation process, Anaconda lists 5 different script variants of Chinese in the language selection interface, but only 3 of them are available on Zanata. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. Start F30-rawhide installation process 2. Search "Chinese" in the language selection interface Actual results: You will see 5 different script variants of Chinese – zh_Hans_CN 简体中文（中国）, zh_Hant_TW 繁體中文（台灣）, 官话 (maybe Mandarin Chinese?), zh_Hant_HK 繁體中文（中華人民共和國香港特别行政區）, and zh_Hans_SG 简体中文（新加坡）. 1. Neither 官话 nor 简体中文（新加坡） is available on Zanata. So, if a user select one of them, he/she will get 0% localization help during the installation process. 2. 官话 is rather a spoken language than a script (aka writing systems). Listing it here may cause Chinese users confused. Expected results: Please consider filtering those two languages to make the list correspond to Zanata. Additional info: See the discussion on GitHub -> https://github.com/rhinstaller/anaconda/issues/1826
"官话" now shows in the GA installation media and is draw some discussion about why it's there. I believe this really need to be filtered out before next release.
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle. Changing version to 31.
Can reproduce with current rawhide: 官话 is not in the list but 简体中文（新加坡）is, and it is not translated.
It's probably worth considering why this works for everything else. I think somewhere in the whole works there's a fallback to the language's default locale-translation in case the selected locale is not translated. (See below why this should be the case.) But why does this not work for zh_SG in particular? Mike, could you please check langtable's data if there is anything special for this case? It might be that there's a default missing, impossible, or something along these lines. ---- Notes to self... The offending item as of now is zh_SG it seems. It might be we keep shooting ourselves in foot, as far localization is concerned. In LangLocaleHandler.initialize() we first list languages with translations, then we put these on a list and add locales to these. Except translations are for the "locales", not languages. My terminology might be wrong here, so an example - we detect files for zh_CN and say we have zh. So the whole premise of get_available_translations() might be wrong. It gives us languages (zh), but these are not actually a correct representation of what is available, it's locales (zh_CN) that get translations. Then later we try to rectify that by adding the actual locales to the second list, as sub-items for the languages. That itself is correct, but not enough. We depend on langtable to give us the locales via get_language_locales(), and disregard the earlier-found (non)presence of translations. That adds a dilemma, as LangLocaleHandler is a mixin used in both the welcome spoke and language/localization spoke. The former should filter by available translations (see this bug), the latter most probably not. One way might be to have some sort of filter on WelcomeLanguageSpoke._add_locale(). That might need memoizing as the locales are apparently re-listed on every language change on that spoke. While at it, it might be beneficial to rethink this whole "store stuff in gtk container instead of a python structure" approach. But it might grow the scope too much, too. Regardless of all that, a langtable-only fix can not be ruled out, per the first paragraph.
Got it now. The bug is actually quite fascinating. It can be interpreted in two ways: - Anaconda should not show locales ("languages") that don't have a translation (see above) - The Chinese translation is laid out in a different manner than others and thus prevents successful fallbacks (see below) ---- How this works. Ultimately we try to set the locale specified by UI (whether translation exists or not), and it all bubbles down to setlocale(3). This somehow decides if a locale is valid or not. Then we try to load the translation for that, with gettext. What happens next is that gettext tries to fall back on simplified specifications of the locale (_nl_make_l10nflist, around line 300-ish). What I *think* is the case -intentionally- is that these fallbacks built into gettext have been used in a clever way to make translations match-able to all locales - or rather most of them. For anaconda specifically, the cases are: - English is already the default, so nothing could be noticed without a deep inspection of the texts. - Many languages have precisely one locale so the distinction between the language and a language+territory is moot (Czech, Filipino...). - Some languages have tons of locales but provide translation only for the base language, so all of the locales fallback onto that if selected (eg. Spanish, French...). - Some languages have the bare language translation and then some of their locales have their specific translation too, so everything is translated. (Portuguese, Serbian...). - Some languages have translation per language, and their locales should differ significantly and do not because they all fall back onto the bare language (Punjabi?). - Some languages have only translations per locale, and not all locales are covered, so their fallbacks can't go anywhere (Chinese). Arguably, having zh_CN translations as zh would fix the problem for us. --- The code-based solution would be to add a check if translation for a particular locale exists, and use that when adding locales. The problem with that is that it would have to take into account gettext's fallback algorithm, and I'm not sure if that can be reused.
allow me to change version to 33 as this bug still present in f33.
PR with fix candidate: https://github.com/rhinstaller/anaconda/pull/2885 (Side note - it's too late to get this into f33.)
PR merged. It can't get into f33, but f34 will be fixed.
(In reply to Vladimír Slávik from comment #8) > PR merged. It can't get into f33, but f34 will be fixed. Thank you for your effort.