This is a tracking bug for Change: Glibc locale subpackaging For more details, see: https://fedoraproject.org//wiki/Changes/Glibc_locale_subpackaging This change should make it possible to install or uninstall locales individually.
This message is a reminder that Fedora 23 Change Checkpoint: Completion deadline (testable) is on 2015-07-28 [1]. At this point, all accepted Changes should be substantially complete, and testable. Additionally, if a change is to be enabled by default, it must be so enabled at Change Completion deadline. This bug should be set at least to the MODIFIED state to indicate that it achieved completeness. Status will be provided to FESCo right after the deadline. If, for any reasons, your Change is not in required state, let me know and we will try to find solution. For Changes you decide to cancel/move to the next release, please use the NEW status and set needinfo on me and it will be acted upon. In case of any questions, don't hesitate to ask Wrangler (jkurik). Thank you. [1] https://fedoraproject.org/wiki/Releases/23/Schedule
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle. Changing version to '23'. (As we did not run this process for some time, it could affect also pre-Fedora 23 development cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.) More information and reason for this action is here: https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23
There are still too many open questions and I have not yet started the implementation. Therefore, I think this must be moved to rawhide.
Thanks Mike for providing update, I am removing(by commenting) this Change from https://fedoraproject.org/wiki/Releases/23/ChangeSet and moving back the category of wiki page as ChangePageIncomplete state.
The proposed change is pushed to the git branch "private-mfabian-locale-subpackaging-folders" in our Fedora git repository for the glibc package. Packages for testing the locale sub-packaging on F23 are here: https://mfabian.fedorapeople.org/glibc/
Copr repo for testing: https://copr.fedoraproject.org/coprs/mfabian/glibc/
New branch for this in the git-dist repo for glibc: private-mfabian-locale-subpackaging-folders-new
On 2016-Feb-23, we have reached Fedora 24 Change Checkpoint: Completion deadline (testable). At this point, all accepted changes should be substantially complete, and testable. Additionally, if a change is to be enabled by default, it must be so enabled at Change Completion deadline. Change tracking bug should be set to the MODIFIED state to indicate it achieved completeness. Incomplete and non testable Changes will be reported to FESCo on 2016-Feb-26 meeting. Contingency plan for System Wide Changes, if planned for Alpha (or in case of serious doubts regarding Change completion), will be activated.
The latest work is now in the private-mfabian-locale-subpackaging-folders-2016-02-25 branch in dist-git. Seems to work fine. After a final review by Carlos O'Donell, I think I can push it to rawhide/f24 today.
Hello. After the today's "yum update" (on f24) I've lost my locale and half of the programs stopped working. Others are unreadable. $ locale locale: Cannot set LC_CTYPE to default locale: No such file or directory locale: Cannot set LC_MESSAGES to default locale: No such file or directory locale: Cannot set LC_ALL to default locale: No such file or directory LANG=ru_RU.UTF-8 LC_CTYPE="ru_RU.UTF-8" LC_NUMERIC="ru_RU.UTF-8" LC_TIME="ru_RU.UTF-8" LC_COLLATE="ru_RU.UTF-8" LC_MONETARY="ru_RU.UTF-8" LC_MESSAGES="ru_RU.UTF-8" LC_PAPER="ru_RU.UTF-8" LC_NAME="ru_RU.UTF-8" LC_ADDRESS="ru_RU.UTF-8" LC_TELEPHONE="ru_RU.UTF-8" LC_MEASUREMENT="ru_RU.UTF-8" LC_IDENTIFICATION="ru_RU.UTF-8" LC_ALL= $ locale -a locale: Cannot set LC_CTYPE to default locale: No such file or directory locale: Cannot set LC_MESSAGES to default locale: No such file or directory locale: Cannot set LC_COLLATE to default locale: No such file or directory C C.utf8 POSIX $ gnome-terminal (process:3770): Gtk-WARNING **: Locale not supported by C library. Using the fallback 'C' locale. Error constructing proxy for org.gnome.Terminal:/org/gnome/Terminal/Factory0: Error calling StartServiceByName for org.gnome.Terminal: GDBus.Error:org.freedesktop.DBus.Error.Spawn.ChildExited: Process org.gnome.Terminal exited with status 9 How to bring the system back to live? Help!
(In reply to Stas Sergeev from comment #10) > Hello. > > After the today's "yum update" (on f24) I've lost my locale > and half of the programs stopped working. Others are unreadable. > > $ locale > locale: Cannot set LC_CTYPE to default locale: No such file or directory > locale: Cannot set LC_MESSAGES to default locale: No such file or directory > locale: Cannot set LC_ALL to default locale: No such file or directory > LANG=ru_RU.UTF-8 > LC_CTYPE="ru_RU.UTF-8" > LC_NUMERIC="ru_RU.UTF-8" > LC_TIME="ru_RU.UTF-8" > LC_COLLATE="ru_RU.UTF-8" > LC_MONETARY="ru_RU.UTF-8" > LC_MESSAGES="ru_RU.UTF-8" > LC_PAPER="ru_RU.UTF-8" > LC_NAME="ru_RU.UTF-8" > LC_ADDRESS="ru_RU.UTF-8" > LC_TELEPHONE="ru_RU.UTF-8" > LC_MEASUREMENT="ru_RU.UTF-8" > LC_IDENTIFICATION="ru_RU.UTF-8" > LC_ALL= > > $ locale -a > locale: Cannot set LC_CTYPE to default locale: No such file or directory > locale: Cannot set LC_MESSAGES to default locale: No such file or directory > locale: Cannot set LC_COLLATE to default locale: No such file or directory > C > C.utf8 > POSIX > > > $ gnome-terminal > (process:3770): Gtk-WARNING **: Locale not supported by C library. > Using the fallback 'C' locale. > Error constructing proxy for > org.gnome.Terminal:/org/gnome/Terminal/Factory0: Error calling > StartServiceByName for org.gnome.Terminal: > GDBus.Error:org.freedesktop.DBus.Error.Spawn.ChildExited: Process > org.gnome.Terminal exited with status 9 > > > How to bring the system back to live? Help! `dnf install glibc-all-langpacks` will fix this. We are working to resolver this issue as quickly as possible.
With glibc-2.23.90-3.fc25 and glibc-2.23.1-5.fc24 we have now fixed the issue in several important ways. See: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/WXOOUGA3YCB2O4O77JGLSJZ25BS4RFK5/ You should now always get glibc-all-langpacks by default.
I still don't understand why things broke in a way they did. If I would be switched the the C locale then fine. Why instead programs started failing to start, or write unreadable chars? Surely the fall-back should be handled better.
(In reply to Stas Sergeev from comment #13) > I still don't understand why things broke in a > way they did. If I would be switched the the C > locale then fine. Why instead programs started > failing to start, or write unreadable chars? > Surely the fall-back should be handled better. The failure is entirely on the part of the glibc team. We didn't test enough end-user applications to determine that several key applications would actually fail if a call to setlocale failed. In theory these applications are poorly written and not robust. These applications need to all be updated to handle running in C or C.UTF-8 locales. In practice we had to fix these cases by changing the dependency requirements to install all the glibc language packs when transitioning from a non-language-pack system to one that supports language-packs. We will continue to work to ensure this transition is smooth. We will also look into running systems in pure C.UTF-8 mode i.e. only glibc-minimal-langpack installed and file bugs for every program that fails to operate.
> The failure is entirely on the part of the glibc team. We didn't test enough > end-user applications to determine that several key applications would > actually fail if a call to setlocale failed. In theory these applications are > poorly written and not robust. It seems to me that even setlocale(LC_ALL,"") fails. According to the man page, For glibc, first (regardless of cate‐ gory), the environment variable LC_ALL is inspected, next the environ‐ ment variable with the same name as the category (see the table above), and finally the environment variable LANG. The first existing environ‐ ment variable is used. If its value is not a valid locale specifica‐ tion, the locale is unchanged, and setlocale() returns NULL. ... so in its current implementation it should indeed fail. Wouldn't it be better if it try some fall-back like "C" first? IMHO that glibc-specific implementation is not robust, not the apps. You want the apps to simply ignore the setlocale(LC_ALL,"") return value?
(In reply to Stas Sergeev from comment #15) > > The failure is entirely on the part of the glibc team. We didn't test enough > > end-user applications to determine that several key applications would > > actually fail if a call to setlocale failed. In theory these applications are > > poorly written and not robust. > It seems to me that even setlocale(LC_ALL,"") fails. Correct. > ... so in its current implementation it should indeed fail. Correct. > Wouldn't it be better if it try some fall-back like "C" first? No. It's the responsibility of the application to do this, because only the application knows if a translation is critical, particuarly when UTF-8 support is needed. > IMHO that glibc-specific implementation is not robust, not the apps. > You want the apps to simply ignore the setlocale(LC_ALL,"") return value? No. The application must decide if the locale is critical to operation or not. In many of the cases I've seen, particularly gnome-terminal, the terminal could have continued to operate with the C or C.UTF-8 locale. I see no reason for gnome-terminal to exit with an error. Therefor it should be robust and (a) try C.UTF-8 to get UTF-8 support, or otherwise (b) fall back to trying C, and using that. For something like ssh the question is more difficult to answer because converting from UTF-8 input to ASCII is lossy, and may result in output that is unrepresentable and makes the terminal useless. Even then ssh could also fall back to C.UTF-8 and if that doesn't work (it would allow you to represent all language characters but not their cultural conventions) then you might fail instead of trying C. Does that clarify our position?
> No. It's the responsibility of the application to do this, because only the > application knows if a translation is critical, particuarly when UTF-8 support > is needed. Could you please confirm that you mean exactly setlocale(LC_ALL,"")? I thought if an apps does this, it implies that the translation is not critical. Please note that the problem is not only that the apps do not start. I suppose the ones that do not start, are indeed locale-sensitive, and they do not do setlocale(LC_ALL,"") at first place (maybe they do setlocale(LC_ALL,"C.UTF-8") or something - then they should be fixed the way you suggest). The problem is that the apps that DO start, still write the unreadable chars. This presumably comes from the fact that setlocale(LC_ALL,"") fails, but they ignore the return value. I wonder if someone ever checks the return value of setlocale(LC_ALL,"") at all. So maybe at least this part of the problem can be fixed on a glibc level?
(In reply to Stas Sergeev from comment #17) > > No. It's the responsibility of the application to do this, because only the > > application knows if a translation is critical, particuarly when UTF-8 support > > is needed. > Could you please confirm that you mean exactly setlocale(LC_ALL,"")? Calling setlocale(LC_ALL,"") simply means that the application has been tested with and expects to work with *all* locales the user might possible set via environment variables. Such a call *may* fail if the user sets invalid values in their environment variables, and the application needs to decide if it should or should not support this case. > I thought if an apps does this, it implies that the translation > is not critical. It does not imply that at all. Calling setlocale(LC_ALL,"") means that the environment is examined to set the locale. The application must still decide if failure is fatal or not. > The problem is that the apps that DO start, still write the unreadable > chars. This presumably comes from the fact that setlocale(LC_ALL,"") > fails, but they ignore the return value. Yes. If you used C.UTF-8 you'd always be able to write *any* characters. > I wonder if someone ever checks the return value of setlocale(LC_ALL,"") > at all. So maybe at least this part of the problem can be fixed on a > glibc level? The only fix at the glibc level is to provide C.UTF-8 for application usage and to make it a standard for all distributions such that upstream can rely on it. If you find a distribution that *doesn't* have C.UTF-8, please contact me and I'll work with the liason for that distribution in upstream glibc to discuss options and solutions.
(In reply to Stas Sergeev from comment #17) > > No. It's the responsibility of the application to do this, because only the > > application knows if a translation is critical, particuarly when UTF-8 support > > is needed. > Could you please confirm that you mean exactly setlocale(LC_ALL,"")? > I thought if an apps does this, it implies that the translation > is not critical. Please note that the problem is not only that > the apps do not start. I suppose the ones that do not start, are > indeed locale-sensitive, and they do not do setlocale(LC_ALL,"") > at first place (maybe they do setlocale(LC_ALL,"C.UTF-8") or something - > then they should be fixed the way you suggest). I think you misunderstand what setlocale(LC_ALL,"") means. man setlocale> If locale is an empty string, "", each part of the locale man setlocale> that should be modified is set according to the environment man setlocale> variables. So if an app uses setlocale(LC_ALL, ""), it does not mean at all that it does not want to use translations. It sets the locales according ot the environment variables, which contain the locales the user wants to use, including the locale the use wants to use for the translation. setlocale(LC_ALL,"C.UTF-8") does *not* use the environment variables but just sets the C.UTF-8 locale. The user will not get translations in that case.
> Calling setlocale(LC_ALL,"") simply means that the application has been tested > with and expects to work with *all* locales the user might possible set via > environment variables. Yes, so since it can work with all and any locale (even "C" or "C.UTF-8"), then I don't see a big problem if such locale is forced to it as a fall-back. As you say, it can work with anything, so why would it care? Why would it break on such a fall-back? So that's the logic I'd like to put for considerations. Thanks for your explanations!
> So if an app uses setlocale(LC_ALL, ""), it does not mean at all that > it does not want to use translations. No, I didn't mean that either. Its just that it can work with anything user wants. And since it can work with anything, it looks exactly like the right place for the fall-back. Is it not? :) Anyway, that's just an idea.
(In reply to Stas Sergeev from comment #21) > > So if an app uses setlocale(LC_ALL, ""), it does not mean at all that > > it does not want to use translations. > No, I didn't mean that either. > Its just that it can work with anything user wants. > And since it can work with anything, it looks exactly > like the right place for the fall-back. Is it not? :) > Anyway, that's just an idea. Except that glibc is the wrong place to decide this and it would violate the specification of the API. Only the application knows if a fallback is OK if the user's selected locale is missing. There is too much application specific knowledge required to make that decision. So we leave it up to the application: ... /* Try to load the locale the user wanted. */ char old_locale; if ((old_locale = setlocale (LC_ALL, "")) == NULL) { /* Given that we are mostly a UI app, we'll ignore the failure, but we'll add an icon in the UI to warn the user that their locale settings are wrong. However, because we might need to dispaly all sorts of characters, we want UTF-8 support, so try C.UTF-8. */ failed_locale_load = true; if ((old_locale = setlocale (LC_ALL, "C.UTF-8")) == NULL) { /* Error! */ ... } } /* Either we set the user's locale or we set C.UTF-8. */ ...
OK, how about the simpler fix: the default locale can be "C.UTF-8" instead of "C". Then the fall-back is not needed and the app still has the control over the wrongly set locale. But if setlocale() fails, the "C.UTF-8" locale will remain. Can this work? > if ((old_locale = setlocale (LC_ALL, "")) == NULL) > { > if ((old_locale = setlocale (LC_ALL, "C.UTF-8")) == NULL) This work-around may be reasonable for an apps that want to handle the wrongly set locale, but 99% of them do not want to deal with that. I maintain at least 2 apps that do setlocale(LC_ALL, "") without checking an error, and I would object to calling them "poorly written and not robust" as this method was very common and widely used - all until yesterday. Lets just find a good compromise, I am sure it exists somewhere. :)
Or if the default can't be changed that easily, how about adding an env var that specifies the default locale? In this case all apps that do not call setlocale(), will work with whatever this var contains, or "C" if it is not set. The same will be true if setlocale() fails, so the problem will be solved.
(In reply to Stas Sergeev from comment #23) > OK, how about the simpler fix: the default > locale can be "C.UTF-8" instead of "C". > Then the fall-back is not needed and the app still > has the control over the wrongly set locale. But if > setlocale() fails, the "C.UTF-8" locale will remain. > Can this work? That can work. We would need to discuss this upstream. Essentially expanding C to support UTF-8 transparently (since C is a subset of C.UTF-8). It would be hard to argue a case where you don't want UTF-8 support. > This work-around may be reasonable for an apps that > want to handle the wrongly set locale, but 99% of them > do not want to deal with that. I maintain at least 2 > apps that do setlocale(LC_ALL, "") without checking > an error, and I would object to calling them "poorly > written and not robust" as this method was very common > and widely used - all until yesterday. That's the exact definition of non-robust e.g. unable to adapt to allowed system changes (removal of locales). > Lets just find a good compromise, I am sure it exists somewhere. :) Agreed.
(In reply to Carlos O'Donell from comment #25) > (In reply to Stas Sergeev from comment #23) > > OK, how about the simpler fix: the default > > locale can be "C.UTF-8" instead of "C". > > Then the fall-back is not needed and the app still > > has the control over the wrongly set locale. But if > > setlocale() fails, the "C.UTF-8" locale will remain. > > Can this work? > > That can work. We would need to discuss this upstream. Essentially expanding > C to support UTF-8 transparently (since C is a subset of C.UTF-8). It would > be hard to argue a case where you don't want UTF-8 support. > We can fix this immediately in Fedora Rawhide and experiment a bit with it. https://bugzilla.redhat.com/show_bug.cgi?id=1313818
(In reply to Carlos O'Donell from comment #26) > We can fix this immediately in Fedora Rawhide and experiment a bit with it. > https://bugzilla.redhat.com/show_bug.cgi?id=1313818 Thanks! I didn't even went for other work-arounds and have the system still broken, in a hope for something like this to happen. :) I'd be glad to test that fix on f24. I am not sure though why you write this: --- * Change setlocale code to try C.UTF-8 loading before falling back to the builtin C/POSIX locales. --- What fix do you have in mind when saying that? The fix I mentioned, didn't involve setlocale code, because I proposed to have "C.UTF-8" by default, i.e. when setlocale() is not even called. So you probably went for something else after all?