Bug 708536

Executive summary: Most of the operating system and apps don't really work well if you're using a non-UTF8 locale.  This change makes that explicit; we don't support non-UTF8 locales.

(IRC log follows; context is a patch to document the encoding return from g_get_current_directory())

<owen> walters: you'll confuse people
 walters: typedefs for char for different types of string content are historically evil (xmlChar * from libxml, etc.)
<owen> readdir is going to return char *'s, nomatter what char * typedefs you put in glib
<walters> the intent is more as a marker for "hey this isn't utf8"
 xmlChar is explicitly unsigned which is a totally different story
<owen> walters: still think it's a bad idea
<walters> fair enough, i was just getting opinions
 in reality i'm sure most apps just directly pass "gfilename"s to pango and blow up in non-utf8 locales
 at least initially, the important ones may get debugged and fixed
 but the gfilename functions are only part of the problem; the byte arrays are another
 if i could go back in time, these would be "guint8*"
<owen> non-utf-8 locales aren't really an issue, corrupt filenames that aren't properly encoded is more of an issue
<walters> owen: the locale is consumed by G_BROKEN_FILENAMES
 which you may remember on Fedora at least is set
<owen> non-utf8-locales can be considered not to exist
 they dont' exist on any operating system we support
--> hadess_ (~hadess.bethere.co.uk) has joined #gnome-os
<owen> but even in a utf-8 locale, the operating systme doens't enforce filenames passing g_utf8_validate()
<walters> well the glib docs clearly describe this situation, i guess we could say we don't care anymore
<owen> walters: not surprisngly, things change in 10 years
<walters> though i think at least frcrozat still runs fr_FR.ISO_8859_15 or something
 ok fair enough, i won't care then
<owen> that doens't chang ethe fact that filenames are null-terminated byte arrays
<walters> ah...
 i am confused
 you're saying that we have to treat gfilename as not being utf8 ?
 where "gfilename" is my hypothetical patch for "places in glib that return char * but are really null-byte-arrays"
<owen> walters: well, preferably you work with a GFile
<owen> but if you are not, then if you want to put a filename into a label, you need to ues g_filename_display_name()
<owen> walters: on the other hand, if you have an entry to enter a filename, I don't really care much if you do g_filename_from_utf8() before using it as a filename, because that affects only hypthetical non-utf8-locales which don't exist
<owen> (any more)
--> caillon (~caillon.ca.comcast.net) has joined #gnome-os
<-- hadess has quit (Ping timeout: 600 seconds)
<walters> ok
 though i would say if you think we shouldn't care about non-utf8-locales, we should remove the G_BROKEN_FILENAMES setting from the Fedora glib2 package
<owen> talk to mclasen
<walters> so i need to fix the annotations for both g_filename_display_name() and g_file_get_basename() then
* owen named the variable, but didn't put it in the RHL/Fedora packaging (pretty sure, hey, I could be wrong 10 years)
<walters> 8e288e5e        (Matthias Clasen        2005-12-12 05:58:51 +0000       6)export G_BROKEN_FILENAMES=1
 thoguh actually
 commit 4b67e11e362c9b0cd701acd161af4c718d47f217
 Author: cvsdist <cvsdist>
 Date:   Thu Sep 9 05:17:09 2004 +0000
     auto-import changelog data from glib2-2.0.6-2.src.rpm
     Tue Aug 13 2002 Havoc Pennington <hp>
     - install glib2.sh and glib2.csh to set G_BROKEN_FILENAMES
     - blow away unpackaged files in install
<owen> G_BROKEN_FILENAMES is a no-op if your locale is utf-8 in any case
<walters> yep i totally understand that =)
 though i am curious whether you have any data for your statement; my understanding was that non-UTF8-locales were very prominent in Asia still, and somewhat in Russia, somewhat less still in parts of europe
<wwoods> they didn't get the "utf8 or gtfo" memo, eh
--> bratsche (~bratsche.res.rr.com) has joined #gnome-os
<pbor> for what is worth we still get complaint about non utf8 encodings for text files
 mostly from russia/east europe
<walters> owen: hm...with this change basically if you have a filename that's say Shift-JIS encoded, and you were logging into jp_JP.SHIFT-JIS, we will replace most of the filename with U+FFFD instead of converting it
<owen> walters: well, if we want to support that, then we should keep G_BROKEN_FILENAMES
<owen> (or set the envvar for filename encodings)
<walters> or, change g_filename_display_name to always try the locale encoding
<owen> But, is just propagating a broken situation a decade after it should have been fixed
 I mean, just forget about it, make glib g_abort() if the locale isn't utf-8
 It's not supportable
<walters> data point: ubuntu 10.04 does not set G_BROKEN_FILENAMES