Bug 470592

*** Bug 470086 has been marked as a duplicate of this bug. ***

Forcing utf8 in the kernel was the wrong idea - see bug 454013 for an example of the problems that caused. Further it made Fedora behave differently from all other Linux distributions when mounting vfat volumes, which is bad.

So dropping that kernel patch was probably the right decision, but it seems that we still need 'utf8' when mounting vfat for a lot of locales. So something in userspace needs to enable utf8 when needed. Unfortunately figuring out *when* it's needed is really hard. 

To maintain compatibility with older Fedora releases we could probably have HAL suggest use that flag by default - as with bug 144600. In a future release we might somehow decide that based on the current locale.

from bug 454013 
> ru_RU.cp1251 or ru_RU.koi8r
you should flag that bug as "non-BUG" becayse
the default locale in fedora for Russian locales is UTF-8 just look at this:

grep '^ru' /usr/share/system-config-language/locale-list
ru_RU.UTF-8 utf8 latarcyrheb-sun16 Russian  -  Русский
ru_UA.UTF-8 utf8 latarcyrheb-sun16 Russian (Ukraine)

so it's his own problem, he should edit his hal files to do his customizations

utf-8 is universal, forcing latin1 or whatever should not be done in an international project as fedora

so if you want to go upstream, then you should move the patch as a hal configuration

and those who want to use non-international encoding let them use some /etc/hal/fdi/policy/ file

if you insist on not making utf8 the default on hal level at least make a package having utf-8 fdi file for hal and make that package a default in the comps file for each *-support group

be aware of the following facts
grep 'utf8' /usr/share/system-config-language/locale-list | wc -l
136
wc -l /usr/share/system-config-language/locale-list
143 /usr/share/system-config-language/locale-list

which mean you are breaking more than 95% of the locales by not making utf8 tobe the default

my suggestion is doing the opposite having the default to be the default as a fdi hal file
and having 2 packages (8859-15 and 8859-2) for all the 7 non-utf-8 locales and make that one a default or optional in the comps file

grep -v 'utf8' /usr/share/system-config-language/locale-list
af_ZA 8859-15 lat0-sun16 Afrikaans (South Africa)
bs_BA 8859-2 lat2-sun16 Bosnian (Bosnia and Herzegowina)
br_FR 8859-15 lat0-sun16 Breton (France)
oc_FR 8859-15 lat0-sun16 Occitan (France)
tl_PH 8859-15 lat0-sun16 Tagalog (Philippines)
uz_UZ 8859-15 lat0-sun16 Uzbek (Uzbekistan)
wa_BE@euro 8859-15 lat0-sun16 Walloon (Belgium)

After much discussion we've decided to restore the kernel patch that enables 'utf8' by default on vfat, at least for F10 final:

* Mon Nov 10 2008 Dave Jones <davej> 2.6.27.5-93
- Make UTF-8 the default for FAT/VFAT filesystems again.

F11 will not include this patch, so this bug will need to be fixed in userspace (HAL etc.) in Rawhide.

Moving to F11Blocker.

This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

I guess it won't be in hal only but also to DeviceKit which is planed for F11

FWIW, KDE 4's Solid simply passes utf8 where available and iocharset=utf8 otherwise (unless that too is unavailable) to HAL when it mounts volumes. See StorageAccess::callHalVolumeMount() in http://websvn.kde.org/branches/KDE/4.2/kdelibs/solid/solid/backends/hal/halstorageaccess.cpp?revision=918779&view=markup for details. (And yes, it still uses HAL, not DeviceKit. Volunteers to write a Solid DeviceKit backend welcome.)

it was just a guess that DeviceKit-disks is involved,
it's important to keep DeviceKit aware of that as hal could be dropped after some long time.

if KDE relies on hal and gnome in device kit and xfce on whatever
this mean that we should fix it in both to keep the system consistent

I don't think there is any hal bug here. hal and DeviceKit-disks are just the mechanisms that are used for mounting. As Kevin says above, kde is using hal and is already passing the utf8 mount options. gnome is using DeviceKit-disks in F11, and the issue of making it pass utf8 mount options is tracked in bug 490347.

the solution should be centralized

I have mentioned some scenario of failure in ML if we let this go per application

let me mentioned one

someone is using kde, which uses HAL and it passes utf8 option
he created a file there

he logged off and log into gnome for example
gnome does not rely on hal so when he tries to open his files

a more tricky is a less famous WM or DE like lxde or Window Maker

he logged off and log into lxde for example

the more tricky part isn't this it's the opposite case


now we have several layers of complexity
and a very likely inconsistency and corruption 
as the default encoding assumed by the kernel can't be validated as opposed to utf-8
and writing in one encoding and reading in another will for sure case problems

what would we tell most of the users as a reply "how to set utf-8 as default encoding for all DEs and WMs just as it was in previous releases of Fedora ?" 

no matter what solution you take it should be centralized
preferably in the kernel or at least in system wide policy used by all the user land tools

...it should be centralized preferably in the kernel...

I hear you. But the kernel guys disagreed...

This is "fixed" in our main desktops (KDE/Gnome) so I'm going to consider it fixed for F11.  This bug entry is a bit of a disaster and probably shouldn't stay open so I'm closing it too.  Further discussion about this should happen on list.

> This is "fixed" in our main desktops (KDE/Gnome)
I guess regarding DeviceKit, yes it's solved and fixed
https://bugzilla.redhat.com/show_bug.cgi?id=489222

but I did not see any hal system-wide policy here
so it's not yet fixed
I call for having a small fdi policy file setting it for whatever desktop relying on HAL containing something like:
<device>
<match key="block.is_volume" bool="true">
<match key="volume.fstype" string="vfat">
<merge key="volume.policy.mount_option.utf8=true"
type="bool">true</merge>
</match>
</match>
</device>

having it fixed in both DeviceKit and hal then I can say we have it almost solved and proudly flag this bug as closed
please reconsider keeping it open until we having fdi file like the above one in hal

> I hear you. But the kernel guys disagreed...  

because they are stupid and close minded
Linux is supposed to be international

It's also solved and fixed in KDE, they pass the utf8 option to HAL as per https://bugs.kde.org/show_bug.cgi?id=161673 .