+++ This bug was initially created as a clone of Bug #470592 +++ Description of problem: fedora has dropped a patch from kernel that forces IOCHARSET=utf-8 to make people with non-UTF-8 locales happy https://www.redhat.com/archives/fedora-kernel-list/2008-October/msg00089.html but it makes people with UTF-8 locales sad when I put my USB stick and copy/paste files with non-ASCII charts or when I chdir there, I got errors if you really want to release fedora with such kerenl you should add a hal fdi configuration file that pass utf8 to mount just like what was done before that patch is introduced https://bugzilla.redhat.com/show_bug.cgi?id=144600 as kernel docs (Documentation/filesystems/vfat.txt) there is a safe utf8 by using utf8 option instead of iocharset=utf8 http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/vfat.txt;h=3a5ddc96901a665e3801d8ebf80c817e2efb5fc2;hb=HEAD Version-Release number of selected component (if applicable): fedora 10 kernel-2.6.27.4-68.fc10.i686 hal-0.5.12-9.20081027git.fc10.i386 How reproducible: always Steps to Reproduce: [alsadi@pc1 ALSADI]$ touch "سلام" touch: setting times of `سلام': No such file or directory [alsadi@pc1 ALSADI]$ touch peace [alsadi@pc1 ALSADI]$ mkdir "الإسلام" mkdir: cannot create directory `الإسلام': Invalid argument Expected results: no seeing the errors and having the files/dirs created with non-ASCII chars Additional info: having non-UTF-8 filesystem encoding could bug many poorly written GTK+ application http://library.gnome.org/devel/gtk/2.14/GtkFileChooser.html#gtkfilechooser-encodings --- Additional comment from wwoods on 2008-11-07 16:42:26 EDT --- *** Bug 470086 has been marked as a duplicate of this bug. *** --- Additional comment from wwoods on 2008-11-07 16:48:11 EDT --- Forcing utf8 in the kernel was the wrong idea - see bug 454013 for an example of the problems that caused. Further it made Fedora behave differently from all other Linux distributions when mounting vfat volumes, which is bad. So dropping that kernel patch was probably the right decision, but it seems that we still need 'utf8' when mounting vfat for a lot of locales. So something in userspace needs to enable utf8 when needed. Unfortunately figuring out *when* it's needed is really hard. To maintain compatibility with older Fedora releases we could probably have HAL suggest use that flag by default - as with bug 144600. In a future release we might somehow decide that based on the current locale. --- Additional comment from alsadi on 2008-11-10 12:33:00 EDT --- from bug 454013 > ru_RU.cp1251 or ru_RU.koi8r you should flag that bug as "non-BUG" becayse the default locale in fedora for Russian locales is UTF-8 just look at this: grep '^ru' /usr/share/system-config-language/locale-list ru_RU.UTF-8 utf8 latarcyrheb-sun16 Russian - Русский ru_UA.UTF-8 utf8 latarcyrheb-sun16 Russian (Ukraine) so it's his own problem, he should edit his hal files to do his customizations utf-8 is universal, forcing latin1 or whatever should not be done in an international project as fedora so if you want to go upstream, then you should move the patch as a hal configuration and those who want to use non-international encoding let them use some /etc/hal/fdi/policy/ file if you insist on not making utf8 the default on hal level at least make a package having utf-8 fdi file for hal and make that package a default in the comps file for each *-support group be aware of the following facts grep 'utf8' /usr/share/system-config-language/locale-list | wc -l 136 wc -l /usr/share/system-config-language/locale-list 143 /usr/share/system-config-language/locale-list which mean you are breaking more than 95% of the locales by not making utf8 tobe the default my suggestion is doing the opposite having the default to be the default as a fdi hal file and having 2 packages (8859-15 and 8859-2) for all the 7 non-utf-8 locales and make that one a default or optional in the comps file grep -v 'utf8' /usr/share/system-config-language/locale-list af_ZA 8859-15 lat0-sun16 Afrikaans (South Africa) bs_BA 8859-2 lat2-sun16 Bosnian (Bosnia and Herzegowina) br_FR 8859-15 lat0-sun16 Breton (France) oc_FR 8859-15 lat0-sun16 Occitan (France) tl_PH 8859-15 lat0-sun16 Tagalog (Philippines) uz_UZ 8859-15 lat0-sun16 Uzbek (Uzbekistan) wa_BE@euro 8859-15 lat0-sun16 Walloon (Belgium) --- Additional comment from wwoods on 2008-11-10 16:20:36 EDT --- After much discussion we've decided to restore the kernel patch that enables 'utf8' by default on vfat, at least for F10 final: * Mon Nov 10 2008 Dave Jones <davej> 2.6.27.5-93 - Make UTF-8 the default for FAT/VFAT filesystems again. F11 will not include this patch, so this bug will need to be fixed in userspace (HAL etc.) in Rawhide. Moving to F11Blocker. --- Additional comment from fedora-triage-list on 2008-11-26 00:00:44 EDT --- This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle. Changing version to '10'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping --- Additional comment from alsadi on 2008-11-26 17:35:32 EDT --- I guess it won't be in hal only but also to DeviceKit which is planed for F11
Created attachment 334479 [details] patch to that effect This should be a F11 blocker.
This has been fixed upstream, we'll get it with the next update.
good news we need similar one for hal, I guess it would be just an fdi policy file
Should be fixed in DeviceKit-disks-004 in rawhide.
I tried this and it doesn't work for me. The upstream fix adds "iocharset=utf8,codepage=437" which behaves differently than just "utf8". Don't know why but it does. Just adding "utf8" to "iocharset=utf8,codepage=437" also doesn't work. So if I patch out the newly added ones and add only "utf8" then everyting works as before. The option that was on by default in the kernel of previous Fedoras was specifically "utf8", where did the "iocharset=utf8,codepage=437" idea even come from ?
DeviceKit-003 breaks loading of my ipod with libgpod. Prior to this fix going in, everything worked correctly.
libgpod reports the filename as file:///media/IPODVIDEO2/iPod_Control/Music/F29/QCAR.mp3 but when I try and load it from the ipod the filesystem name is /media/IPODVIDEO2/iPod_Control/Music/f29/qcar.mp3 Rawhide mount reports /dev/sdb1 on /media/IPODVIDEO2 type vfat (rw,nosuid,nodev,uhelper=hal,shortname=lower,uid=500) F10 mount reports /dev/sdb1 on /media/IPODVIDEO2 type vfat (rw,nosuid,nodev,uhelper=hal,shortname=lower,uid=500) The device works properly on F10, but is broken in rawhide
(In reply to comment #7) > libgpod reports the filename as > file:///media/IPODVIDEO2/iPod_Control/Music/F29/QCAR.mp3 > > but when I try and load it from the ipod the filesystem name is > > /media/IPODVIDEO2/iPod_Control/Music/f29/qcar.mp3 > > Rawhide mount reports > /dev/sdb1 on /media/IPODVIDEO2 type vfat > (rw,nosuid,nodev,uhelper=hal,shortname=lower,uid=500) > > F10 mount reports > /dev/sdb1 on /media/IPODVIDEO2 type vfat > (rw,nosuid,nodev,uhelper=hal,shortname=lower,uid=500) > > The device works properly on F10, but is broken in rawhide Please paste the appropriate lines from from /proc/self/mountinfo since mount(8) just prints out /etc/mtab (which doesn't show all the options used). FWIW, it's not unlikely we want to change the default of "iocharset=utf8,codepage=437" for vfat to something else (to match F10 behavior and not break things like libgpod). However before that we need to know exactly what that "something else" should be.
I remounted the ipod without the iocharset,codepage options and it returned to working like before.
The issue is that fopen should work on FAT/VFAT no matter the case of the filename, as long as the actual name is right, so using the utf8 option is probably incorrect for vfat formatted devices, since vfat should not be case sensitive. If there is a way to make utf8 and case insensitivity work, I'm all for it.
The kernel docs specifically say: NOTE: "iocharset=utf8" is not recommended. If unsure, you should consider the following option [utf8] instead. (http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/vfat.txt;hb=HEAD#l55) I think the correct behavior here is to leave off iocharset & codepage and either: a) Just use 'utf8' by default everywhere (matches old behavior), or b) have something in the desktop environment (gnome-mount?) choose either 'utf8' or an appropriate iocharset/codepage, depending on the user's locale. ..but I don't know what component should make that choice. Probably easiest just to use 'utf8' everywhere by default, and document how you can override that, for the benefit those few people still using non-UTF8 data (e.g. bug 454013 and the 7 locales mentioned previously).
(In reply to comment #7) > libgpod reports the filename as > file:///media/IPODVIDEO2/iPod_Control/Music/F29/QCAR.mp3 > > but when I try and load it from the ipod the filesystem name is > > /media/IPODVIDEO2/iPod_Control/Music/f29/qcar.mp3 > > Rawhide mount reports > /dev/sdb1 on /media/IPODVIDEO2 type vfat > (rw,nosuid,nodev,uhelper=hal,shortname=lower,uid=500) > > F10 mount reports > /dev/sdb1 on /media/IPODVIDEO2 type vfat > (rw,nosuid,nodev,uhelper=hal,shortname=lower,uid=500) > > The device works properly on F10, but is broken in rawhide I pasted the incorrect rawhide mount information, it should have been /dev/sdb1 on /media/IPODVIDEO2 type vfat (rw,nosuid,nodev,uhelper=devkit,uid=500,gid=500,shortname=lower,dmask=0077,iocharset=utf8,codepage=437) As you can see it includes the iochars and codepage options that F10 does not
as for Comment #10 yes that is solved by passing utf8 not iocharset=utf8 passing iocharset=utf8 is not recommended please try passing utf8 not iocharset=utf8 and it's the default since F7, if you have problem with it it would appear before this
It seems that the proposal is that we should use utf8=true,shortname=lower to match F10 and earlier releases. I'd appreciate if someone can check if that makes libgpod work. Thanks.
Yes, that appears to work for me... /dev/sdb1 on /media type vfat (rw,nosuid,nodev,uid=500,gid=500,shortname=lower,dmask=0077,utf8=true)
Reopening until we're sure this is set straight. Or at least firmly crooked. For the record, 'shortname=lower' is the documented default, and 'utf8' is the documented version of the mount option, so an equivalent option string would be: 'rw,nosuid,nodev,uid=500,gid=500,dmask=0077,utf8'
Any ETA for a new DeviceKit-disks rpm in rawhide?
DeviceKit-disks-004-0.5.20090408git.fc11 will use utf8=1,shortname=lower by default. http://koji.fedoraproject.org/koji/taskinfo?taskID=1286129
Running with DeviceKit-disks-004-0.6.20090408git.fc11.x86_64 from koji and my ipod with libgpod seems to be working correctly again.