Bug 483622

I cloned the bug as it is also a problem in RHEL 5.

+++ This bug was initially created as a clone of Bug #454013 +++

Description of problem:
Because of kernel patch linux-2.6-defaults-fat-utf8.patch
vfat volumes is mounting ONLY in utf8.
When we have 8-bit locale (ru_RU.cp1251 or ru_RU.koi8r for example) we see wrong
symbols instead of russians. And we can't create any file/catalog with russians
symbol.

mount options codepage=866,iocharset=cp1251 don't work at all.

Version-Release number of selected component (if applicable):
kernels 2.6.23-25
file linux-2.6-defaults-fat-utf8.patch has date May 28 2007.
About since that time that bug was. Even in Fedora 8

How reproducible:
In system with russian 8-bit locale (cp1251, koi8-r) try mount vfat partition.
See to wrong russian symbol. Then try to create/copy file/directory with russian
symbol.

Steps to Reproduce:
Couse of a lot options to switch to Russian (codepage, fonts, keyboard) it's
easy to install Russian system from scratch.
1. During installation select Russian. By default it's "ru_RU.UTF-8". Install OS.
2. Togle to cp1251. Write to /etc/sysconfig/i18n string
LANG="ru_RU.cp1251" (or LANG="ru_RU.koi8-r")
3. In case of cp1251 make locale for glibc:
localedef -c -i ru_RU -f CP1251 ru_RU.CP1251
4. Reboot
5. Mount vfat partition
mount -t vfat -o codepage=866,iocharset=cp1251 /dev/sdX1 /mnt/fattest
(or without "-o codepage=866,iocharset=cp1251" - it's all one don't work right)
6. If vfat partition has Russian names, we'll see wrong characters and their
numbers are twice.
7. Togle to Russians (Alt-Ctrl by default, or Ctrl-Shift). Try to make directory
in fat with digits and russians letters. There will be directory with only
digits in name.
8. Create directory with rusians letters on not-fat partition. Then copy to fat.
Russians letters will not coping.
  
Actual results:
Wrong characters instead of russians.
It's not possible create or copy dirs/files with russians. 

Expected results:


Additional info:
After removing patch all work rights.

--- Additional comment from davej on 2008-07-03 16:57:12 EDT ---

hmm. I'm not sure how to fix this in a manner that pleases everyone.
We added that patch in response to bug 181963.  Without it, it displays for eg,
polish characters incorrectly.


--- Additional comment from andy on 2008-07-05 04:44:38 EDT ---

Kernel must give users possibility to mount FS with any codepages.
In case of "bug 181963" there is enough to mount with _documented_ mount option
"utf8". That is problem may be (and must be) resolved with user space program.

But with this patch users lose possibility to choose codepage at all. Only utf8.
Many device may read fat (mp3 players, cameras, PDA, (smart)phones, etc). I
don't know how in another languages, but if this devices can read russian names
they do it in non-utf8 (on fat). Therefore if we've Linux with kernel with this
patch we can't exchange files with this device.

Well, Andrew Zabolotny say that there is _undocumented_ option "utf8=0" in mount
utility, but:
1) this is NOT DOCUMENTED
2) mouinting may be done with other user space utiluties (gnome-mount, pmount,
hal, etc).
Are we sure that this utilities have adequate "undocumented" options?

In my mine there is some solutions:
1) keep all like in vanilla kernel. Who is needed utf will use utf8 options
2) use CONFIG_FAT_DEFAULT_IOCHARSET="utf8" options in fedora precompiled kernels
3) make utf8 option by default. BUT! if kernel receives "iocharset" option other
than "utf8" it discard utf8 flag.
Approximately so:

inode.c:
...
static int parse_options (...
...
case Opt_charset:
   if (strcmp(opts->iocharset, "utf8"))
       opts->utf8 = 0;
...

---------------
P.S. I find itfun but there is some strings from inode.c:
"if (!strcmp(opts->iocharset, "utf8")) {
   printk(KERN_ERR "FAT: utf8 is not a recommended IO charset"
     " for FAT filesystems, filesystem will be case sensitive!\n");
 }"

utf8 IS NOT A RECOMMENDED for vat!!!


--- Additional comment from andy on 2008-07-05 05:03:13 EDT ---

The same (case 3) in patch format:

--- linux-2.6.25.i386/fs/fat/inode.c.orig       2008-07-05 15:58:06.000000000 +0700
+++ linux-2.6.25.i386/fs/fat/inode.c    2008-07-05 15:58:50.000000000 +0700
@@ -1049,6 +1049,8 @@ static int parse_options(char *options,
                        if (!iocharset)
                                return -ENOMEM;
                        opts->iocharset = iocharset;
+                       if (strcmp(opts->iocharset, "utf8"))
+                              opts->utf8 = 0;
                        break;
                case Opt_shortname_lower:
                        opts->shortname = VFAT_SFN_DISPLAY_LOWER


--- Additional comment from andy on 2008-07-05 05:16:14 EDT ---

I'm sorry. There is not ability to correct messages, therefore I've to write new
one.
The  patch linux-2.6-defaults-fat-utf8.patch must be (in my opinion):


diff -up linux-2.6.25.i386/fs/fat/inode.c.orig linux-2.6.25.i386/fs/fat/inode.c
--- linux-2.6.25.i386/fs/fat/inode.c.orig       2008-07-05 15:58:50.000000000 +0700
+++ linux-2.6.25.i386/fs/fat/inode.c    2008-07-05 16:11:04.000000000 +0700
@@ -945,7 +945,8 @@ static int parse_options(char *options,
                opts->shortname = 0;
        opts->name_check = 'n';
        opts->quiet = opts->showexec = opts->sys_immutable = opts->dotsOK =  0;
-       opts->utf8 = opts->unicode_xlate = 0;
+       opts->utf8 = 1;
+       opts->unicode_xlate = 0;
        opts->numtail = 1;
        opts->usefree = opts->nocase = 0;
        *debug = 0;
@@ -1049,8 +1050,6 @@ static int parse_options(char *options,
                        if (!iocharset)
                                return -ENOMEM;
                        opts->iocharset = iocharset;
-                       if (strcmp(opts->iocharset, "utf8"))
-                              opts->utf8 = 0;
                        break;
                case Opt_shortname_lower:
                        opts->shortname = VFAT_SFN_DISPLAY_LOWER

--- Additional comment from andy on 2008-07-05 05:19:34 EDT ---

What is the day today. Error on error.
There must be 2 "+" instead of "-" near at end.

--- Additional comment from cebbert on 2008-10-23 23:08:26 EDT ---

We should drop the fat-default-utf8 patch. Upstream developers say it's wrong too.

--- Additional comment from cebbert on 2008-10-28 21:42:59 EDT ---

Default UTF-8 patch dropped in Fedora 10. We should consider dropping it in F9 when we go to kernel 2.6.27.

--- Additional comment from cebbert on 2008-10-31 13:26:56 EDT ---

UTF8 patch has been dropped in the F9 2.6.27 kernels.

--- Additional comment from benjavalero on 2008-11-03 18:00:03 EDT ---

With kernel 2.6.26.6-79.fc9, my MP3 players are automatically mounted and I have no problem with encodings, but booting with 2.6.27.4-68.fc10 these are mounted showing '?' instead of every non-ASCII character. With this last kernel, if I mount the players manually with 'utf8' option, then all works well as before (as with the FC9 kernel). Hope that helps.

--- Additional comment from hedayat on 2008-11-05 16:54:54 EDT ---

Maybe kernel or userspace programs should mount volumes considering current locale?! (in all .utf8 locales using utf8 mount option)

--- Additional comment from cebbert on 2008-11-07 16:36:11 EDT ---

Reverted the change for Fedora 9, but Fedora 10 will be using the upstream default .