Bug 454013

Summary: fat mounted only in utf8. This is wrong in 8-bit codepages
Product: [Fedora] Fedora Reporter: Andy Neverowsky <andy>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 9CC: benjavalero, hedayatv
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 483622 (view as bug list) Environment:
Last Closed: 2009-07-14 18:06:05 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Andy Neverowsky 2008-07-03 20:11:59 UTC
Description of problem:
Because of kernel patch linux-2.6-defaults-fat-utf8.patch
vfat volumes is mounting ONLY in utf8.
When we have 8-bit locale (ru_RU.cp1251 or ru_RU.koi8r for example) we see wrong
symbols instead of russians. And we can't create any file/catalog with russians
symbol.

mount options codepage=866,iocharset=cp1251 don't work at all.

Version-Release number of selected component (if applicable):
kernels 2.6.23-25
file linux-2.6-defaults-fat-utf8.patch has date May 28 2007.
About since that time that bug was. Even in Fedora 8

How reproducible:
In system with russian 8-bit locale (cp1251, koi8-r) try mount vfat partition.
See to wrong russian symbol. Then try to create/copy file/directory with russian
symbol.

Steps to Reproduce:
Couse of a lot options to switch to Russian (codepage, fonts, keyboard) it's
easy to install Russian system from scratch.
1. During installation select Russian. By default it's "ru_RU.UTF-8". Install OS.
2. Togle to cp1251. Write to /etc/sysconfig/i18n string
LANG="ru_RU.cp1251" (or LANG="ru_RU.koi8-r")
3. In case of cp1251 make locale for glibc:
localedef -c -i ru_RU -f CP1251 ru_RU.CP1251
4. Reboot
5. Mount vfat partition
mount -t vfat -o codepage=866,iocharset=cp1251 /dev/sdX1 /mnt/fattest
(or without "-o codepage=866,iocharset=cp1251" - it's all one don't work right)
6. If vfat partition has Russian names, we'll see wrong characters and their
numbers are twice.
7. Togle to Russians (Alt-Ctrl by default, or Ctrl-Shift). Try to make directory
in fat with digits and russians letters. There will be directory with only
digits in name.
8. Create directory with rusians letters on not-fat partition. Then copy to fat.
Russians letters will not coping.
  
Actual results:
Wrong characters instead of russians.
It's not possible create or copy dirs/files with russians. 

Expected results:


Additional info:
After removing patch all work rights.

Comment 1 Dave Jones 2008-07-03 20:57:12 UTC
hmm. I'm not sure how to fix this in a manner that pleases everyone.
We added that patch in response to bug 181963.  Without it, it displays for eg,
polish characters incorrectly.


Comment 2 Andy Neverowsky 2008-07-05 08:44:38 UTC
Kernel must give users possibility to mount FS with any codepages.
In case of "bug 181963" there is enough to mount with _documented_ mount option
"utf8". That is problem may be (and must be) resolved with user space program.

But with this patch users lose possibility to choose codepage at all. Only utf8.
Many device may read fat (mp3 players, cameras, PDA, (smart)phones, etc). I
don't know how in another languages, but if this devices can read russian names
they do it in non-utf8 (on fat). Therefore if we've Linux with kernel with this
patch we can't exchange files with this device.

Well, Andrew Zabolotny say that there is _undocumented_ option "utf8=0" in mount
utility, but:
1) this is NOT DOCUMENTED
2) mouinting may be done with other user space utiluties (gnome-mount, pmount,
hal, etc).
Are we sure that this utilities have adequate "undocumented" options?

In my mine there is some solutions:
1) keep all like in vanilla kernel. Who is needed utf will use utf8 options
2) use CONFIG_FAT_DEFAULT_IOCHARSET="utf8" options in fedora precompiled kernels
3) make utf8 option by default. BUT! if kernel receives "iocharset" option other
than "utf8" it discard utf8 flag.
Approximately so:

inode.c:
...
static int parse_options (...
...
case Opt_charset:
   if (strcmp(opts->iocharset, "utf8"))
       opts->utf8 = 0;
...

---------------
P.S. I find itfun but there is some strings from inode.c:
"if (!strcmp(opts->iocharset, "utf8")) {
   printk(KERN_ERR "FAT: utf8 is not a recommended IO charset"
     " for FAT filesystems, filesystem will be case sensitive!\n");
 }"

utf8 IS NOT A RECOMMENDED for vat!!!


Comment 3 Andy Neverowsky 2008-07-05 09:03:13 UTC
The same (case 3) in patch format:

--- linux-2.6.25.i386/fs/fat/inode.c.orig       2008-07-05 15:58:06.000000000 +0700
+++ linux-2.6.25.i386/fs/fat/inode.c    2008-07-05 15:58:50.000000000 +0700
@@ -1049,6 +1049,8 @@ static int parse_options(char *options,
                        if (!iocharset)
                                return -ENOMEM;
                        opts->iocharset = iocharset;
+                       if (strcmp(opts->iocharset, "utf8"))
+                              opts->utf8 = 0;
                        break;
                case Opt_shortname_lower:
                        opts->shortname = VFAT_SFN_DISPLAY_LOWER


Comment 4 Andy Neverowsky 2008-07-05 09:16:14 UTC
I'm sorry. There is not ability to correct messages, therefore I've to write new
one.
The  patch linux-2.6-defaults-fat-utf8.patch must be (in my opinion):


diff -up linux-2.6.25.i386/fs/fat/inode.c.orig linux-2.6.25.i386/fs/fat/inode.c
--- linux-2.6.25.i386/fs/fat/inode.c.orig       2008-07-05 15:58:50.000000000 +0700
+++ linux-2.6.25.i386/fs/fat/inode.c    2008-07-05 16:11:04.000000000 +0700
@@ -945,7 +945,8 @@ static int parse_options(char *options,
                opts->shortname = 0;
        opts->name_check = 'n';
        opts->quiet = opts->showexec = opts->sys_immutable = opts->dotsOK =  0;
-       opts->utf8 = opts->unicode_xlate = 0;
+       opts->utf8 = 1;
+       opts->unicode_xlate = 0;
        opts->numtail = 1;
        opts->usefree = opts->nocase = 0;
        *debug = 0;
@@ -1049,8 +1050,6 @@ static int parse_options(char *options,
                        if (!iocharset)
                                return -ENOMEM;
                        opts->iocharset = iocharset;
-                       if (strcmp(opts->iocharset, "utf8"))
-                              opts->utf8 = 0;
                        break;
                case Opt_shortname_lower:
                        opts->shortname = VFAT_SFN_DISPLAY_LOWER

Comment 5 Andy Neverowsky 2008-07-05 09:19:34 UTC
What is the day today. Error on error.
There must be 2 "+" instead of "-" near at end.

Comment 6 Chuck Ebbert 2008-10-24 03:08:26 UTC
We should drop the fat-default-utf8 patch. Upstream developers say it's wrong too.

Comment 7 Chuck Ebbert 2008-10-29 01:42:59 UTC
Default UTF-8 patch dropped in Fedora 10. We should consider dropping it in F9 when we go to kernel 2.6.27.

Comment 8 Chuck Ebbert 2008-10-31 17:26:56 UTC
UTF8 patch has been dropped in the F9 2.6.27 kernels.

Comment 9 Benjamín Valero Espinosa 2008-11-03 23:00:03 UTC
With kernel 2.6.26.6-79.fc9, my MP3 players are automatically mounted and I have no problem with encodings, but booting with 2.6.27.4-68.fc10 these are mounted showing '?' instead of every non-ASCII character. With this last kernel, if I mount the players manually with 'utf8' option, then all works well as before (as with the FC9 kernel). Hope that helps.

Comment 10 Hedayat Vatankhah 2008-11-05 21:54:54 UTC
Maybe kernel or userspace programs should mount volumes considering current locale?! (in all .utf8 locales using utf8 mount option)

Comment 11 Chuck Ebbert 2008-11-07 21:36:11 UTC
Reverted the change for Fedora 9, but Fedora 10 will be using the upstream default .

Comment 12 Bug Zapper 2009-06-10 01:54:19 UTC
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 13 Bug Zapper 2009-07-14 18:06:05 UTC
Fedora 9 changed to end-of-life (EOL) status on 2009-07-10. Fedora 9 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.