Bug 144551 - wrong mount param means Nautilus can't copy to USB drive
wrong mount param means Nautilus can't copy to USB drive
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: hal (Show other bugs)
4
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Ray Strode [halfline]
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2005-01-07 23:35 EST by jarki
Modified: 2007-11-30 17:10 EST (History)
2 users (show)

See Also:
Fixed In Version: FC4 update
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-04-28 13:57:32 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
This one works for me in FC4 (2.66 KB, patch)
2006-01-18 20:36 EST, William Lovaton
no flags Details | Diff

  None (edit)
Description jarki 2005-01-07 23:35:13 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5)
Gecko/20041111 Firefox/1.0

Description of problem:
If you copy with Nautilus from hard drive to USB drive and filename has
characters like '���' then copy fails and you get error 'Invalid
argument'.
Also found that if filenames are almost equal, like 
img_0067.html 
IMG_0067.html (uppercase)
then file IMG_0067.html is not copied.

Looking issue more in bash shell:

root@b-194-31 test]# ll
total 72
-rw-r--r--  1 root root 10750 Jan  8 05:49 img_0067.html
-rwxr-xr-x  1 root root 10750 Jan  8 05:49 IMG_0067.html
-rwxr-xr-x  1 root root 36688 Jan  8 05:49 img_0067.jpg
[root@b-194-31 test]#
[root@b-194-31 test]# cp  -r /test/  /media/LACIE/
[root@b-194-31 test]# ll /media/LACIE/test/
total 96
-rwxr-xr-x  1 riti riti 10750 Jan  8 06:23 img_0067.html
-rwxr-xr-x  1 riti riti 36688 Jan  8 06:23 img_0067.jpg
[root@b-194-31 test]#
[root@b-194-31 test]# touch ���.txt
[root@b-194-31 test]# ll
total 76
-rw-r--r--  1 root root     0 Jan  8 06:24 ���.txt
-rw-r--r--  1 root root 10750 Jan  8 05:49 img_0067.html
-rwxr-xr-x  1 root root 10750 Jan  8 05:49 IMG_0067.html
-rwxr-xr-x  1 root root 36688 Jan  8 05:49 img_0067.jpg
[root@b-194-31 test]#
[root@b-194-31 test]# cp  -r /test/  /media/LACIE/
cp: cannot create regular file `/media/LACIE/test/���.txt': Invalid
argument
[root@b-194-31 test]# ll /media/LACIE/test/
total 96
-rwxr-xr-x  1 riti riti 10750 Jan  8 06:25 img_0067.html
-rwxr-xr-x  1 riti riti 36688 Jan  8 06:25 img_0067.jpg
[root@b-194-31 test]#





Version-Release number of selected component (if applicable):
Nautilus 2.8.1

How reproducible:
Always

Steps to Reproduce:
1. See above.

    

Actual Results:  See above.

Expected Results:  All files should copy. This means that you can't
use USB drive as backup media because you can't be sure that all files
are copied.

Additional info:
Comment 1 jarki 2005-01-08 03:05:10 EST
Seems that characters missing above. '���' should be 'åäö'
(scandinavian characters).
Comment 2 William Lovaton 2005-01-17 18:32:54 EST
Hey, I checked this problem on my system and it has something to do
with #145233 but it's not exactly the same.

I discovered that this happens depending on the locale.  A solution
for this would be adding the "utf8" argument in fstab for VFAT
filesystems but since you are talking about USB drives the entry is
generated dinamically by udev and thus you can't put it in fstab.  Can
you try to reproduce the problem with a VFAT partition in your hard
disk with and without the utf8 option in fstab?? check #145233 for
more info.  When you use utf8 it sholud work.

My test is as follow:
Using a non english locale, spanish from Colombia in my case
(es_CO.UTF-8 according to locale) Try to create a new folder in a VFAT
FS without the "utf8" option (HD or USB drive).  Nautilus will tell
you the same error message, in my case: "Error «Parámetros no válidos»
al crear una carpeta nueva." which means something like "Error
«Invalid arguments» when creating a new folder".  The reason for this
is that when you create a folder with nautilus it will use "untitle
folder" as a temporal name while the user writes the final name. This
string in spanish means "Carpeta sin título".  Do you see the accent
in "título"? (in the i).  Thus it will fail since the encoding used
during mount won't accept that string.

Doing the same test with the english locale will fail again and if you
use gnome-terminal, you will see:
[william@nalwalovaton Mis documentos]$ mkdir título
mkdir: cannot create directory `título': Invalid argument

So the problem is not in nautilus actually, it is deeper in the OS. 
Like in #145233 a sane solution would be adding "utf8" in VFAT
filesystem, even in the USB drives detected by udev.  Solving both:
displaying and creating files with special chars like tildes and accents.

Note that this could happen in other filesystems too, or at least it
dependes on the charset used to create that filesystem but this is a
bit of black magic to me.  I just expect things to work.  ;-)

So, please, fix this for FC4.
Comment 3 jarki 2005-11-20 07:10:30 EST
Bug still exists in FC4, 2.6.14-1.1637_FC4.
Comment 4 William Lovaton 2005-11-20 09:12:18 EST
Yes the bug is still there, jarki... please update the version from fc3 to fc4.

As I said in comment #2 this happens because the FAT filesystem is mounted by
udev without the 'utf8' option and that's why it's not accepting special chars
on file names.

A month ago I installed OpenSUSE and this bug doesn't happen there because it
uses the parameter in question.

Please Alex, Could you correct this problem? or at least tell us where to
instruct udev to put the utf8 option? I did that for FC3 in my office system but
I can't remember anymore.
Comment 5 Alexander Larsson 2005-11-21 05:14:42 EST
Nautilus isn't mounting the filesystem. I'm moving this to hal, as I think its
responsible for this.
Comment 6 jarki 2005-11-23 10:23:14 EST
Yes, upgrade fc3-->fc4 done already in June.
Instructions how to set utf8 for udev will help a lot.
Thanks.
Comment 7 William Lovaton 2006-01-06 19:17:48 EST
I have been try to discover how the hell I fixed this in FC3, I can't find the
place where I configured HAL to mount my usb key with the utf8 option.

J5, please check this bug report... what do you think about this? this is a
serious usability problem for us.  We are not able to create new directories
directly in the usb key using Nautilus (see comment #2).

I have not tested it yet but I guess rawhide still has this problem.  I'd really
like to see this fixed when FC5 gets released.
Comment 8 William Lovaton 2006-01-09 10:09:46 EST
The same applies to floppy disks with VFAT format... is there a way for hal to
know the file format of the floppy disk and change the mount options accordingly?

Sometimes I use floppy disks and files with accents (áéíóú) or tildes (ñÑ) just
show up as (?)... I guess this is a huge problem for non US locales.
Comment 9 jarki 2006-01-15 14:55:06 EST
Agree, this can be a huge problem for non US locales for example if you want
copy photos to USB drive or use USB drive as backup media.
Seems this still reproduces in FC5 test2.
Comment 10 William Lovaton 2006-01-15 19:09:44 EST
The other day my father was working on his laptop (without floppy drive) and
needed to copy some file from a floppy disk to his usb memory.  I was next to
him working on my desktop computer and he asked me to copy the files for him.

Pretty easy I thought, inserted the floppy disk (vfat) and double clicked the
floppy icon in the Computer window: mounted ok.  Then inserted his usb memory
(vfat) in one of the usb ports: auto mounted ok.

Then dragged and dropped the files from one to the another: failed!.  Some of
the files in the floppy disk contained accents and tildes as part of the name. 
When you look at nautilus for the floppy disk the problematic characters
appeared as '?' and when trying to copy them to the usb memory it said the file
system can't receive those characters (or something like that).

So, the problem is not specific to usb storage but with floppy disks too.  In
fact, with any kind of file system which use the vfat format.

That situation was kind of embarrassing.  I had to replace those characters with
"simple" ones.  Thank god those were like 8 files or something... Imagine
copying a huge tree.

As a side note: for my Windows partition in my hard drive I have the following
in fstab and it works fine:

/dev/hda1 /mnt/c vfat defaults,noauto,users,utf8 0 0
Comment 11 jarki 2006-01-16 15:11:34 EST
Adding utf8 to fstab and then mount manually does not help - the same problem
exists.
Comment 12 jarki 2006-01-17 14:19:46 EST
And no matter which locale values I have (en_US.UTF8, en_US.ISO-8859-1, etc) -->
the same error.
Comment 13 John (J5) Palmieri 2006-01-17 17:19:46 EST
As of tomorrows update this should work on FC5.  As for FC4 it needs a tweak of
the storage policy.  What would be helpfull would be to attach a couple of these
problem files to this report so I can test with them.  Thanks.
Comment 14 jarki 2006-01-18 13:39:34 EST
Thanks for update John:) 
I'll not attach any files since this is very easy to reproduce with empty file:
touch ö.txt
cp ö.txt /media/...


ltrace shows:

malloc(40)                                       = 0x8643ed8
calloc(131, 8)                                   = 0x8643f08
__xstat64(3, "/media/LACIE/test/", 0xbfa150e0)   = 0
memcpy(0xbfa15070, "\303\266.txt", 7)            = 0xbfa15070
malloc(25)                                       = 0x8644328
mempcpy(0x8644328, 0xbfa15bf1, 17, 0x80518f8, 17) = 0x8644339
mempcpy(0x864433a, 0xbfa15070, 6, 0x80518f8, 17) = 0x8644340
__xstat64(3, "\303\266.txt", 0xbfa14f4c)         = 0
__xstat64(3, "/media/LACIE/test/\303\266.txt", 0xbfa14eec) = -1
__errno_location()                               = 0xb7efc690
open64("\303\266.txt", 0, 00)                    = 3
__fxstat64(3, 3, 0xbfa14c84)                     = 0
open64("/media/LACIE/test/\303\266.txt", 65, 0100644) = -1
__errno_location()                               = 0xb7efc690
__errno_location()                               = 0xb7efc690
__ctype_get_mb_cur_max(0x611fd8, 0xb7efd280, 0x7657e3, 0xbfa14cc0, 0x612000) =
6dcgettext(0, 0x805554a, 5, 0x805739c, 0xb7efc690) = 0x805554a
dcgettext(0, 0x80558d0, 5, 0x805739c, 0x805554a) = 0x80558d0
mbrtowc(0xbfa14ab0, 0x864433a, 6, 0xbfa14ab4, 0x612000) = 2
iswprint(246, 0x864433a, 6, 0xbfa14ab4, 0x612000) = 1
mbsinit(0xbfa14ab4, 0x864433a, 6, 0xbfa14ab4, 0x612000) = 1
dcgettext(0, 0x805519d, 5, 0xbfa14f0c, 0xbfa14cf0) = 0x805519d
__errno_location()                               = 0xb7efc690
error(0, 22, 0x805519d, 0x80575c0, 0xbfa14cf0cp: cannot create regular file
`/media/LACIE/test/ö.txt': Invalid argument


Let me know if need some more info.

Thanks.


Comment 15 William Lovaton 2006-01-18 17:21:19 EST
One way to create an "invalid" file on a USB memory is on Windows (eg: ö.txt)
and then mount it on Linux.  You'll see the "invalid name" thing.

The other way is to mount it with utf8, create the invalid file and remount the
partition without utf8.  

But Jarki says that mounting with utf8 option doesn't work for him.  It does for
me, I just checked it before posting this using FC3, I'll have to test it at
home with FC4 but I guess it should be the same.
Comment 16 William Lovaton 2006-01-18 20:36:07 EST
Created attachment 123407 [details]
This one works for me in FC4

I edited the file /usr/share/hal/fdi/policy/10osvendor/10-storage-policy.fdi
and activated the utf8 option in several places.

I did the patch with the following command:
diff -U 6 10-storage-policy-orig.fdi 10-storage-policy.fdi

The one in the "floppy drives" section is neccesary to make my floppy work. 
May be there is a problem since the file system detection is "auto" and no
other file system uses the utf8 option.  But I guess that those file systems
just ignore that option at mount time so it might be ok.

Now my usb memory and my floppy drive accepts special chars, I could even
create the example file posted by Jarki (ö.txt)... I am happy again  :-)

Jarki, Could you test this patch?

Jhon, Is this the right solution?
Comment 17 John (J5) Palmieri 2006-01-19 10:46:45 EST
Yes that is correct.  Assigning to Ray to push the update.  Thanks Ray.
Comment 18 Ray Strode [halfline] 2006-01-19 13:56:50 EST
Hi guys, do the rpms on 

http://people.redhat.com/~rstrode/hal

fix your problems?
Comment 19 jarki 2006-01-19 14:02:00 EST
Quickly tested this patch and seems it works in FC4 for me too:)
Looked this same 10-storage-policy.fdi file just two days ago if it could give a
resolution, but this patch saved my time for testing.
Many thanks William!
Comment 20 John (J5) Palmieri 2006-01-19 14:05:39 EST
Actually the patch is not quite correct.  Ray fixed it up in the new release. 
The  right way to do it was to check on the volume.fstype='vfat' key and then
merge the utf8 option.
Comment 21 Ray Strode [halfline] 2006-01-19 14:07:52 EST
Thanks Jarki,

I'll push an update
Comment 22 jarki 2006-01-19 14:12:41 EST
Ray,
Tested on FC4.
These rpms seems also fix the issue:)
Thanks.
Comment 23 William Lovaton 2006-01-19 20:33:02 EST
(In reply to comment #20)
> Actually the patch is not quite correct.  Ray fixed it up in the new release. 
> The  right way to do it was to check on the volume.fstype='vfat' key and then
> merge the utf8 option.

Yeah, I supposed that but I didn't know how to make that check within the xml
configuration file.

Ray, the update does work for me for the USB case but not for the floppy disk,
they don't seem to get the utf8 option.  I guess it's because it's no exactly
VFAT but FAT12 or something like that (the typical windows floppy).  I hope you
can look into that.

I guess this is getting into RawHide too, right?.

Thanx for the new RPM any way.
Comment 24 Ray Strode [halfline] 2006-01-20 11:22:11 EST
It might work with USB floppy drives.

I think the problem is HAL doesn't know until mount time the filesystem type for
floppies. (I don't think standard PC floppy drives have media detection).

I think the fix is to apply the first chunk of your patch, which is to
unconditionally use utf8 with floppies.  Most floppies are floppies that are
formatted are formatted FAT12, so I don't think it will be a problem.  It might
break HFS formatted floppies though.

I think floppies are pretty edge case these days, and HFS floppies are even more
edge case, so I think your fix will be good enough for now.  If we get a lot of
bug reports from users with HFS formatted floppies we can reinvestigate. I'll
push another update with the first chunk of your patch.  

John say he thinks this is already fixed in rawhide by different means, so if it
ends up being broken in rawhide, just file a different report.
Comment 25 Fedora Update System 2006-01-20 18:00:59 EST
From User-Agent: XML-RPC

hal-0.5.2-2.fc4.1 has been pushed for FC4, which should resolve this issue.  If these problems are still present in this version, then please make note of it in this bug report.
Comment 26 jarki 2006-04-28 09:40:40 EDT
Seems this fix is not complete yet.
If we have a semicolon in file name, then the copy fails

touch 'test: file'

cp  'test: file'  /media/LACIE/
cp: cannot create regular file `/media/LACIE/test: file': Invalid argument

Comment 27 jarki 2006-04-28 10:32:24 EDT
1) And more - file names with lower case/upper case still cause problems, see

mkdir test 
touch test/testfile.txt
touch test/TestFile.txt
ll test
total 8
-rw-r--r--  1 root root 0 Apr 28 17:24 testfile.txt
-rw-r--r--  1 root root 0 Apr 28 17:24 TestFile.txt

rsync -av test/ /media/LACIE/
building file list ... done
./
rsync: chown "/media/LACIE/." failed: Operation not permitted (1)
TestFile.txt
testfile.txt
rsync: chown "/media/LACIE/.TestFile.txt.5Su2ua" failed: Operation not permitted (1)
rsync: chown "/media/LACIE/.testfile.txt.I4a9XE" failed: Operation not permitted (1)
rsync: chown "/media/LACIE/." failed: Operation not permitted (1)

sent 182 bytes  received 70 bytes  504.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files could not be transferred (code 23) at main.c(789)
[root@b-170-249 ~]#
[root@b-170-249 ~]# ll /media/LACIE/
total 0
-rw-------  1 riti riti 0 Apr 28 17:24 testfile.txt


2) The ownership can not be preserved, but not sure if it's bug or feature?


The NLS issue (original post in this bug) works.
Comment 28 John (J5) Palmieri 2006-04-28 10:41:57 EDT
This is a vfat file system correct?  I think you are just seeing the limitations
of a vfat file system (such as not being able to chown files).
Comment 29 jarki 2006-04-28 11:11:39 EDT
It's removable hard disk drive LACIE plugged to USB port and automounted by FC4
(/dev/sda1). Not sure what file system is used by HAL.
Should be quite easy to reproduce if you have a removable hard disk drive. 
The problem is annoying because can't take backups to removable hard disk - you
will be lost files!!!
The ownership issue is not so important because you can change the ownership
later back, after you restored files from removable hard disk.
Comment 30 John (J5) Palmieri 2006-04-28 11:37:19 EDT
The file system is important.  Not all removable drives are equal.  If it is a
vfat file system there are things you just can not do with it.  vfat sees
TestFile.txt and testfile.txt as the same files.  It can't understand certaint
characters like the colon ':' either.  Colons are special characters denoting
drives under dos/windows.  This is a fact and not a bug.  Vfat only supports a
subset of what ext file system support.

If you want to back up an ext3 filesystem to a vfat one your best bet is to not
place files which differ only in case in the same directory and don't user
special characters.  If you can live without a windows machine reading the files
then you can always format your drive to be ext2 or ext3.

Please attach the output of lshal to this bug report so I can confirm that your
drive has a vfat filesystem on it.  Thanks.

Comment 31 jarki 2006-04-28 13:48:41 EDT
It was indeed the vfat file system (as default) - so obviously explains those
thinks mentioned before.
Created an ext3 filesystem on it and know everything works just fine:)

John, Thanks for fast response!

Note You need to log in before you can comment on or make changes to this bug.