Bug 493759 - Amarok cannot load files from a directory containing characters in another encoding
Amarok cannot load files from a directory containing characters in another en...
Product: Fedora
Classification: Fedora
Component: phonon (Show other bugs)
i686 Linux
low Severity medium
: ---
: ---
Assigned To: Rex Dieter
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2009-04-02 20:25 EDT by João Felipe Santos
Modified: 2009-09-07 22:27 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-09-07 22:27:33 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Folder with an "unvalid" name (10.00 KB, application/x-gzip)
2009-04-02 20:25 EDT, João Felipe Santos
no flags Details

External Trackers
Tracker ID Priority Status Summary Last Updated
KDE Software Compilation 172242 None None None Never

  None (edit)
Description João Felipe Santos 2009-04-02 20:25:29 EDT
Created attachment 337956 [details]
Folder with an "unvalid" name

Description of problem:

I mounted a DVD with music files and found that Amarok cannot load files from a directory whose name contains special characters. For example, I have a directory called Lúnasa on the DVD, and if I try to load the music files from it, nothing happens. Also if I try to load a file with special characters directly, nothing happens. When running Amarok from console, the message "QDir::exists: Empty or null file name" is printed at the moment I try to load a file. If I try to use the file browser from Amarok to navigate into the folder, the same message is shown and I cannot navigate it. I noticed that Dolphin has the same problem.

The character "ú" (\372 or any other "special one" from this DVD) was encoded in a non-UTF encoding. Checking from console, I can see these characters show up as "?". I have a unicode font installed, because I can see cyrillic and japanese characters at my console.

Version-Release number of selected component (if applicable): Amarok 2.0.2 (amarok-2.0.2-3.fc10.i386)

How reproducible: Always.

Steps to Reproduce:
1. Extract the gzipped folder (it is just a folder, without files inside it)
2. Copy a music file inside it
3. Try to load the folder (dragging and dropping it to the playlist) or to navigate inside it
Actual results: Nothing happens. It is impossible to browse the folder or load any music files.

Expected results: It is possible to browse and load files from the folder.

Additional info: this trouble is also reproducible under Dolphin.
Comment 1 Rex Dieter 2009-04-02 20:33:02 EDT
rpm -q qt kdelibs kdebase-runtime

Also, can you clarify what "this trouble is also reproducible under Dolphin" means exactly?
Comment 2 João Felipe Santos 2009-04-02 21:03:53 EDT
sure, here they are:


I cannot browse to this folder using Dolphin too. This error message appears on the footer of the window: "The file or folder L<garbage>nasa does not exist". By <garbage>, I mean three special characters ('ï', inverted ? and the "one-half" symbol).
Comment 3 Rex Dieter 2009-05-04 11:18:22 EDT
Is this still reproducible using the latest qt/kde updates (qt-4.5.0/kde-4.2.2) ?
Comment 4 João Felipe Santos 2009-05-04 11:28:08 EDT
Yes, I updated my system and it still has the same behaviour. The versions of the packages now are:

Comment 5 Rex Dieter 2009-05-04 11:37:50 EDT
Dang, I understood that qt-4.5.0 was supposed to help address issues like this.

There's quite a few upstream reports, lemme dig up a few to provide some cross-references.
Comment 6 Rex Dieter 2009-05-04 11:54:22 EDT
looks like a phonon issue,

I'll see if our phonon pkg requires patching.
Comment 7 Fedora Update System 2009-05-04 12:31:39 EDT
phonon-4.3.1-4.fc11 has been submitted as an update for Fedora 11.
Comment 8 Fedora Update System 2009-05-04 12:34:47 EDT
phonon-4.3.1-4.fc10 has been submitted as an update for Fedora 10.
Comment 9 Rex Dieter 2009-05-04 12:36:06 EDT
If able, please test this build to see if it helps for you:
Comment 10 João Felipe Santos 2009-05-04 19:21:32 EDT

I upgraded to the new Phonon packages but the problem is not solved yet. The installed versions are:


I've read the Phonon bug report (http://bugs.kde.org/172242) and it is very similar to my trouble.
Comment 11 Kevin Kofler 2009-05-05 09:21:32 EDT
Why do we support non-UTF-8 characters at all?
Comment 12 Kevin Kofler 2009-05-05 09:34:30 EDT
IMHO, the real problem here is that userspace is seeing non-UTF-8 filenames in the first place. Normally, the UDF file system should get mounted with iocharset=utf8. At least KDE is expected to do that as per: http://websvn.kde.org/branches/KDE/4.2/kdelibs/solid/solid/backends/hal/halstorageaccess.cpp?revision=918779&view=markup . So either iocharset doesn't work as advertised or whatever mounting method you use doesn't use it.

A few questions:
* What is your operating system locale (i.e. the output of: echo $LANG)? (It should be an UTF-8 locale.)
* Are you using your desktop environment's automatic mounting or some manual fstab entry? What desktop environment are you using? (I'm trying to figure out how your disk is getting mounted.)
Comment 13 João Felipe Santos 2009-05-05 20:07:32 EDT
You are right. In fact, last night I tested mounting the same DVD with a fuse module called convmvfs, that allows one to mount a filesystem under another encoding setting. I converted the filesystem encoding from iso-8859-2 to utf8 and everything (Amarok and Dolphin) worked as expected with this command:

convmvfs -o srcdir=/media/mydvd -o icharset=iso-8859-2 -o ocharset=utf-8 /home/myuser/

I'm running KDE, with the default automount system for the disk. Looks like it is mounting the disk with the wrong encoding settings. Now I don't know if the problem is in HAL or Solid. What information can I give you to help finding it?
Comment 14 Kevin Kofler 2009-05-05 20:33:36 EDT
Well, it could be that the disk is using filenames encoded in ISO-8859-2 (by the way, are you sure it's -2 and not -1?) rather than UTF-16 and the kernel doesn't want to convert those (it seems iocharset is only used for UTF-16).

As for why the Phonon update doesn't help: it fixes using characters which match your locale, but are not UTF-8 (something which doesn't exist if you're using a UTF-8 locale as is the default in Fedora), but it doesn't fix using characters which do not match your locale, which is the problem you're seeing (the characters are ISO-8859-2 (or -1 or -15 or whatever), the locale is UTF-8). I'm afraid I guess convmvfs is really the only reliable solution for that problem. :-( Many applications use Unicode (UTF-16 or UTF-8) internally to represent filenames, not raw 8-bit characters, so they cannot reliably represent characters which are not valid in the system's locale. Unfortunately, it looks like this is the case for Phonon, so there's no good fix here.
Comment 15 João Felipe Santos 2009-05-05 21:27:05 EDT
Yes, the characters are ISO-8859-1 (Western European), sorry for the mistake.

Ok, I'll use the workaround and forget about this issue for now. Thank you for all the help!
Comment 16 Fedora Update System 2009-05-06 19:28:28 EDT
phonon-4.3.1-4.fc10 has been pushed to the Fedora 10 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update phonon'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-4271
Comment 17 João Felipe Santos 2009-05-06 19:39:17 EDT
As I reported in comments #10 and #13, this specific bug was not corrected by this update. I installed the RPMs before they were pushed to testing (got them directly from Koji).
Comment 18 Fedora Update System 2009-05-09 00:02:45 EDT
phonon-4.3.1-4.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 19 Fedora Update System 2009-05-09 00:11:25 EDT
phonon-4.3.1-4.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 20 Steven M. Parrish 2009-06-28 19:50:42 EDT
Per comment #17 this is still an issue.  Changing back to ASSIGNED

Steven M. Parrish - KDE Triage Master
                  - PackageKit Triager
Fedora Bugzappers volunteer triage team
Comment 21 Steven M. Parrish 2009-09-07 22:17:35 EDT
Is this an issue with the 4.3.1 release?

Steven M. Parrish - KDE Triage Master
                  - PackageKit Triager
Fedora Bugzappers volunteer triage team
Comment 22 Kevin Kofler 2009-09-07 22:27:33 EDT
This is not a bug at all.

There was a bug which already got fixed by the Phonon update (phonon-4.3.1-4): Phonon didn't support non-ASCII characters in non-UTF-8 locales properly. But the reporter's problem is that his file system uses a character set which doesn't match his locale, which is not supported and cannot be sanely supported in most applications (in any application which internally uses Unicode for all strings like Qt does – accessing those files can only work if the file name is kept as a string of bytes all the time).

The best solution there is to use convmv or convmvfs to fix the encoding of the file names to match the locale (which is what the reporter ended up doing). (Convmv renames the files outright, but that requires a writable volume. Convmvfs layers above the file system to do the encoding on the fly.)

Note You need to log in before you can comment on or make changes to this bug.