Red Hat Bugzilla – Bug 303991
Transifex can't handle non-UTF or invalid UTF files
Last modified: 2009-07-03 16:07:17 EDT
Description of problem:
Clicking 'preview' gets the following error.
"500 Internal error" (more details, see attachment)
Version-Release number of selected component (if applicable):
I did a couple of time, got same error message.
Steps to Reproduce:
1. go to http://publictest5.fedora.redhat.com/submit/module/ and choose revisor
2a. At 'select local file', select ~/revisor/de.po
2b. At 'select local file', select ~/revisor/de.po
3. At 'select a destination file', select de.po from dropdown menu of
'Overwrite existing file'
4. At 'write a commit message', enter 'updated' (or something else)
5. Click 'preview'
6. Check the preview and click 'submit'
7. '500 Internal error'
At step 7, the submission to be successful
Created attachment 204551 [details]
I think this is because de.po on the server is not a valid UTF-8 file. Can you
confirm this by testing it with something like `iconv -f utf-8 -t utf-8 de.po`?
Nevertheless, transifex shouldn't choke with this and should commit the file
without producing diff and patches.
A minor update: I tried looking into how we could fix this, but it requires more
than just a small change in the code. We'll need to test thoroughly before
putting in production.
I'd suggest to open a bug report on smolt, so that one of the developers
converts the file to UTF, or delete it completely, so that you can upload it for
The submitting through transifex works now. It's no longer a bug...
I think the submission was successful because the file was fixed to be UTF:
Still, Transifex shouldn't panic when the remote file isn't UTF-encoded. We
should be able to choose "overwrite" on the submission form, which will
completely replace the remote file.
Re-opening with a changed subject to remember fixing it.
*** Bug 430998 has been marked as a duplicate of this bug. ***
So, the thing is, we currently only support valid UTF-encoded files. Not even
ISO-8859-1. We should fix this. In bug #430998, I think the files on the server
were ISO-8859-1 or something non-UTF, so Tx panicked.
To fix this, we'll need to open up both files with the correct encoding, which
is *always* the one mentioned in the PO-file's header. If the uploaded file is
invalid (ie the file's encoding doesn't much with the header entry and fopen
fails), raise an error. If the server file is invalid, offer the choice to the
user to overwrite it.
Workaround: If the upload fails with "500 Internal error", the file can be
uploaded successfully, if the user write the filename by hand in the field, like
if he want upload a new file, and not by selecting the filename from the drop
down box, and skipping the preview.
I have tested it on firstboot it.po.
My case was ISO8859 stored on server and UTF-8 to be uploaded.
(In reply to comment #8)
Doesn't work for me, still shows error. New file and the one stored on VCS are
UTF-8. Maybe it's not the same issue?
(In reply to comment #9)
Yes, you are right.
My solution was a bug, the file uploaded was appended to the existing one and
should be fixed.
An update was pushed yesterday (v0.2) that allows submission of non-UTF files,
and also overwriting of remote non-UTF files. It should go live sometime today,
I'll send an email to the list as well.
The only caveat is that preview isn't available for such submissions. To fix
this as well, we'll have to support encoding detection: open the file as UTF,
read up to the "Content-type" line, and open the file with the correct encoding.
So this bug is FIXED now?
Yes, this is fixed. Closing bug report, thanks for the feedback.
Mass change of component from Transifex to Website, since all our websites are handled by Transifex now.