Red Hat Bugzilla – Bug 303991
Transifex can't handle non-UTF or invalid UTF files
Last modified: 2009-07-03 16:07:17 EDT
Description of problem: Clicking 'preview' gets the following error. "500 Internal error" (more details, see attachment) Version-Release number of selected component (if applicable): http://publictest5.fedora.redhat.com/submit/module/smolt How reproducible: I did a couple of time, got same error message. Steps to Reproduce: 1. go to http://publictest5.fedora.redhat.com/submit/module/ and choose revisor 2a. At 'select local file', select ~/revisor/de.po 2b. At 'select local file', select ~/revisor/de.po 3. At 'select a destination file', select de.po from dropdown menu of 'Overwrite existing file' 4. At 'write a commit message', enter 'updated' (or something else) 5. Click 'preview' 6. Check the preview and click 'submit' 7. '500 Internal error' Actual results: At step 7, the submission to be successful Expected results: Successful submission
Created attachment 204551 [details] Traceback
I think this is because de.po on the server is not a valid UTF-8 file. Can you confirm this by testing it with something like `iconv -f utf-8 -t utf-8 de.po`? Nevertheless, transifex shouldn't choke with this and should commit the file without producing diff and patches.
A minor update: I tried looking into how we could fix this, but it requires more than just a small change in the code. We'll need to test thoroughly before putting in production. I'd suggest to open a bug report on smolt, so that one of the developers converts the file to UTF, or delete it completely, so that you can upload it for now.
The submitting through transifex works now. It's no longer a bug...
I think the submission was successful because the file was fixed to be UTF: http://git.fedoraproject.org/git/?p=smolt.git;a=commit;h=bc90a25813ed0c628ab3e4b53af8438833ec1ddf Still, Transifex shouldn't panic when the remote file isn't UTF-encoded. We should be able to choose "overwrite" on the submission form, which will completely replace the remote file. Re-opening with a changed subject to remember fixing it.
*** Bug 430998 has been marked as a duplicate of this bug. ***
So, the thing is, we currently only support valid UTF-encoded files. Not even ISO-8859-1. We should fix this. In bug #430998, I think the files on the server were ISO-8859-1 or something non-UTF, so Tx panicked. To fix this, we'll need to open up both files with the correct encoding, which is *always* the one mentioned in the PO-file's header. If the uploaded file is invalid (ie the file's encoding doesn't much with the header entry and fopen fails), raise an error. If the server file is invalid, offer the choice to the user to overwrite it.
Workaround: If the upload fails with "500 Internal error", the file can be uploaded successfully, if the user write the filename by hand in the field, like if he want upload a new file, and not by selecting the filename from the drop down box, and skipping the preview. I have tested it on firstboot it.po. My case was ISO8859 stored on server and UTF-8 to be uploaded.
(In reply to comment #8) Doesn't work for me, still shows error. New file and the one stored on VCS are UTF-8. Maybe it's not the same issue?
(In reply to comment #9) Yes, you are right. My solution was a bug, the file uploaded was appended to the existing one and should be fixed.
An update was pushed yesterday (v0.2) that allows submission of non-UTF files, and also overwriting of remote non-UTF files. It should go live sometime today, I'll send an email to the list as well. The only caveat is that preview isn't available for such submissions. To fix this as well, we'll have to support encoding detection: open the file as UTF, read up to the "Content-type" line, and open the file with the correct encoding.
So this bug is FIXED now?
Yes, this is fixed. Closing bug report, thanks for the feedback.
Mass change of component from Transifex to Website, since all our websites are handled by Transifex now.