Bug 303991 - Transifex can't handle non-UTF or invalid UTF files
Transifex can't handle non-UTF or invalid UTF files
Status: CLOSED RAWHIDE
Product: Fedora Localization
Classification: Fedora
Component: Website (Show other bugs)
unspecified
All Linux
medium Severity high
: ---
: ---
Assigned To: Dimitris Glezos
Diego Búrigo Zacarão
: Reopened
: 430998 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-09-24 16:39 EDT by Fabian Affolter
Modified: 2009-07-03 16:07 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-03-11 16:54:51 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Traceback (3.19 KB, application/octet-stream)
2007-09-24 16:39 EDT, Fabian Affolter
no flags Details

  None (edit)
Description Fabian Affolter 2007-09-24 16:39:56 EDT
Description of problem:
Clicking 'preview' gets the following error.
"500 Internal error" (more details, see attachment)

Version-Release number of selected component (if applicable):
http://publictest5.fedora.redhat.com/submit/module/smolt

How reproducible:
I did a couple of time, got same error message.

Steps to Reproduce:
1. go to http://publictest5.fedora.redhat.com/submit/module/ and choose revisor
2a. At 'select local file', select ~/revisor/de.po
2b. At 'select local file', select ~/revisor/de.po
3. At 'select a destination file', select de.po from dropdown menu of
'Overwrite existing file'
4. At 'write a commit message', enter 'updated' (or something else)
5. Click 'preview'
6. Check the preview and click 'submit'
7. '500 Internal error'
  
Actual results:
At step 7, the submission to be successful

Expected results:
Successful submission
Comment 1 Fabian Affolter 2007-09-24 16:39:56 EDT
Created attachment 204551 [details]
Traceback
Comment 2 Dimitris Glezos 2007-09-24 17:27:39 EDT
I think this is because de.po on the server is not a valid UTF-8 file. Can you
confirm this by testing it with something like `iconv -f utf-8 -t utf-8 de.po`?

Nevertheless, transifex shouldn't choke with this and should commit the file
without producing diff and patches.
Comment 3 Dimitris Glezos 2008-01-25 03:24:13 EST
A minor update: I tried looking into how we could fix this, but it requires more
than just a small change in the code. We'll need to test thoroughly before
putting in production.

I'd suggest to open a bug report on smolt, so that one of the developers
converts the file to UTF, or delete it completely, so that you can upload it for
now.
Comment 4 Fabian Affolter 2008-01-29 04:06:10 EST
The submitting through transifex works now. It's no longer a bug...
Comment 5 Dimitris Glezos 2008-01-29 05:05:03 EST
I think the submission was successful because the file was fixed to be UTF:

http://git.fedoraproject.org/git/?p=smolt.git;a=commit;h=bc90a25813ed0c628ab3e4b53af8438833ec1ddf

Still, Transifex shouldn't panic when the remote file isn't UTF-encoded. We
should be able to choose "overwrite" on the submission form, which will
completely replace the remote file.

Re-opening with a changed subject to remember fixing it.
Comment 6 Dimitris Glezos 2008-02-06 18:19:05 EST
*** Bug 430998 has been marked as a duplicate of this bug. ***
Comment 7 Dimitris Glezos 2008-02-06 18:25:01 EST
So, the thing is, we currently only support valid UTF-encoded files. Not even
ISO-8859-1. We should fix this. In bug #430998, I think the files on the server
were ISO-8859-1 or something non-UTF, so Tx panicked.

To fix this, we'll need to open up both files with the correct encoding, which
is *always* the one mentioned in the PO-file's header. If the uploaded file is
invalid (ie the file's encoding doesn't much with the header entry and fopen
fails), raise an error. If the server file is invalid, offer the choice to the
user to overwrite it.
Comment 8 Francesco Tombolini 2008-02-07 13:18:54 EST
Workaround: If the upload fails with "500 Internal error", the file can be
uploaded successfully, if the user write the filename by hand in the field, like
if he want upload a new file, and not by selecting the filename from the drop
down box, and skipping the preview.
I have tested it on firstboot it.po.
My case was ISO8859 stored on server and UTF-8 to be uploaded.
Comment 9 Piotr Drąg 2008-02-07 13:36:10 EST
(In reply to comment #8)

Doesn't work for me, still shows error. New file and the one stored on VCS are
UTF-8. Maybe it's not the same issue?
Comment 10 Francesco Tombolini 2008-02-07 14:24:27 EST
(In reply to comment #9)
Yes, you are right.
My solution was a bug, the file uploaded was appended to the existing one and
should be fixed.
Comment 11 Dimitris Glezos 2008-02-11 07:34:32 EST
An update was pushed yesterday (v0.2) that allows submission of non-UTF files,
and also overwriting of remote non-UTF files. It should go live sometime today,
I'll send an email to the list as well.

The only caveat is that preview isn't available for such submissions. To fix
this as well, we'll have to support encoding detection: open the file as UTF,
read up to the "Content-type" line, and open the file with the correct encoding.
Comment 12 Piotr Drąg 2008-03-11 14:53:24 EDT
So this bug is FIXED now?
Comment 13 Dimitris Glezos 2008-03-11 16:54:51 EDT
Yes, this is fixed. Closing bug report, thanks for the feedback.
Comment 14 Piotr Drąg 2009-07-03 16:07:17 EDT
Mass change of component from Transifex to Website, since all our websites are handled by Transifex now.

Note You need to log in before you can comment on or make changes to this bug.