Bug 864280 - RFE: upload/download raw file types with the Maven plugin
Summary: RFE: upload/download raw file types with the Maven plugin
Alias: None
Product: Zanata
Classification: Retired
Component: Component-Maven
Version: development
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 2.1
Assignee: David Mason
QA Contact: Ding-Yi Chen
Depends On: 864277 886711
Blocks: 884347
TreeView+ depends on / blocked
Reported: 2012-10-09 03:34 UTC by David Mason
Modified: 2013-02-26 04:06 UTC (History)
2 users (show)

Fixed In Version: 2.1-SNAPSHOT (20121214-0032)
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 884347 (view as bug list)
Last Closed: 2013-02-26 04:06:28 UTC

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 864277 0 unspecified CLOSED [RFE] Allow upload and download of txt, dtd and open document format through REST interface 2021-02-22 00:41:40 UTC

Internal Links: 864277

Description David Mason 2012-10-09 03:34:53 UTC
Description of problem:
Implement upload and download from Maven client for documents of type DTD, plain text (.txt), and Open Document Format (odt, ods, odp, etc.).

Expected results:
New commands are available in the Maven plugin to push and pull projects made up of any of the above file types.

Additional info:
Commands will be named 'rawpush' and 'rawpull'

Comment 1 David Mason 2012-11-13 03:08:05 UTC
Implemented as project type 'raw' that works with standard 'push' and 'pull' commands.

raw project type assumes that all translations are in sibling directories of the source directory, named with locale code and containing the same directory structure as the source directory.

File types to include must be specified when invoking push or pull for a raw project. Specified file types are checked against supported types on the server.

For more details, see the detailed help available in the client.

Comment 2 Ding-Yi Chen 2012-11-19 06:36:58 UTC
Tested with 
client API version is 2.0.0, but server API version is 2.0.1-SNAPSHOT

1. mvn zanata:help -Dgoal=push -Ddetail  
   Should show the help on projectType=raw, and its releationship with fileType for supported file type.

2. mvn zanata:help -Dgoal=pull -Ddetail 
   Should show the help on projectType=raw, and its releationship with fileType for supported file type.
   Also need to show the help on fileType

3. Provide an option that toggle whether to apply fuzzy messages or not.

Comment 3 David Mason 2012-11-22 09:06:11 UTC
Updated in 2.0.2-SNAPSHOT

 - mentioned 'raw' in projectType parameter.
 - specified that fileType parameter applies to 'raw' project type
 - specified that maxChunkSize parameter applies to 'raw' project type

Note that fileType and maxChunkSize do not affect the pull goal at this time.

See: https://github.com/zanata/zanata-client/commit/ca4e59cd6984d9401750aa3c3cac5c139ddcc587

Comment 4 David Mason 2012-11-22 09:34:05 UTC
Updated in 2.0.2-SNAPSHOT

 - added parameter 'includeFuzzy' for pull command that applies to 'raw' project type

See https://github.com/zanata/zanata-client/commit/c5d58ca7d7cb6b4ed5bf09c1ada9fcea1b85884e

Comment 5 Ding-Yi Chen 2012-11-28 06:54:41 UTC
VERIFIED with Zanata version 2.0.3-SNAPSHOT (20121128-1507)
Client 2.0.2-SNAPSHOT

Comment 6 Ding-Yi Chen 2012-11-28 07:13:34 UTC
After some further test, it appeared that by default, it mistakenly push the translation files as source as well.

Step to reproduce:
1. push raw files
2. translate the raw file
3. pull raw files without specify -Dzanata.transDir
4. push raw files again

The translation files are pushed as source

The translation files are NOT pushed as source

Comment 7 Ding-Yi Chen 2012-11-28 07:21:33 UTC
For push

 mvn zanata:push -Dzanata.projectType=raw -Dzanata.fileType=txt 

and for pull
 mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt

Comment 8 Sean Flanigan 2012-11-28 23:49:46 UTC
Whether you push source files, target files, or both, is controlled by pushType, not by the transDir option.  The default value for pushType is 'source'.  

Were the console messages (about what was going to happen) misleading in some way?

Comment 9 Ding-Yi Chen 2012-11-29 04:38:47 UTC
The default pushType (i.e. without explicitly specified -Dzanata.pushType) is source. Even when I set the -Dzanata.pushType=source, the problem remain the same.

Here is what happended:
Suppose I have README.txt as source. and de, es as locales.

Step 1. mvn zanata:push -Dzanata.projectType=raw -Dzanata.fileType=txt

  README.txt is in server as source.

Step 2. mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt
  Following tranlation files are download to local.

Step 3. mvn zanata:push -Dzanata.projectType=raw -Dzanata.fileType=txt
  Maven client deems all following 3 files as source 
  Now you have 3 files to be translated, which is incorrect.

Pull them down reveals how the situation getting worse:
Step 4. mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt
Now you have 

Comment 10 David Mason 2012-12-04 04:00:10 UTC
With current implementation, the source directory must not contain translation directories for raw project type. It should work normally if you change structure to the following, with appropriate directories specified:


Comment 11 David Mason 2012-12-06 01:41:16 UTC
Updated help text in 2.0.2-SNAPSHOT to indicate that source and translations cannot use the same directory or nested directories.

See: https://github.com/zanata/zanata-client/commit/38e484d45787391d5ae2021387fb3d2a5a80efce

Comment 12 Ding-Yi Chen 2012-12-10 06:20:46 UTC
Tested with client API version: 2.0.0, server API version: 2.0.1
Zanata version 2.1-SNAPSHOT (20121206-1113)

With the pull command

mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt -Dzanata.createSkeletons

Error message shows as follows:
[ERROR] Failed to execute goal org.zanata:zanata-maven-plugin:2.0.2-SNAPSHOT:pull (default-cli) on project null: Zanata mojo exception: operation returned 500 (Internal Server Error): <html><head><title>JBoss Web/2.1.12.GA-patch-01 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 - java.lang.NullPointerException</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>java.lang.NullPointerException</u></p><p><b>description</b> <u>The server encountered an internal error (java.lang.NullPointerException) that prevented it from fulfilling this request.</u></p><HR size="1" noshade="noshade"><h3>JBoss Web/2.1.12.GA-patch-01</h3></body></html>, uri: http://zanata-master-auto.lab.eng.bne.redhat.com:8080/rest/projects/p/IBusChewing/iterations/i/master/r, annotations: [@javax.ws.rs.GET(), @javax.ws.rs.Path(value=/translation/{projectSlug}/{iterationSlug}/{locale}/{fileType})] -> [Help 1]

And the server log shows:
2012-12-10 16:11:30,956 WARN  [org.jboss.resteasy.core.SynchronousDispatcher] (http- Failed executing GET /file/translation/IBusChewing/master/cgg/baked
org.jboss.resteasy.spi.WriterException: java.lang.NullPointerException
        at org.jboss.resteasy.core.ServerResponse.writeTo(ServerResponse.java:233)
        at org.jboss.resteasy.core.SynchronousDispatcher.writeJaxrsResponse(SynchronousDispatcher.java:563)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:484)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119)


Comment 13 David Mason 2012-12-13 05:34:09 UTC
Fixed related bug that was causing error message on pull ( https://bugzilla.redhat.com/show_bug.cgi?id=886711 ). Please QA after the other bug is verified.

Comment 14 Ding-Yi Chen 2012-12-14 07:55:27 UTC
VERIFIED with Zanata version 2.1-SNAPSHOT (20121214-0032)

Note You need to log in before you can comment on or make changes to this bug.