Bug 864280 - RFE: upload/download raw file types with the Maven plugin
RFE: upload/download raw file types with the Maven plugin
Status: CLOSED CURRENTRELEASE
Product: Zanata
Classification: Community
Component: Component-Maven (Show other bugs)
development
Unspecified Unspecified
unspecified Severity medium
: ---
: 2.1
Assigned To: David Mason
Ding-Yi Chen
:
Depends On: 864277 886711
Blocks: 884347
  Show dependency treegraph
 
Reported: 2012-10-08 23:34 EDT by David Mason
Modified: 2013-02-25 23:06 EST (History)
2 users (show)

See Also:
Fixed In Version: 2.1-SNAPSHOT (20121214-0032)
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 884347 (view as bug list)
Environment:
Last Closed: 2013-02-25 23:06:28 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description David Mason 2012-10-08 23:34:53 EDT
Description of problem:
Implement upload and download from Maven client for documents of type DTD, plain text (.txt), and Open Document Format (odt, ods, odp, etc.).

Expected results:
New commands are available in the Maven plugin to push and pull projects made up of any of the above file types.

Additional info:
Commands will be named 'rawpush' and 'rawpull'
Comment 1 David Mason 2012-11-12 22:08:05 EST
Implemented as project type 'raw' that works with standard 'push' and 'pull' commands.

raw project type assumes that all translations are in sibling directories of the source directory, named with locale code and containing the same directory structure as the source directory.

File types to include must be specified when invoking push or pull for a raw project. Specified file types are checked against supported types on the server.

For more details, see the detailed help available in the client.
Comment 2 Ding-Yi Chen 2012-11-19 01:36:58 EST
Tested with 
client API version is 2.0.0, but server API version is 2.0.1-SNAPSHOT

Comments:
1. mvn zanata:help -Dgoal=push -Ddetail  
   Should show the help on projectType=raw, and its releationship with fileType for supported file type.

2. mvn zanata:help -Dgoal=pull -Ddetail 
   Should show the help on projectType=raw, and its releationship with fileType for supported file type.
   Also need to show the help on fileType

3. Provide an option that toggle whether to apply fuzzy messages or not.
Comment 3 David Mason 2012-11-22 04:06:11 EST
Updated in 2.0.2-SNAPSHOT

 - mentioned 'raw' in projectType parameter.
 - specified that fileType parameter applies to 'raw' project type
 - specified that maxChunkSize parameter applies to 'raw' project type

Note that fileType and maxChunkSize do not affect the pull goal at this time.


See: https://github.com/zanata/zanata-client/commit/ca4e59cd6984d9401750aa3c3cac5c139ddcc587
Comment 4 David Mason 2012-11-22 04:34:05 EST
Updated in 2.0.2-SNAPSHOT

 - added parameter 'includeFuzzy' for pull command that applies to 'raw' project type

See https://github.com/zanata/zanata-client/commit/c5d58ca7d7cb6b4ed5bf09c1ada9fcea1b85884e
Comment 5 Ding-Yi Chen 2012-11-28 01:54:41 EST
VERIFIED with Zanata version 2.0.3-SNAPSHOT (20121128-1507)
Client 2.0.2-SNAPSHOT
Comment 6 Ding-Yi Chen 2012-11-28 02:13:34 EST
After some further test, it appeared that by default, it mistakenly push the translation files as source as well.

Step to reproduce:
1. push raw files
2. translate the raw file
3. pull raw files without specify -Dzanata.transDir
4. push raw files again

Actual:
The translation files are pushed as source

Expected:
The translation files are NOT pushed as source
Comment 7 Ding-Yi Chen 2012-11-28 02:21:33 EST
For push

 mvn zanata:push -Dzanata.projectType=raw -Dzanata.fileType=txt 

and for pull
 
 mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt
Comment 8 Sean Flanigan 2012-11-28 18:49:46 EST
Whether you push source files, target files, or both, is controlled by pushType, not by the transDir option.  The default value for pushType is 'source'.  

Were the console messages (about what was going to happen) misleading in some way?
Comment 9 Ding-Yi Chen 2012-11-28 23:38:47 EST
The default pushType (i.e. without explicitly specified -Dzanata.pushType) is source. Even when I set the -Dzanata.pushType=source, the problem remain the same.

Here is what happended:
Suppose I have README.txt as source. and de, es as locales.

Step 1. mvn zanata:push -Dzanata.projectType=raw -Dzanata.fileType=txt

  README.txt is in server as source.

Step 2. mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt
  Following tranlation files are download to local.
  de/README.txt
  es/README.txt

Step 3. mvn zanata:push -Dzanata.projectType=raw -Dzanata.fileType=txt
  Maven client deems all following 3 files as source 
  README.txt
  de/README.txt
  es/README.txt
  
  Now you have 3 files to be translated, which is incorrect.

Pull them down reveals how the situation getting worse:
Step 4. mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt
Now you have 
  README.txt
  de/README.txt
  es/README.txt
  de/de/README.txt
  de/es/README.txt
  es/de/README.txt
  es/es/README.txt
Comment 10 David Mason 2012-12-03 23:00:10 EST
With current implementation, the source directory must not contain translation directories for raw project type. It should work normally if you change structure to the following, with appropriate directories specified:

  source/README.txt
  de/README.txt
  es/README.txt
Comment 11 David Mason 2012-12-05 20:41:16 EST
Updated help text in 2.0.2-SNAPSHOT to indicate that source and translations cannot use the same directory or nested directories.

See: https://github.com/zanata/zanata-client/commit/38e484d45787391d5ae2021387fb3d2a5a80efce
Comment 12 Ding-Yi Chen 2012-12-10 01:20:46 EST
Tested with client API version: 2.0.0, server API version: 2.0.1
Zanata version 2.1-SNAPSHOT (20121206-1113)

With the pull command

mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt -Dzanata.createSkeletons

Error message shows as follows:
============================================================
[ERROR] Failed to execute goal org.zanata:zanata-maven-plugin:2.0.2-SNAPSHOT:pull (default-cli) on project null: Zanata mojo exception: operation returned 500 (Internal Server Error): <html><head><title>JBoss Web/2.1.12.GA-patch-01 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 - java.lang.NullPointerException</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>java.lang.NullPointerException</u></p><p><b>description</b> <u>The server encountered an internal error (java.lang.NullPointerException) that prevented it from fulfilling this request.</u></p><HR size="1" noshade="noshade"><h3>JBoss Web/2.1.12.GA-patch-01</h3></body></html>, uri: http://zanata-master-auto.lab.eng.bne.redhat.com:8080/rest/projects/p/IBusChewing/iterations/i/master/r, annotations: [@javax.ws.rs.GET(), @javax.ws.rs.Path(value=/translation/{projectSlug}/{iterationSlug}/{locale}/{fileType})] -> [Help 1]
============================================================

And the server log shows:
============================================================
2012-12-10 16:11:30,956 WARN  [org.jboss.resteasy.core.SynchronousDispatcher] (http-0.0.0.0-8080-1) Failed executing GET /file/translation/IBusChewing/master/cgg/baked
org.jboss.resteasy.spi.WriterException: java.lang.NullPointerException
        at org.jboss.resteasy.core.ServerResponse.writeTo(ServerResponse.java:233)
        at org.jboss.resteasy.core.SynchronousDispatcher.writeJaxrsResponse(SynchronousDispatcher.java:563)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:484)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119)
============================================================

Reassigned.
Comment 13 David Mason 2012-12-13 00:34:09 EST
Fixed related bug that was causing error message on pull ( https://bugzilla.redhat.com/show_bug.cgi?id=886711 ). Please QA after the other bug is verified.
Comment 14 Ding-Yi Chen 2012-12-14 02:55:27 EST
VERIFIED with Zanata version 2.1-SNAPSHOT (20121214-0032)

Note You need to log in before you can comment on or make changes to this bug.