Bug 884347

Summary: Users are not warned when using an unsupported directory layout for 'file' project type
Product: [Retired] Zanata Reporter: Ding-Yi Chen <dchen>
Component: UsabilityAssignee: David Mason <damason>
Status: CLOSED UPSTREAM QA Contact: Zanata-QA Mailling List <zanata-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: developmentCC: damason, dchen, mkim, sflaniga, zanata-bugs
Target Milestone: ---Keywords: Reopened, screened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 864280 Environment:
Last Closed: 2015-07-29 02:51:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 864277, 864280    
Bug Blocks:    

Description Ding-Yi Chen 2012-12-06 01:56:08 UTC
This bug addressed that the translation files that were pulled down from zanata should be excluded by subsequential Zanata push by default.


+++ This bug was initially created as a clone of Bug #864280 +++

Description of problem:
Implement upload and download from Maven client for documents of type DTD, plain text (.txt), and Open Document Format (odt, ods, odp, etc.).

Expected results:
New commands are available in the Maven plugin to push and pull projects made up of any of the above file types.

Additional info:
Commands will be named 'rawpush' and 'rawpull'

--- Additional comment from David Mason on 2012-11-13 13:08:05 EST ---

Implemented as project type 'raw' that works with standard 'push' and 'pull' commands.

raw project type assumes that all translations are in sibling directories of the source directory, named with locale code and containing the same directory structure as the source directory.

File types to include must be specified when invoking push or pull for a raw project. Specified file types are checked against supported types on the server.

For more details, see the detailed help available in the client.

--- Additional comment from Ding-Yi Chen on 2012-11-28 17:13:34 EST ---

After some further test, it appeared that by default, it mistakenly push the translation files as source as well.

Step to reproduce:
1. push raw files
2. translate the raw file
3. pull raw files without specify -Dzanata.transDir
4. push raw files again

Actual:
The translation files are pushed as source

Expected:
The translation files are NOT pushed as source

--- Additional comment from Ding-Yi Chen on 2012-11-28 17:21:33 EST ---

For push

 mvn zanata:push -Dzanata.projectType=raw -Dzanata.fileType=txt 

and for pull
 
 mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt

--- Additional comment from Sean Flanigan on 2012-11-29 09:49:46 EST ---

Whether you push source files, target files, or both, is controlled by pushType, not by the transDir option.  The default value for pushType is 'source'.  

Were the console messages (about what was going to happen) misleading in some way?

--- Additional comment from Ding-Yi Chen on 2012-11-29 14:38:47 EST ---

The default pushType (i.e. without explicitly specified -Dzanata.pushType) is source. Even when I set the -Dzanata.pushType=source, the problem remain the same.

Here is what happended:
Suppose I have README.txt as source. and de, es as locales.

Step 1. mvn zanata:push -Dzanata.projectType=raw -Dzanata.fileType=txt

  README.txt is in server as source.

Step 2. mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt
  Following tranlation files are download to local.
  de/README.txt
  es/README.txt

Step 3. mvn zanata:push -Dzanata.projectType=raw -Dzanata.fileType=txt
  Maven client deems all following 3 files as source 
  README.txt
  de/README.txt
  es/README.txt
  
  Now you have 3 files to be translated, which is incorrect.

Pull them down reveals how the situation getting worse:
Step 4. mvn zanata:pull -Dzanata.projectType=raw -Dzanata.fileType=txt
Now you have 
  README.txt
  de/README.txt
  es/README.txt
  de/de/README.txt
  de/es/README.txt
  es/de/README.txt
  es/es/README.txt

--- Additional comment from David Mason on 2012-12-04 14:00:10 EST ---

With current implementation, the source directory must not contain translation directories for raw project type. It should work normally if you change structure to the following, with appropriate directories specified:

  source/README.txt
  de/README.txt
  es/README.txt

Comment 1 Michelle Kim 2015-03-10 05:54:00 UTC
Is our new mapping rule going to fix this issue? Can you please confirm if this still a issue? If not, can we close the bug? thanks

Comment 2 Ding-Yi Chen 2015-03-23 00:47:10 UTC
(In reply to Michelle Kim from comment #1)
> Is our new mapping rule going to fix this issue? Can you please confirm if
> this still a issue? If not, can we close the bug? thanks

No,

Tested with zanata-server-3.6.0, zanata-maven-plugin-3.4.2
Project freemind:

It has
./FM_Key_Mappings_Quick_Guide_hr.odt
./FM_Key_Mappings_Quick_Guide_fr.odt
./FM_Key_Mappings_Quick_Guide_et.odt
./FM_Key_Mappings_Quick_Guide_ru.odt
./FM_Key_Mappings_Quick_Guide_it.odt
./FM_Key_Mappings_Quick_Guide.odt

The mapping rules.
<rules>
    <rule pattern="FM_Key_Mappings_Quick_Guide.odt">{filename}_{locale_with_underscore}.odt</rule>
</rules>

Push with:
mvn zanata:push -Dzanata.pushType=both -Dzanata.disableSSLCert

FM_Key_Mappings_Quick_Guide_fr.odt is now push as both
 fr translation of FM_Key_Mappings_Quick_Guide.odt, as well as 
 source document by its own.

Current zanata can avoid this by providing src-dir, trans-dir, includes and excludes, but it needs quite a lot of trial and error.

Most of the project expected the document translation (except podir and publican) like

{filename}-{locale_with_underscore}.

so it is better to make it as default.






Actually zanata can 
The mapping rule can fix the issue, a

Comment 3 David Mason 2015-03-27 09:42:34 UTC
This is not a bug, it is a documentation-reading issue. Read this attentively:

> raw project type assumes that all translations are in sibling directories of the source directory, named with locale code and containing the same directory structure as the source directory.

Sibling directories, not subdirectories. The source directory must not contain the translation directories.

In short, this layout is not valid for raw project type because srcdir contains the translation dirs:

 - {project}/document.txt
 - {project}/de/document.txt
 - {project}/ja/document.txt

the correct structure has a separate source directory that does not contain translation directories:

 - {project}/source/document.txt
 - {project}/de/document.txt
 - {project}/ja/document.txt


I am closing this as not a bug (it is the implemented behaviour and it is correctly documented). Support for different directory structure should be an RFE.

Comment 4 Ding-Yi Chen 2015-03-29 23:36:24 UTC
It is usability bug then.

Except for publican (podir) or man page, most projects I have seen are like

Source: document.ext
Translation: document_<locale>.ext

Freemind and LibreOffice are two exapmles.

Please try to push those to Zanata, yes, I have tried.
It is achievable, but time consuming.

Comment 5 Sean Flanigan 2015-03-30 01:08:25 UTC
I agree, it is a usability bug.  If raw projects can't support overlapping source and target directories (they is supported for .properties, although it isn't pretty), we should at least detect this unsupported configuration and warn the user.

Comment 6 David Mason 2015-03-31 01:46:30 UTC
(In reply to Sean Flanigan from comment #5)
> I agree, it is a usability bug.  If raw projects can't support overlapping
> source and target directories (they is supported for .properties, although
> it isn't pretty), we should at least detect this unsupported configuration
> and warn the user.

That makes sense, we should definitely implement that ASAP.

I have created a separate RFE for changing how the directory structure works, with just the relevant information. People should not have to read all the cruft here to understand that change: https://bugzilla.redhat.com/show_bug.cgi?id=1207475

Comment 7 Zanata Migrator 2015-07-29 02:51:07 UTC
Migrated; check JIRA for bug status: http://zanata.atlassian.net/browse/ZNTA-231