Bug 1006063

Summary: RFE: Enforcing validation for translation update
Product: [Retired] Zanata Reporter: Alex Eng <aeng>
Component: UsabilityAssignee: Alex Eng <aeng>
Status: CLOSED CURRENTRELEASE QA Contact: Ding-Yi Chen <dchen>
Severity: high Docs Contact:
Priority: unspecified    
Version: developmentCC: aeng, dchen, pahuang, sflaniga
Target Milestone: ---   
Target Release: 3.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 3.1-SNAPSHOT (20131008-1236) Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1012706 (view as bug list) Environment:
Last Closed: 2013-11-27 03:25:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1012706, 1013942, 1014423    

Description Alex Eng 2013-09-09 22:47:39 UTC 
* Hard validations to be turned on and off by maintainer
* This will show as an Error
* Can save as fuzzy if the string shows as having an error
* Run enforced validations on translation update (UI/REST/Editor)
* 3 states for validations: 
   ** off
   ** warning, translator can turn off the validation in their session, can save translation
   ** error, translator can't turn off the validation, can only save as fuzzy

If there's validation error(Validation with 'Error' state):
- In translation when user try to save as 'Translated' in editor, a warning popup will appear.
- When TM Merge and Search & Replace, translation update will be ignore and return warning messages.
- When upload translation via WebUI or REST, translation update will be ignore and return warning messages
Comment 1 Ding-Yi Chen 2013-09-25 08:46:09 UTC 
Test Basis: https://github.com/zanata/zanata-server/wiki/Translation-Validation
Comment 2 Ding-Yi Chen 2013-09-26 02:48:46 UTC 
Test with Zanata version 3.1-SNAPSHOT (20130926-0037)

I. Since the filter "Has Warning" now includes error as well, 
  please change the label to "Invalid" with tooltip that says "Translation that does not pass the validator, includes warning and error."

II. When you make an invalid error and click somewhere else, 
    click on "return to editor" should make the input focus back to the confronting translation. 

 Step to reproduce:
 1. Set the validation level to Error on "printf"
 2. Go to any document, click on translation entry for row 2.
 3. Trigger the validation error, 
    such as type excess "%d", DO NOT save now.
 4. Click on translation entry of row 2.
 5. In the "You're trying to save translation that contains validation error",
    Click "Return to editor"

 Expected:
   Input focus is back to row 1. 

 Actual:
   Input focus stays on row 2.

III. Upload using GWT Document list
 1. Set the validation level to Error on "printf"
 2. Set the validation level to warning on "tab"
 3. Prepare a translated document.
    a. 1st translation is valid
    b. 2nd translation trigger the error (such as excess %d)
    c. 3rd translation trigger the warning (such as excess tab \t)
    d. 4th translation and rest are empty
 4. Go to Projects->[ProjectName]->[VersionName]->[LocaleName]
    GWT Document lis should be shown.
 5. Upload with upload button

 Expected:
   After upload, message 2 should produced an error message.
   After upload, message 3 should produced an warning message.
   The stat on the document should be updated.
   Message 1 and 3 are saved as translated. 
   Message 3 are shown as invalid.
   The other messages are empty.

 Actual:
   ! After upload, message 2 DID NOT produce an error message.
   ! After upload, message 3 DID NOT produce an warning message.
   ! The stat on the document DID NOT update.
   Message 1 and 3 are saved as translated. 
   Message 3 are shown as invalid.
   The other messages are empty.

 (!) stands for where actual does not meet expectation.

IV. Upload using JSF Document list
 1. Set the validation level to Error on "printf"
 2. Set the validation level to warning on "tab"
 3. Prepare a translated document.
    a. 1st translation is valid
    b. 2nd translation trigger the error (such as excess %d)
    c. 3rd translation trigger the warning (such as excess tab \t)
    d. 4th translation and rest are empty
 4. Go to Projects->[ProjectName]->[VersionName]->Documents Icon in [LocaleName]
    A JSF Document list should be shown.
 5. Upload the translation with upload button

 Expected:
   After upload, message 2 should produced an error message.
   After upload, message 3 should produced an warning message.
   The stat on the document should be updated.
   Message 1 and 3 are saved as translated. 
   Message 3 are shown as invalid.
   The other messages are empty.

 Actual:
   After upload, message 2 produced an error message.
   ! After upload, message 3 DID NOT produce an warning message.
   The stat on the document was updated.
   Message 1 and 3 are saved as translated. 
   Message 3 are shown as invalid.
   The other messages are empty.
 (!) stands for where actual does not meet expectation.

Reassigned.
Comment 3 Ding-Yi Chen 2013-09-26 03:38:43 UTC 
In case that validation level is changing, validation should also be applied during download and client pull.

Data preparation:
 1. Set the validation level to Off on "printf"
 2. Set the validation level to warning on "tab"
 3. In translation editor, edit a translated document.
    a. 1st translation is valid
    b. 2nd translation trigger the error (such as excess %d)
    c. 3rd translation trigger the warning (such as excess tab \t)
    d. 4th translation and rest are empty
 4. Set the validation level to Error on "printf"


Steps to reproduce:
I. Download using GWT Document list
 1. Go to Projects->[ProjectName]->[VersionName]->[LocaleName]
    GWT Document list should be shown.
 2. Download by clicking .po

 Expected:
   After download, message 2 should produced an error message.
   After download, message 3 should produced an warning message.
   In .po file, message 1 and 3 should be saved. 
   In .po file, message 2 should NOT be saved. 
   The other messages are empty.

 Actual:
   ! After download, message 2 DID NOT produce an error message.
   ! After download, message 3 DID NOT produce an warning message.
   In .po file, message 1 and 3 were saved. 
   ! In .po file, message 2 was MISTAKENLY saved. 
   The other messages were empty.

 (!) stands for where actual does not meet expectation.

II. Download using JSF Document list
 1. Go to Projects->[ProjectName]->[VersionName]->Documents Icon in [LocaleName]
    A JSF Document list should be shown.
 2. Download by clicking .po

 Expected:
   After download, message 2 should produced an error message.
   After download, message 3 should produced an warning message.
   In .po file, message 1 and 3 should be saved. 
   In .po file, message 2 should NOT be saved. 
   The other messages are empty.

 Actual:
   ! After download, message 2 DID NOT produce an error message.
   ! After download, message 3 DID NOT produce an warning message.
   In .po file, message 1 and 3 were saved. 
   ! In .po file, message 2 was MISTAKENLY saved. 
   The other messages were empty.

III. Client pull
 1. mvn zanata:pull -Dzanata.locales=<locale>

 Expected:
   After download, message 2 should produced an error message.
   After download, message 3 should produced an warning message.
   In .po file, message 1 and 3 should be saved. 
   In .po file, message 2 should NOT be saved. 
   The other messages are empty.

 Actual:
   ! After download, message 2 DID NOT produce an error message.
   ! After download, message 3 DID NOT produce an warning message.
   In .po file, message 1 and 3 were saved. 
   ! In .po file, message 2 was MISTAKENLY saved. 
   The other messages were empty.
Comment 4 Alex Eng 2013-09-26 20:04:56 UTC 
The current requirements does not require "Warning" state check, the only validation checking done during translation updates are those with "Error" state set on the project/version.
Comment 5 Alex Eng 2013-09-26 20:09:25 UTC 
https://bugzilla.redhat.com/show_bug.cgi?id=1006063#c3, there's no requirements on running validation check when pulling/downloading translations.

I agree it would be useful to have another layer of checking, but if sufficient checking has been done during any translation update, our data should be valid enough during the pulling/downloading.

If needed, it should be a separate RFE.
Comment 6 Ding-Yi Chen 2013-09-26 23:41:49 UTC 
(In reply to Alex Eng from comment #5)
> https://bugzilla.redhat.com/show_bug.cgi?id=1006063#c3, there's no
> requirements on running validation check when pulling/downloading
> translations.
> 
> I agree it would be useful to have another layer of checking, but if
> sufficient checking has been done during any translation update, our data
> should be valid enough during the pulling/downloading.
> 
> If needed, it should be a separate RFE.

The use case is:

Default validation level is Warning.
However a project maintainer find out that the are too many errors in translations, so he decides to set the validation to Error to enforce validation.

Should the existing invalid translation be pulled? Definitely not.

Anyway, I will file a RFE for easier tracking.
Comment 7 Ding-Yi Chen 2013-09-30 00:21:49 UTC 
After talk with Alex, the scope of this bug is for enforcing the validation of new translation from Translation editor, upload, and client pull.

Validation on existing translation is addressed in Bug 1012706.
Comment 8 Ding-Yi Chen 2013-09-30 08:18:19 UTC 
Tested with Zanata version 3.1-SNAPSHOT (20130930-1511)

Bug discovered in comment 2:

I and II are fixed. However, III and IV are not fixed.


III. Upload using GWT Document list
 1. Set the validation level to Error on "printf"
 2. Set the validation level to warning on "tab"
 3. Prepare a translated document.
    a. 1st translation is valid
    b. 2nd translation trigger the error (such as excess %d)
    c. 3rd translation trigger the warning (such as excess tab \t)
    d. 4th translation and rest are empty
 4. Go to Projects->[ProjectName]->[VersionName]->[LocaleName]
    GWT Document lis should be shown.
 5. Upload with upload button

 Expected:
   After upload, message 2 should produced an error message.
   After upload, message 3 should produced an warning message.
   The stat on the document should be updated.
   Message 1 and 3 are saved as translated. 
   Message 3 are shown as invalid.
   The other messages are empty.

 Actual:
   ! After upload, message 2 DID NOT produce an error message.
   ! After upload, message 3 DID NOT produce an warning message.
   ! The stat on the document DID NOT update.
   Message 1 and 3 are saved as translated. 
   Message 3 are shown as invalid.
   The other messages are empty.

 (!) stands for where actual does not meet expectation.

IV. Upload using JSF Document list
 1. Set the validation level to Error on "printf"
 2. Set the validation level to warning on "tab"
 3. Prepare a translated document.
    a. 1st translation is valid
    b. 2nd translation trigger the error (such as excess %d)
    c. 3rd translation trigger the warning (such as excess tab \t)
    d. 4th translation and rest are empty
 4. Go to Projects->[ProjectName]->[VersionName]->Documents Icon in [LocaleName]
    A JSF Document list should be shown.
 5. Upload the translation with upload button

 Expected:
   After upload, message 2 should produced an error message.
   After upload, message 3 should produced an warning message.
   The stat on the document should be updated.
   Message 1 and 3 are saved as translated. 
   Message 3 are shown as invalid.
   The other messages are empty.

 Actual:
   After upload, message 2 produced an error message.
   ! After upload, message 3 DID NOT produce an warning message.
   The stat on the document was updated.
   Message 1 and 3 are saved as translated. 
   Message 3 are shown as invalid.
   The other messages are empty.
 (!) stands for where actual does not meet expectation.

There are also some other issues:

V. Push with maven client
 1. Set the validation level to Error on "printf"
 2. Set the validation level to warning on "tab"
 3. Prepare a translated document.
    a. 1st translation is valid
    b. 2nd translation trigger the error (such as excess %d)
    c. 3rd translation trigger the warning (such as excess tab \t)
    d. 4th translation and rest are empty
 4. Push with maven client:
    i.e. mvn zanata:push -Dzanata.pushType=trans 

 Expected:
   After push, message 3 should produced an warning message.

 Actual:
   After push, message 3 DID NOT produce an warning message.

VI. When entered an invalid translation move to other page or document, click "Return to editor" cleans the entry, translators will be very frustrated to see their hard work wiped out, especially after the invalid translations are very long.

 Step to reproduce:
 1. Go to a multi-page document.
 2. Copy source text and trigger an validation error by typing an excessive %d at page 1, row 1.
 3. Switch to page 2.
 4. Click "Return to Editor"

 Expected:
   The translation in page 1, row 1 is kept.

 Actual:
   The translation in page 1, row 1 is wiped out.
  

Reassigned.
Comment 9 Alex Eng 2013-09-30 22:11:15 UTC 
For III-2, IV-2, and V
As I mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1006063#c4, we only run validation which are set to "Error". 

I think its should be a seperate RFE for running warning validation as the defined scope of this RFE is just for "Error". 
  
Bare in mind running validation on translation an expensive process and will definitely slows down the process.
Comment 10 Ding-Yi Chen 2013-10-01 07:34:32 UTC 
The warning validation level is tracked by Bug 1013942.
Comment 11 Ding-Yi Chen 2013-10-02 03:45:14 UTC 
Tested with Zanata version 3.1-SNAPSHOT (20131002-1234)

VI. is mostly fixed, and the translation is not wiped out, same as expectation.

Yet there is one more thing: Click on documents (locale) will also trigger the 	
dialog "You're trying to save translation that contains validation error", but clicking on return to editor does not return to editor.

Step to reproduce:

 1. Go to a document, assume it is DOCUMENT1
 2. Copy source text and trigger an validation error by typing an excessive %d at page 1, row 1.
 3. Click on the "Documents(locale)" in breadcomb.
    Dialog "You're trying to save translation that contains validation error" should be shown.
 4. Click "Return to Editor"

 Expected:
   Input focus is in row1, page 1, DOCUMENT1

 Actual:
   Document list is shown.
Comment 12 Ding-Yi Chen 2013-10-02 04:14:27 UTC 
Finding in review:

1. Invalidation dialog:
   Title: "You're trying to save translation that contains validation error"
   Should be "You are trying to save an invalid translation"

2. Show the msgid and msgstr instead of REST id.
Comment 13 Sean Flanigan 2013-10-04 05:51:57 UTC 
See https://github.com/zanata/zanata-server/pull/211 for the recent changes.
Comment 14 Ding-Yi Chen 2013-10-04 08:08:30 UTC 
Tested with Zanata version 3.1-SNAPSHOT (20131004-1234):

When entered an invalid translation move to other document, click "Return to editor" cleans the entry, translators will be very frustrated to see their hard work wiped out, especially after the invalid translations are very long.

Step to reproduce:

 1. Go to a document, assume it is DOCUMENT1
 2. Copy source text and trigger an validation error by typing an excessive %d at page 1, row 1.
 3. Click on the "Documents(locale)" in breadcomb.
    Dialog "You're trying to save translation that contains validation error" should be shown.
 4. Click "Return to Editor"

 Expected:
   Translation in in row1, page 1, DOCUMENT1 is kept.

 Actual:
   Translation in in row1, page 1, DOCUMENT1 is wiped out.

Reassigned.
Comment 15 Sean Flanigan 2013-10-08 00:28:58 UTC 
For the benefit of bugzilla (and me), what's the story?
Comment 16 Alex Eng 2013-10-08 00:37:59 UTC 
Pull request, https://github.com/zanata/zanata-server/pull/214

Issue only happens in Firefox, where our event firing was in different order compare chrome causing the datacopyevent(copy of 'invalid' translation into editor) triggered before page got loaded.
Comment 17 Ding-Yi Chen 2013-10-08 03:02:14 UTC 
VERIFIED with Zanata version 3.1-SNAPSHOT (20131008-1236)

Browsers tested:
Firefox 17.0.9
Chromium 27.0.1453.93
Comment 18 Sean Flanigan 2013-11-27 03:16:25 UTC 
Closing VERIFIED bugs for Zanata versions <= 3.1.
Comment 19 Sean Flanigan 2013-11-27 03:17:21 UTC 
Closing VERIFIED bugs for Zanata versions <= 3.1.
Comment 20 Sean Flanigan 2013-11-27 03:19:48 UTC 
Closing VERIFIED bugs for Zanata versions <= 3.1.
Comment 21 Sean Flanigan 2013-11-27 03:25:45 UTC 
Closing VERIFIED bugs for Zanata versions <= 3.1.
Comment 22 Sean Flanigan 2013-11-27 03:34:37 UTC 
Closing VERIFIED bugs for Zanata versions <= 3.1.
Comment 23 Sean Flanigan 2013-11-27 03:36:45 UTC 
Closing VERIFIED bugs for Zanata versions <= 3.1.
Comment 24 Damian Jansen 2013-12-23 03:45:35 UTC 
*** Bug 981690 has been marked as a duplicate of this bug. ***