Bug 1092193 - RFE: Individual Translator Statistics
RFE: Individual Translator Statistics
Status: CLOSED CURRENTRELEASE
Product: Zanata
Classification: Community
Component: Component-API, Component-UI, Usability (Show other bugs)
3.3
Unspecified Unspecified
unspecified Severity medium
: ---
: 3.6
Assigned To: Alex Eng
Ding-Yi Chen
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2014-04-28 20:22 EDT by Carlos Munoz
Modified: 2015-03-19 03:10 EDT (History)
5 users (show)

See Also:
Fixed In Version: 3.6.0-SNAPSHOT (git-jenkins-zanata-server-verify-integration-master-3978-SUCCESS-1-g9e86c7d)
Doc Type: Bug Fix
Doc Text:
Story Points: 5
Clone Of:
Environment:
Last Closed: 2015-03-19 03:10:32 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Carlos Munoz 2014-04-28 20:22:33 EDT
This RFE is about adding visible (and queryable through the API) statistics for a translator.

Some stats that might be worth while looking at:
- A single translator's 'work' in a given period of time, in terms of number of words translated, number of projects/versions affected.
- Same as above, but comparing more than one translator for a single project/version.

This is still up for discussion and we need some feedback from stakeholders.
Comment 1 Carlos Munoz 2014-05-25 20:56:38 EDT
After discussing the requirements with the dev team, this is the proposal we are putting forward for implementation of a Rest API for this. The UI portion is of a lower priority:

The API should allow to query a single translator's "work" in a given period of time (from/to) on a project.

Stats to get:

 - # of translated words (entered latest 'translated' translation)
 - # of reviewed words  (most recent marked approved or rejected)
   - We could split this number up and give each (approved/rejected) separately


## Notes:
 - This API will return stats based on word numbers (not strings)
 - Queries will be allowed to filter by a certain project version, and within a certain timeframe.
 - The results will show counts for translated, reviewed, and fuzzy words.
 - The API will return results according to the current state of the project.
 - If time between translation and review is long, there might be inconsistencies.
 - Provide an auxiliary endpoint to query a list of all contributors to a project version.
Comment 2 Carlos Munoz 2014-09-18 19:40:22 EDT
A few notes from a discussion with Daisy (OpenStack i18n):

- Every contribution should count, even fuzzy revisions of Strings; this means that the whole history of a String should be taken into account when calculating a translator's contribution stats.
- Stats should be shown for the different states: Reviewed, Translated, fuzzy, approved, and reviewed.
- It would be ideal to count contributions *ONLY* for changes to translations other than your own. This means that if a translator overwrites another revision of his/her own authorship, this contribution should not be counted.
- Word-level stats should suffice, although a future option to add String level stats should be considered.
- The OpenStack team needs an API to query translator's information for a language team (this must include email), for external processing. (This should be taken a separate task / user story)
- Stats in this comment refer to word-counts.
Comment 3 Luke Brooker 2014-09-18 20:55:50 EDT
Thanks for the update Carlos this is good to know.

Having some sort of "reviewed" statistic would be interesting, how would we define this?

Approved *and* rejected/issue?

The review interface is something I think we can do a lot better than other tools.
Comment 4 Alex Eng 2014-10-14 18:51:25 EDT
Pull request:
zanata-api: https://github.com/zanata/zanata-api/pull/20
Comment 5 Alex Eng 2014-10-15 00:20:05 EDT
Pull request:
zanata-server: https://github.com/zanata/zanata-server/pull/607
Comment 6 Ding-Yi Chen 2014-10-27 23:11:15 EDT
Tested with Zanata 3.6.0-SNAPSHOT (git-jenkins-zanata-server-github-pull-requests-2237)
1. Query on archived version should return 404, but it returned 200.
2. In most of the cases, after changing to Approved and Reject, the statistics did not seem updated.
Comment 7 Ding-Yi Chen 2014-10-29 19:39:09 EDT
Alex has clarified that the stats is based on history. And the stats for each user and each string will count exactly once.
For example, if Irene save Grace's FUZZY as TRANSLATED, If Irene has not "TRANSLATED" the string, Irene will get TRANSLATED, but Grace's stat won't change, regardless whether Irene changed Grace's string or not.
Comment 8 Alex Eng 2014-10-29 22:00:44 EDT
Just to clarify, current implementation is to return contribution statistic of translation from a person in a project-version.

 
Expected behavior:

1. When a translator change a translation and save as different state as what it was, statistic should return (total-occurrence  + 1) * word-count.  

2. When a translator change a translation and save as same state as what it was, statistic should return total-occurrence * word-count (no addition).

3. When a reviewer Person B approve or reject a translation of Person A, statistic count goes to Person A. (only translation statistic)

4. When a reviewer Person B starts to translate an empty translation, and save as approved/rejected, statistic count goes to Person B. (Person B translated a string).

5. When a reviewer Person B change a not empty translation done by Person A, and save as approved/rejected, statistic count goes to Person B. (We are assuming reviewer should not make any changes on translation while in review process) 


There's contradiction in 4. and 5. on the assumption of reviewer should not make any changes on translation while in review process. The reason 5. is implemented based on the assumption was because we have 'rejected' state in translation. Reviewer should reject rather than making changes to translation during the process. 

I think we can improve this by having "review" and "translate" mode in our editor and restrict save options.
Comment 9 Alex Eng 2014-10-29 22:23:52 EDT
Correction on 5. 

5. When a reviewer Person B change a not empty translation done by Person A, and save as approved/rejected, statistic count goes to Person A. (We are assuming reviewer should not make any changes on translation while in review process)
Comment 10 Ding-Yi Chen 2014-10-30 02:03:12 EDT
VERIFIED with Zanata 3.6.0-SNAPSHOT (git-jenkins-zanata-server-github-pull-requests-2242)
Comment 11 Carlos Munoz 2014-11-11 18:24:43 EST
Reopening this bug as there are a couple of things that we need to fix:
- 'Untranslated' should not show up as a contribution.
- There should be no 'Total' field in the returned API. Total does not make sense in the context of contributions.

These two fields were not mentioned in the original story discussions.
Comment 13 Alex Eng 2014-11-17 18:56:39 EST
Updated behavior:

1. Contribution stats only consists of translated, needReview, approved, rejected.

1. When Person A update translation X, contribution stats should +1 on whichever state Person A last save translation X as.

2. When a reviewer Person B approve or reject a translation of Person A, statistic count goes to Person A. (only translation statistic)

3. When a reviewer Person B starts to translate an empty translation, and save as approved/rejected, statistic count goes to Person B. (Person B translated a string).

4. When a reviewer Person B change a not empty translation done by Person A, and save as approved/rejected, statistic count goes to Person B. Person A count should not be affected. (We are assuming reviewer should not make any changes on translation while in review process)
Comment 14 Ding-Yi Chen 2014-11-17 19:43:46 EST
(In reply to Luke Brooker from comment #3)
> Thanks for the update Carlos this is good to know.
> 
> Having some sort of "reviewed" statistic would be interesting, how would we
> define this?
> 
> Approved *and* rejected/issue?
> 
> The review interface is something I think we can do a lot better than other
> tools.

The scope of this bug is statistic for translators' contribution.
In other words, we deal with "Translator A was reviewed" not "Reviewer B reviewed".
Comment 15 Ding-Yi Chen 2014-11-18 00:07:13 EST
Test basis and decision table:

https://github.com/zanata/zanata-server/wiki/Translator-Statistics
Comment 16 Alex Eng 2014-11-19 22:52:06 EST
Correction on 4th point: 
When a reviewer Person B change a not empty translation done by Person A, and save as approved/rejected, statistic count goes to Person A. (We are assuming reviewer should not make any changes on translation while in review process)(In reply to Alex Eng from comment #13)

> Updated behavior:
> 
> 1. Contribution stats only consists of translated, needReview, approved,
> rejected.
> 
> 1. When Person A update translation X, contribution stats should +1 on
> whichever state Person A last save translation X as.
> 
> 2. When a reviewer Person B approve or reject a translation of Person A,
> statistic count goes to Person A. (only translation statistic)
> 
> 3. When a reviewer Person B starts to translate an empty translation, and
> save as approved/rejected, statistic count goes to Person B. (Person B
> translated a string).
> 
> 4. When a reviewer Person B change a not empty translation done by Person A,
> and save as approved/rejected, statistic count goes to Person B. Person A
> count should not be affected. (We are assuming reviewer should not make any
> changes on translation while in review process)
Comment 17 Ding-Yi Chen 2014-11-20 01:37:42 EST
VERIFIED Zanata 3.6.0-SNAPSHOT (git-jenkins-zanata-server-verify-integration-master-3978-SUCCESS-1-g9e86c7d)

Note You need to log in before you can comment on or make changes to this bug.