Bug 1787204 - please migrate to the new Fedora translation platform
Summary: please migrate to the new Fedora translation platform
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 32
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Marek Blaha
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-01-01 12:56 UTC by jibecfed
Modified: 2020-02-16 07:06 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-02-13 04:38:17 UTC
Type: Bug


Attachments (Terms of Use)

Description jibecfed 2020-01-01 12:56:11 UTC
Hello, the Fedora project migrates its translation platform to Weblate [1].

This tool directly interact with your git repository, and requires us to know:
* [mandatory] which branch is your development branch?
* [mandatory] have you merged latest translation from Zanata and locked the project?
* [info] Weblate will handle updates when pot file changes, don't edit po files for this [2]
* [optional] what is the license of translation? (please use a code from https://spdx.org/licenses/)
* [optional] do you have any announcement/warning you would like to display to the translators? (it will be displayed in Weblate)
* [optional] do you need us to activate any specific checks? (this is a setting per component [3])
* [optional] do you need us to automatically detect new translation files? (typical usecase: website translation with one translation file per page)

Please note:
* For github hosted projects, Weblate open pull request. For other project, you'll have to add a key to allow commits.
* In Weblate's vocable, one project is a group of component. Each component is a translation file. You can have many projects or many components or both.
* You can change your mind over time, just reach trans@lists.fedoraproject.org

[1] https://communityblog.fedoraproject.org/fedora-localization-platform-migrates-to-weblate/ 
[2] https://docs.weblate.org/en/latest/admin/continuous.html#avoiding-merge-conflicts
[3] https://docs.weblate.org/en/latest/user/checks.html#translation-checks

Comment 1 Marek Blaha 2020-01-03 07:41:32 UTC
(In reply to jibecfed from comment #0)
> * [mandatory] which branch is your development branch?

All development happens in the "master" branch. We are also using maintenance branches specific to particular releases - especially for RHEL releases. Our translation workflow is like this:
- each day we push current version of the dnf.pot file into zanata
- before each release we pull *.po files from Zanata

> * [mandatory] have you merged latest translation from Zanata and locked the
> project?

We are merging translations from Zanata right before every release. I'm not sure about locking the Zanata project - there are branches in Zanata used for RHEL releases and we probably still need them.

> * [info] Weblate will handle updates when pot file changes, don't edit po
> files for this [2]

The *.po files are not manually edited - they are pulled from Zanata.

> * [optional] what is the license of translation? (please use a code from
> https://spdx.org/licenses/)
> * [optional] do you have any announcement/warning you would like to display
> to the translators? (it will be displayed in Weblate)
> * [optional] do you need us to activate any specific checks? (this is a
> setting per component [3])

We need to make sure that number of leading/trailing spaces are the same in the original message and in translations. Currently the translators on Zanata are not careful enough and we must do some post-processing of *.po files to be sure that numbers of these spaces match.

> * [optional] do you need us to automatically detect new translation files?
> (typical usecase: website translation with one translation file per page)

We use just one pot file per component.

> 
> Please note:
> * For github hosted projects, Weblate open pull request. For other project,
> you'll have to add a key to allow commits.

Is there any project already using Weblate with pull requests on github? I'm a bit afraid how big the number of translation commits could be. Is it possible to use some CLI tool to pull the *.po files from weblate (the same way as we do it currently with zanata)?

> * In Weblate's vocable, one project is a group of component. Each component
> is a translation file. You can have many projects or many components or both.
> * You can change your mind over time, just reach
> trans@lists.fedoraproject.org

I'm not sure about (dis)advantages of using one project with multiple component vs. multiple projects each with just one component. What we need is to have multiple branches of each component - basically upstream, and maintenance branches for releases.

Comment 2 jibecfed 2020-01-06 13:06:00 UTC
thanks for your answers, the key question you asked: https://github.com/ibus/ibus/pull/2169
What Weblate do: a git squash per author (https://docs.weblate.org/en/weblate-3.10/admin/addons.html#squash-git-commits)

Please not we are talking about upstream translation. If RHEL branches are hosted on fedora.zanata.org, they should be included in this migration.

I suggest you to have a dedicated repository for all DNF projects, provide direct commit to weblate, and store them in one folder per branch.
Your current process would push the new pot to the repository, and pull translated po files.

Your repository would looks like:

dnf-l10n:
* dnf-master:
  a.po
  b.po
  c.po
  dnf.pot
* dnf-rhel8:
  a.po
  b.po
  c.po
  dnf.pot
* dnf-plugins-master:
  a.po
  b.po
  c.po
  dnf-plugins.pot

I'll take care of Weblate's configuration to keep your life easy.

Comment 3 jibecfed 2020-01-07 20:48:29 UTC
Slight modification in what I said earlier: please create one repository per project so that I can properly configure "Repository browser" setting of a component, see https://docs.weblate.org/en/weblate-3.10/admin/projects.html#component

Comment 4 Ludek Janda 2020-01-14 12:47:03 UTC
Please can you delay migration and start after February 18? We are in the middle of RHEL 8.2 localization cycle, our team is working in Zanata and I want to minimize potential risk. February 18 is the deadlin for RHEL 8.2 translations. Thanks.

Comment 5 jibecfed 2020-01-17 18:26:31 UTC
I asked the community for opinion on this, but you were included in the conversation, please answer: https://lists.fedoraproject.org/archives/list/trans@lists.fedoraproject.org/thread/I4JWI7KS2IO2SER3QHFOOTOFTEC6KGEW/

I see no reasons to delay and ask you to not delay this migration.

Comment 6 Ludek Janda 2020-02-05 09:57:22 UTC
Hi Marek, no objections now. Our localization work for 8.2 is delivered now. You can go ahead and start migration to the new platform. Thanks.

Comment 7 jibecfed 2020-02-05 20:55:12 UTC
as we have most of our discussion here, I comment for all of our 4 tickets related to dnf.

1. thanks for continuing the work here, I appreciate it
2. DNF is a big project, would you like to start with one component to experiment and then apply the same strategy on all others?

It would probably be easier for you and me to coordinate.

Comment 8 Marek Blaha 2020-02-06 06:59:57 UTC
Thanks Ludek. Let's start then.

@jibecfed yes, we could begin with dnf. I would prefer not to mix multiple component into one github repo (as suggested in comment#2). I'd rather create one l10n repo per component - libdnf, dnf, dnf-plugins-core and dnf-plugins-extras. At the moment I'm not sure whether having all our current branches (basically master, rhel-x) converted into directories is a good idea - does it have any advantages to the translators? If not, I'd rather go with the same branching scheme in l10s repos as in the main ones.

Comment 9 Marek Blaha 2020-02-06 07:27:06 UTC
I created translations repository here: https://github.com/rpm-software-management/dnf-l10n and pushed current version of dnf.pot file from dnf/master into it. Should I also add *.po files from zanata? Current zanata translations do not respect leading/trailing spaces so they need a bit of post-processing which I can do.

Comment 10 jibecfed 2020-02-06 20:09:07 UTC
Translation license should be the same as the software. Please use the same license as dnf:  GPL 2.0 or later.

Yes, you should pull all po files from Zanata.
You can fix leading/trailing space if needed. For my knowledge, why would you do so, can it crash dnf?

About versions. What is the most logical to me: provide as much translation branch as software branches you maintain.

If you work with target releases (like fedora-x, fedora-y, rhel-x,...), then provide the same branches to translators.
The translation platform will take care of syncing files.

Comment 11 Ludek Janda 2020-02-07 06:21:02 UTC
Hi Jean-Baptiste,

is there an easy way to syncronize between branches in weblate? I just did not have a good experience with cockpit where they have 8.2 and master branches now. When I pushed our latest translations to 8.2, confirm their QA check issues there, I basically had to repeat everything in master, ie. upload again, confirm QA. If that's the case, I would actually prefer the opposite, ie. having as least branches as possible. 

Thanks, Ludek

Comment 12 jibecfed 2020-02-07 11:09:15 UTC
Weblate propagates automatically translations on all existing component of a project. Here: project and components are Weblate's one.
In addition, Weblate detects inconsistencies between component translation. As an example: https://translate.fedoraproject.org/checks/inconsistent/cockpit/?language=fr
Worse case scenario, I can import the translation of another component by using the automatic translation feature: https://translate.fedoraproject.org/projects/cockpit/master/fr/#auto

So, as a translator, having one or many component isn't complex situation to handle.

The question here is: what is your detailed workflow?
I feel like this is about how to have processes that allow both community work and RH internal process, which is out of scope of the current issue.

Could you please describe it in details in a ticket in this repository? http://pagure.io/fedora-l10n/tickets
From there, we can work to lower the work for you.

Comment 13 Marek Blaha 2020-02-10 08:38:26 UTC
@jibecfed the licence was changed and I pushed also *.po files from zanata (the master branch at the moment). The trailing space do not crash dnf per se, but they definitely can break its output. I think weblate has some checks to ensure that leading/trailing spaces in original and translated strings match, hasn't it?

Comment 14 jibecfed 2020-02-10 21:41:13 UTC
first configuration was done: https://translate.fedoraproject.org/projects/dnf/master/

please allow commits from https://github.com/weblate

Comment 15 Marek Blaha 2020-02-11 14:19:39 UTC
Weblate user was given write access to dnf-l10n

Comment 16 Ben Cotton 2020-02-11 16:30:46 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 17 jibecfed 2020-02-13 04:38:17 UTC
everything looks like to work, have a nice day

Comment 18 Marek Blaha 2020-02-14 11:49:21 UTC
@jibecfed please, can I get admin rights on the weblate dnf project? my user name there is mblaha.

Comment 19 jibecfed 2020-02-16 07:06:32 UTC
Done


Note You need to log in before you can comment on or make changes to this bug.