Bug 1547166

This is weird, because Zanata has correct plural forms for Belarusian. How can we get someone from Zanata to look into this?

Hi,

Do not get me wrong, but if you decided to change the plural forms anyway...

The plural form will be changed for all files at once. And it's not a good thing.

Please consider the following form:

Plural-Forms: nplurals=4; plural=n==1 ? 3 : n%10==1 && n%100!=11 ? 0 : n%10>=2 
&& n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2;\n"

or in human readable form:

3rd form for n=1
0th form for (n div 10=1) and (n div 100 not equal 11) or simpler "21, 31..."
1st form for 2, 3, 4, 22, 23 , 24...
2nd form for 5, 6, 7, 8...

The standard form (nplurals=3) works only for the messages with a replacer.

Consider the examples from gettext man:

https://www.gnu.org/savannah-checkouts/gnu/gettext/manual/html_node/Plural-forms.html

1.

ngettext ("%d file removed", "%d files removed", n)

Perfect. The standard form works just fine. %d is replaced by the number and 
all 3 standard forms fit.

2.

ngettext ("One file removed", "%d files removed", n)

Trouble. The form 0 for Slavic languages is for 1, 21, 31... But in words they 
are not "One", they are "One, Twenty-one..." Such plurals have problems with Zanata web editor.

If you translate

"One file removed"
"%d files removed"
"%d files removed"

the validation will be failed because the first form (0th case) does not 
contain enough cases to be split into "1, 21, 31..."

If you translate

"%d file removed"
"%d files removed"
"%d files removed"

the validation will be failed because there is "%d" in the first case, but the 
checker thinks there should not be any replacer...

3.

ngettext ("Delete the selected file?",
                "Delete the selected files?", n)

Fail. Without number (replacer), there are only two plural forms in the Slavic languages: "one" and "many". But the 0th rule in the standard (CLDR) combines 
"one" and "many". They should be split to give the correct translation.

That's why Russian, Serbian and Ukrainian KDE and GNOME teams use the plural 
rules with 4 forms (n=1 is separated in its own case).

As a Fedora example (DNF package manager):

https://github.com/rpm-software-management/dnf/blob/master/po/uk.po#L2131

msgid "Package"
msgid_plural "Packages"

Is there some way to translate it into Belarusian right with 3 plural forms? ;)

GNOME uses “nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2)” for Belarusian (as you can see on https://l10n.gnome.org/teams/be/), as well as for Russian and Polish. You shouldn’t assume that every Slavic language works the same, and your second and third cases are incorrect uses of ngettext, not something we should work around for. Ideally someone should report bugs/pull requests to upstream projects that do it.

DNF example is special in that it does have a variable, but it’s outside the string. In https://bugzilla.redhat.com/attachment.cgi?id=1240059 it’s these lines:

Instalacja     2 pakiety
Aktualizacja  12 pakietów

This bug is specifically about Zanata and how it doesn’t use the plural rules set for a language.

If I get it correctly, zanata handles nplurals correctly for Belarusian: it provides me 4 placeholders for translation. For Polish, Ukrainian and Russian there are 3 placeholders for 3 plural forms in the same anaconda master translation web interface, which is correct according to nplurals in the corresponding .po files, downloaded via zanata web interface for the corresponding languages. In other words, in all 3 languages: Polish, Ukrainian, and Russian nplurals are correct, but in Belarusian it is not. And the fix is needed not in zanata code, but rather in configuration for Belarusian language, e.g. be.po for anaconda and elsewhere where needed.

As for the proposal of Yuri Chornoivan to set nplurals=4 with the new 3rd case for exactly “one”, then I’d better use for Belarusian the formula similar to existing formulae for Ukrainian and Russian.

(In reply to Viktar Siarheichyk from comment #4)
> If I get it correctly, zanata handles nplurals correctly for Belarusian: it
> provides me 4 placeholders for translation. For Polish, Ukrainian and
> Russian there are 3 placeholders for 3 plural forms in the same anaconda
> master translation web interface, which is correct according to nplurals in
> the corresponding .po files, downloaded via zanata web interface for the
> corresponding languages. In other words, in all 3 languages: Polish,
> Ukrainian, and Russian nplurals are correct, but in Belarusian it is not.
> And the fix is needed not in zanata code, but rather in configuration for
> Belarusian language, e.g. be.po for anaconda and elsewhere where needed.
> 

Here’s the thing: plural forms in Zanata for Belarusian are configured to use the default “nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2)”. I don’t know why be.po for anaconda has nplurals=4.

> As for the proposal of Yuri Chornoivan to set nplurals=4 with the new 3rd
> case for exactly “one”, then I’d better use for Belarusian the formula
> similar to existing formulae for Ukrainian and Russian.

I disagree with Yuri’s assessment, but that’s a problem for another day.

(In reply to Piotr Drąg from comment #5)
> (In reply to Viktar Siarheichyk from comment #4)
> > If I get it correctly, zanata handles nplurals correctly for Belarusian: it
> > provides me 4 placeholders for translation. For Polish, Ukrainian and
> > Russian there are 3 placeholders for 3 plural forms in the same anaconda
> > master translation web interface, which is correct according to nplurals in
> > the corresponding .po files, downloaded via zanata web interface for the
> > corresponding languages. In other words, in all 3 languages: Polish,
> > Ukrainian, and Russian nplurals are correct, but in Belarusian it is not.
> > And the fix is needed not in zanata code, but rather in configuration for
> > Belarusian language, e.g. be.po for anaconda and elsewhere where needed.
> > 
> 
> Here’s the thing: plural forms in Zanata for Belarusian are configured to
> use the default “nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 &&
> n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2)”. I don’t know why be.po for
> anaconda has nplurals=4.

It might happen that the check for standard plural forms has been removed for a while and the wrong form was uploaded. There is a Python script to change plural forms automatically in KDE's pology (by Chusslove Illich):

http://docs.linux.org.ua/%D0%9F%D0%B5%D1%80%D0%B5%D0%BA%D0%BB%D0%B0%D0%B4%D0%B8/%D0%9A%D0%BE%D1%80%D0%BE%D1%82%D0%BA%D0%B8%D0%B9_%D0%B4%D0%BE%D0%B2%D1%96%D0%B4%D0%BD%D0%B8%D0%BA_%D0%BF%D0%B5%D1%80%D0%B5%D0%BA%D0%BB%D0%B0%D0%B4%D0%B0%D1%87%D0%B0_KDE/

Or you can try and change the plural forms manually on local copy, then try to re-upload translation.

> > As for the proposal of Yuri Chornoivan to set nplurals=4 with the new 3rd
> > case for exactly “one”, then I’d better use for Belarusian the formula
> > similar to existing formulae for Ukrainian and Russian.
> 
> I disagree with Yuri’s assessment, but that’s a problem for another day.

We all free to disagree. ;)

Is there any bug report against the official gettext manual stated wrong things, btw?

(In reply to Yuri Chornoivan from comment #6)
> Or you can try and change the plural forms manually on local copy, then try
> to re-upload translation.
I successfully managed to run zanata-cli push and nplurals=3 now on server, and zanata web UI presents 3 placeholders to translate plural forms, so it seems everything OK now. Thanks. The bug can be closed.

Apparently not all is OK. Even though zanata web UI presents 3 placeholders to translate plural forms, zanata-cli pull returns a misformatted po file:

[WARN] Marking as fuzzy: too many plural forms for text flow: resId=1273d0db45545a9b269637da4d9d2889, doc=anaconda
[WARN] Marking as fuzzy: too many plural forms for text flow: resId=0a7d55be9d12a369a6a8da0fb517fba4, doc=anaconda
[WARN] Marking as fuzzy: too many plural forms for text flow: resId=6cc492bd732baf97c15d577104134a1b, doc=anaconda
[WARN] Marking as fuzzy: too many plural forms for text flow: resId=e4c936ce2cbdbab6c303bdd0a6166ee8, doc=anaconda

because there are several msgstr[3] entries present whereas nplurals=3 in the received .po file. When I remove the excess msgstr[3] entries and do zanata-cli push again, it merges with the existing "invalid" .po on the server and nothing changes. 

I seem to need a person with admin rights to clean up the be.po for anaconda on the server or whatever file or DB is under the hood there.

Besides I messed up with the .po header trying to update it, instead of changing the initial 3 lines:

# Belarusian translations for anaconda.
# Copyright (C) 2018 Red Hat, Inc.
# This file is distributed under the same license as the anaconda package.

- it added my changes between the contributor copyright strings while preserving the original boilerplate I intended to change.

(In reply to Yuri Chornoivan from comment #6)
> (In reply to Piotr Drąg from comment #5)
> > I disagree with Yuri’s assessment, but that’s a problem for another day.
> 
> We all free to disagree. ;)
> 
> Is there any bug report against the official gettext manual stated wrong
> things, btw?

Where is it wrong, exactly?

(In reply to Viktar Siarheichyk from comment #8)
> because there are several msgstr[3] entries present whereas nplurals=3 in
> the received .po file. When I remove the excess msgstr[3] entries and do
> zanata-cli push again, it merges with the existing "invalid" .po on the
> server and nothing changes. 
> 

Have you tried unchecking the merge option? It should then completely replace the .po file in Zanata.

(In reply to Piotr Drąg from comment #9)
> (In reply to Yuri Chornoivan from comment #6)
> > (In reply to Piotr Drąg from comment #5)
> > > I disagree with Yuri’s assessment, but that’s a problem for another day.
> > 
> > We all free to disagree. ;)
> > 
> > Is there any bug report against the official gettext manual stated wrong
> > things, btw?
> 
> Where is it wrong, exactly?

I do not know. That's why I'm asking. You wrote:

> your second and third cases are incorrect uses of ngettext, not something we should work around for.

But these cases are from the official gettext manual:

https://www.gnu.org/savannah-checkouts/gnu/gettext/manual/html_node/Plural-forms.html

These cases are used in GNOME (e.g., Shotwell) and KDE (e.g., digiKam) translations, so if you are right we should aware people that it is wrong. ;)

I’ll file a bug against upstream gettext one day, and then file bugs against projects abusing ngettext. But it is not this day. :)

Let’s focus on the bug in Zanata here.

(In reply to Piotr Drąg from comment #9)

> Have you tried unchecking the merge option? It should then completely
> replace the .po file in Zanata.

I get: "You do not have permission to access this resource." when uploading unchecking the merge option in the web UI or 

[ERROR] Operation failed: Failed while pushing document translations: [Failed to obtain permission('import-translation') with following facts([Project version(slug=master, status=ACTIVE)])]

when using the commandline tool.

I have always treated such cases as the application (developer's) bug rather than something that needs the plural rules to be adjusted. Unfortunately, developers are often unaware that not in all languages a singular number is used if and only if n==1 and there are languages where rules are different. The GNU gettext manual is correct: it says that in Russian, Belarusian, and few more languages if a number ends with 1 (except 11), which means 21, 31, ... then a singular number is used. This means that programmers should not use a "smart" hack to display special messages when the number is singular because it does not always mean that n==1. For example "Delete a file?", "You have a message" or "Yesterday" will not always work as expected. Of course, we can introduce one more plural rule for Russian, Belarusian and more languages to distinguish between "singular, n==1" and "singular, n!=1" and it works but isn't it confusing for the translators?

Note that even if it works for these languages where it will not work. For example:

- some languages also use a singular number when n==0 (French, IIRC);
- some languages do not have plurals, they have only one form.

See also this discussion: https://github.com/abrt/gnome-abrt/issues/112#issuecomment-152846843

Created attachment 1401966 [details]
Fixed anaconda file with 3 plural forms

Can you try the attached file?

May it be that some temporary removal of plurals using web editor can help?

(In reply to Yuri Chornoivan from comment #14)
> Created attachment 1401966 [details]
> Fixed anaconda file with 3 plural forms
> 
> Can you try the attached file?

In your file "Project-Id-Version: anaconda 28.22\n" and "POT-Creation-Date: 2018-02-19 18:09+0100\n while in the currently downloaded .po file: "Project-Id-Version: anaconda 29.1\n" and "POT-Creation-Date: 2018-02-28 14:54+0100\n" and several msgids differ. I fear that something might break if I upload an outdated .po. If you suggest to remove msgstr[3] entries then I did this actually, and they appear again when merging the uploaded .po file.


> May it be that some temporary removal of plurals using web editor can help?

If nplurals=3 in the .po file as is the case now, then only 3 plural forms are presented in the web editor.

@Alex Eng: There is not much time left to translation freeze. It seems that we have tried everything on our side. Can you help us, please?

(In reply to Yuri Chornoivan from comment #16)
> @Alex Eng: There is not much time left to translation freeze. It seems that
> we have tried everything on our side. Can you help us, please?

Belarusian translation is far from complete, so we will not catch the freeze anyway, however there is one error I cannot fix myself:

in anaconda 

$ make po-pull

$ LANG=C msgfmt -cv po/be.po 
po/be.po:14: nplurals = 3...
po/be.po:157: ...but some messages have 4 plural forms
msgfmt: found 1 fatal error
260 translated messages, 4 fuzzy translations, 990 untranslated messages.

and I can not fix that: I have no rights to replace .po in zanata, and if merging the abovementioned messages having 4 plural forms are not replacing if I upload .po files with fixed 3 plural forms (see comment 12 https://bugzilla.redhat.com/show_bug.cgi?id=1547166#c12 )

Is there any other way to upload the .po file? git?