Bug 879030

So the bit that needs fixing is actually yum. Specifically yum.comps.xml() . It needs to know about langpacks, and I guess anything else we've put in comps lately...

Do I understand correctly this could be a Final blocker?
 All critical path actions on release-blocking desktop environments should correctly display all sufficiently complete translations available for use 
https://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria

If DVD installation fails to install language packs, then we can't correctly display translations.

yes, it very likely is.

Adam, I have done F18 Beta DVD/netinst/Live installation with Czech language, and yum-langpacks is installed every time.

$ grep -E '(-lang|-cs|-czech)' dvd_packages
yum-langpacks-0.3.0-3.fc18.noarch
$ grep -E '(-lang|-cs|-czech)' netinst_packages
yum-langpacks-0.3.0-4.fc18.noarch
$ grep -E '(-lang|-cs|-czech)' live_packages
yum-langpacks-0.3.0-3.fc18.noarch

No other l10n packages are installed, but I can't even found any in the repositories. There used to be language specific yum groups, now there are none.

I wanted to document this on CommonBugs page, but I fail to understand this problem properly. yum-langpacks are installed every time and no other language packs are available. What am I missing?

kamil: AIUI, this, from comps-f18.xml.in:

  <langpacks>
    <match install="LabPlot-doc-%s" name="LabPlot-doc"/>
    <match install="aspell-%s" name="aspell"/>
    <match install="autocorr-%s" name="autocorr-en"/>
    <match install="calligra-l10n-%s" name="calligra-core"/>
    <match install="childsplay-alphabet_sounds_%s" name="childsplay"/>
    <match install="eclipse-nls-%s" name="eclipse-platform"/>
    <match install="firefox-langpack-%s" name="firefox"/>
    <match install="gcompris-sound-%s" name="gcompris"/>
    <match install="gimp-help-%s" name="gimp-help"/>
    <match install="hunspell-%s" name="hunspell"/>
    <match install="hyphen-%s" name="hyphen"/>
    <match install="kde-l10n-%s" name="kdelibs"/>
    <match install="kde-i18n-%s" name="kdelibs3"/>
    <match install="libreoffice-langpack-%s" name="libreoffice-core"/>
    <match install="man-pages-%s" name="man-pages"/>
    <match install="moodle-%s" name="moodle"/>
    <match install="mythes-%s" name="mythes"/>
    <match install="nqc-doc-%s" name="nqc-doc"/>
    <match install="openoffice.org-langpack-%s" name="openoffice.org-core"/>
    <match install="tesseract-langpack-%s" name="tesseract"/>
    <match install="tkgate-%s" name="tkgate"/>
  </langpacks>

If I'm understanding correctly, what this means is that, say, if libreoffice-core is being installed, you should get libreoffice-langpack-cs . If I'm correct about this bug, if you do a stock install via netinst you'll get probably at least aspell-cs , firefox-langpack-cs , libreoffice-langpack-cs , man-pages-cs , and maybe some others; if you do a stock install via DVD you'll get none of these.

it seems pretty obvious how this works from the above snippet - comps defines what packages have their translations split out into locale-named subpackages, and the exact format for the names, and then yum does a simple 'if this then that' operation when installing.

though, hum, now I see your post again, it looks like it failed even in the netinst case. can you check if you remove libreoffice-core and install it again, does libreoffice-langpack-cs get picked up?

I just did a netinst from Beta RC1 netinst iso using French as the language, picked minimal + Standard, and got man-pages-fr and hunspell-fr. Will compare DVD. If it's not working for Czech, maybe there's another bug, to do with the Czech country code or smth?

Yup, on my French DVD install, man-pages-fr and hunspell-fr were missing. So in French the bug is exactly as I thought it was.

Adam, I can confirm your findings. With DVD Czech, hunspell-cs is not installed, but it is installed when I do "yum reinstall hunspell". When I do "yum install libreoffice", libreoffice-langpack-cs is correctly installed.

With netinst Czech, hunspell-cs is installed out of the box. I have no idea why it was not the case in comment 4. Maybe I did some mistake.

Discussed at 2012-12-03 blocker review meeting: http://meetbot.fedoraproject.org/fedora-qa/2012-12-03/f18final-blocker-review-1.2.2012-12-03-17.25.log.txt . This is accepted as a blocker per criterion cited in comment #2 (note that web use is in the critpath, and Firefox translations are affected). This acceptance is based on our belief that a simple 'yum update firefox' (for e.g.) after install will not cause the translation to be added - I'm currently testing this. If it does cause the translation to be added, we may re-evaluate the blocker status for this bug.

I confirmed that a simple 'yum update firefox' after installing with this bug does not appear to add the firefox translation to the package set. So the above comment stands, this is a blocker.

This needs pungi and yum to be patched.
I think I've got patches for both.

Yum patch is attached to bug#881756.
I'll provide pungi patches today or tomorrow at latest (need to review my git first).

Dan,
thanks - I'm going to ping yum guys to include the patch. And this bug should be split to two bugs for Fedora - yum and pungi one.

I just tried a test compose with a pungi built using Daniel's patches but I'm getting a TB pretty early on in the process:

Start: chroot pungi --force --nosource --nodebuginfo -G -C -B -I --force --flavor Fedora --name Fedora --ver 18 -c /tmp/fedora-install-fedora.ks
Pungi:INFO: Adding repo fedora
Pungi:INFO: URL for repo fedora is ['http://infrastructure.phx2.fedoraproject.org/pub/fedora/linux/development/18/x86_64/os/']
Pungi:INFO: Adding repo siderepo
Pungi:INFO: URL for repo siderepo is ['file:///tmp/siderepo/']
Pungi:INFO: Getting sacks for arches ['x86_64', 'noarch', 'src']
fedora                                                   | 4.2 kB     00:00     
siderepo                                                 | 2.9 kB     00:00 !!! 
Pungi:WARNING: The --nobase kickstart option is no longer supported; Please add @base to the kickstart if you want it in.
Warning: Reusing existing destination directory.
Traceback (most recent call last):
  File "/usr/bin/pungi", line 264, in <module>
     main()
  File "/usr/bin/pungi", line 102, in main
     mypungi.getPackageObjects()
  File "/usr/lib/python2.7/site-packages/pypungi/__init__.py", line 608, in getPackageObjects
     if "core" not in self.ksparser.handler.packages.groupList:
  File "/usr/lib/python2.7/site-packages/pykickstart/parser.py", line 252, in __cmp__
     if self.name < other.name:
AttributeError :  'str' object has no attribute 'name' 

The warning about @base is a little annoying. I'm not using --nobase and I did add @base and @core to the ks.

I'm using pykickstart-1.99.22-1.fc18 on the builder and did upgrade yum to yum-3.4.3-47.fc18

Thanks for cathing that; here's a new build:
http://koji.fedoraproject.org/koji/taskinfo?taskID=4764307

(In reply to comment #15)
> Thanks for cathing that; here's a new build:
> http://koji.fedoraproject.org/koji/taskinfo?taskID=4764307

Updated to this build and got a different TB:

Start: chroot pungi --force --nosource --nodebuginfo -G -C -B -I --force --flavor Fedora --name Fedora --ver 18 -c /tmp/fedora-install-fedora.ks
Pungi:INFO: Adding repo fedora
Pungi:INFO: URL for repo fedora is ['http://infrastructure.phx2.fedoraproject.org/pub/fedora/linux/development/18/x86_64/os/']
Pungi:INFO: Adding repo siderepo
Pungi:INFO: URL for repo siderepo is ['file:///tmp/siderepo/']
Pungi:INFO: Getting sacks for arches ['x86_64', 'noarch', 'src']
fedora                                                   | 4.2 kB     00:00     
siderepo                                                 | 2.9 kB     00:00 !!! 
Warning: Reusing existing destination directory.
Traceback (most recent call last):
  File "/usr/bin/pungi", line 264, in <module>
     main()
  File "/usr/bin/pungi", line 102, in main
     mypungi.getPackageObjects()
  File "/usr/lib/python2.7/site-packages/pypungi/__init__.py", line 621, in getPackageObjects
     searchlist.extend(self.getPackagesFromGroup(group))
  File "/usr/lib/python2.7/site-packages/pypungi/__init__.py", line 550, in getPackagesFromGroup
     pkgs = self.ayum.bestPackagesFromList(pkgs, arch=self.ayum.compatarch)
AttributeError :  'PungiYum' object has no attribute 'compatarch' 
Finish: chroot pungi --force --nosource --nodebuginfo -G -C -B -I --force --flavor Fedora --name Fedora --ver 18 -c /tmp/fedora-install-fedora.ks

yum-3.4.3-47.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/FEDORA-2012-19846/yum-3.4.3-47.fc18

I've fixed it but found a single dependency incorrectly resolved when I tested the pungi build on RHEL. I'll need to write new tests to cover this corner-case.

I also realized there might be a workaround.
You could achieve all this by using the original pungi (pungi-2.12-1.fc18) and patched yum (yum-3.4.3-47.fc18) and updating the langpack patterns in f18 kickstart (see the # Langpacks section in /usr/share/spin-kickstarts/fedora-install-fedora.ks).

(In reply to comment #18)
> I also realized there might be a workaround.
> You could achieve all this by using the original pungi (pungi-2.12-1.fc18)
> and patched yum (yum-3.4.3-47.fc18) and updating the langpack patterns in
> f18 kickstart (see the # Langpacks section in
> /usr/share/spin-kickstarts/fedora-install-fedora.ks).

OK, but I don't quite understand what you mean about updating the langpack patters in the kickstart. Can you be a bit more specific?

yum-3.4.3-47.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

(In reply to comment #19)
> (In reply to comment #18)
> > I also realized there might be a workaround.
> > You could achieve all this by using the original pungi (pungi-2.12-1.fc18)
> > and patched yum (yum-3.4.3-47.fc18) and updating the langpack patterns in
> > f18 kickstart (see the # Langpacks section in
> > /usr/share/spin-kickstarts/fedora-install-fedora.ks).
> 
> OK, but I don't quite understand what you mean about updating the langpack
> patters in the kickstart. Can you be a bit more specific?


/usr/share/spin-kickstarts/fedora-install-fedora.ks

# Langpacks
aspell-*
autocorr-*
eclipse-nls-*
hunspell-*
hyphen-*
calligra-l10n-*
kde-l10n-*
kde-i18n-*
libreoffice-langpack-*
man-pages-*
mythes-*


If there are some other langpacks to be pulled into composes, then they have to be added here (AFAIK there are more langpacks in comps than these).

We still need to verify, setting ON_QA.

Those langpacks should be all the ones *for things that are pulled onto the DVD*.

I lied. I'll add gimp-help-*.

I built a smoke image with yum-3.4.3-47 and the changes to fedora-install-fedora.ks (and a few other changes that shouldn't affect this bug)

http://dl.fedoraproject.org/pub/alt/qa/20121212_f18-smoke6/

Small problem with adding gimp-help-*: it don't fit.

Each gimp-help package is approx. 30MB:

Name        : gimp-help-de
Arch        : noarch
Version     : 2.8.0
Release     : 3.fc18
Size        : 31 M

and there are 20 languages. So smoke6 DVD, built with this change, is 600MB larger than smoke5 DVD, and is now oversize.

Maybe we should be more specific on supported languages (do we actually have a list of "the must" languages supported on install media???) - for gimp - 600 MB just for langpacks is really a lot.

So there are two options:
a) cut down supported languages (could be a little bit messy in ks as all languages has to be listed but with limited set...)
b) be more selective in what langpacks we want on DVD (but it's probably cleaner to install only specific language than selectivly not install langpack for a few packages)
c) but it's long term one - do the review of what we already have now and cut it down...

So, yeah, gimp-help is way too big for the ~150MB of space we have available. I've removed it and we will just have a langpack gap with gimp on the DVD. If it wasn't the last minute, we could discuss more nuanced cutting of other groups or content.

So, with a smoke6 French install, I get hunspell-fr and man-pages-fr . So the bug seems fixed.

There are a few nitty issues in the langpacks themselves - the openoffice one looks like it needs updating to libreoffice, and the Firefox langpack packages no longer seem to exist, they seem to be back in the main firefox package. But I believe this bug is fixed.