Red Hat Bugzilla – Full Text Bug Listing
|Summary:||pungi strips langpacks metadata from comps, breaks langpacks installation for DVD|
|Product:||[Fedora] Fedora||Reporter:||Adam Williamson <awilliam>|
|Component:||yum||Assignee:||Fedora Packaging Toolset Team <packaging-team>|
|Status:||CLOSED ERRATA||QA Contact:||Fedora Extras Quality Assurance <extras-qa>|
|Version:||18||CC:||admiller, dennis, dmach, ffesti, jreznik, kparal, notting, packaging-team, robatino, tflink, tla, zpavlas|
|Target Milestone:||---||Keywords:||CommonBugs, Reopened|
|Fixed In Version:||yum-3.4.3-47.fc18||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|Last Closed:||2012-12-13 18:46:17 EST||Type:||Bug|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
|Bug Depends On:||881756|
Description Adam Williamson 2012-11-21 16:09:32 EST
As figured out by notting: pungi is stripping the <langpacks> section from comps, which is breaking install of langpacks from DVD (and if you do a network install using a repo created by pungi). * comps from Beta RC1 DVD image: [adamw@adam repodata]$ grep langpacks b662714c38fb63326868be3d82fc6229a5abd40d11f49e39a9e484bcf07d1502-Fedora-18-Beta-comps.xml <packagereq type="mandatory">yum-langpacks</packagereq> * comps from https://dl.fedoraproject.org/pub/fedora/linux/development/18/x86_64/os/repodata/ : [adamw@adam images]$ grep langpacks 7231ecf2aa3d3a16c8b8c2483992f72b0d71d09b4ec06348b4c3c840171fdf84-comps-f18.xml <packagereq>yum-langpacks</packagereq> <langpacks> </langpacks> Spot the difference. As notting put it: <notting> pungi assembles the comps files from all the repos it uses for the compose, and uses yum.comps.xml() to write out a merged version. unfortunately, that means every section that yum.comps doesn't specifically know about falls out Proposing NTH, to match the install 'langpacks doesn't work' bug - 868869 - which was NTH.
Comment 1 Adam Williamson 2012-11-21 21:49:18 EST
So the bit that needs fixing is actually yum. Specifically yum.comps.xml() . It needs to know about langpacks, and I guess anything else we've put in comps lately...
Comment 2 Kamil Páral 2012-11-23 10:42:53 EST
Do I understand correctly this could be a Final blocker? All critical path actions on release-blocking desktop environments should correctly display all sufficiently complete translations available for use https://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria If DVD installation fails to install language packs, then we can't correctly display translations.
Comment 3 Adam Williamson 2012-11-24 03:17:54 EST
yes, it very likely is.
Comment 4 Kamil Páral 2012-11-26 10:19:59 EST
Adam, I have done F18 Beta DVD/netinst/Live installation with Czech language, and yum-langpacks is installed every time. $ grep -E '(-lang|-cs|-czech)' dvd_packages yum-langpacks-0.3.0-3.fc18.noarch $ grep -E '(-lang|-cs|-czech)' netinst_packages yum-langpacks-0.3.0-4.fc18.noarch $ grep -E '(-lang|-cs|-czech)' live_packages yum-langpacks-0.3.0-3.fc18.noarch No other l10n packages are installed, but I can't even found any in the repositories. There used to be language specific yum groups, now there are none. I wanted to document this on CommonBugs page, but I fail to understand this problem properly. yum-langpacks are installed every time and no other language packs are available. What am I missing?
Comment 5 Adam Williamson 2012-11-27 05:43:24 EST
kamil: AIUI, this, from comps-f18.xml.in: <langpacks> <match install="LabPlot-doc-%s" name="LabPlot-doc"/> <match install="aspell-%s" name="aspell"/> <match install="autocorr-%s" name="autocorr-en"/> <match install="calligra-l10n-%s" name="calligra-core"/> <match install="childsplay-alphabet_sounds_%s" name="childsplay"/> <match install="eclipse-nls-%s" name="eclipse-platform"/> <match install="firefox-langpack-%s" name="firefox"/> <match install="gcompris-sound-%s" name="gcompris"/> <match install="gimp-help-%s" name="gimp-help"/> <match install="hunspell-%s" name="hunspell"/> <match install="hyphen-%s" name="hyphen"/> <match install="kde-l10n-%s" name="kdelibs"/> <match install="kde-i18n-%s" name="kdelibs3"/> <match install="libreoffice-langpack-%s" name="libreoffice-core"/> <match install="man-pages-%s" name="man-pages"/> <match install="moodle-%s" name="moodle"/> <match install="mythes-%s" name="mythes"/> <match install="nqc-doc-%s" name="nqc-doc"/> <match install="openoffice.org-langpack-%s" name="openoffice.org-core"/> <match install="tesseract-langpack-%s" name="tesseract"/> <match install="tkgate-%s" name="tkgate"/> </langpacks> If I'm understanding correctly, what this means is that, say, if libreoffice-core is being installed, you should get libreoffice-langpack-cs . If I'm correct about this bug, if you do a stock install via netinst you'll get probably at least aspell-cs , firefox-langpack-cs , libreoffice-langpack-cs , man-pages-cs , and maybe some others; if you do a stock install via DVD you'll get none of these. it seems pretty obvious how this works from the above snippet - comps defines what packages have their translations split out into locale-named subpackages, and the exact format for the names, and then yum does a simple 'if this then that' operation when installing.
Comment 6 Adam Williamson 2012-11-27 05:44:19 EST
though, hum, now I see your post again, it looks like it failed even in the netinst case. can you check if you remove libreoffice-core and install it again, does libreoffice-langpack-cs get picked up?
Comment 7 Adam Williamson 2012-11-27 05:54:39 EST
I just did a netinst from Beta RC1 netinst iso using French as the language, picked minimal + Standard, and got man-pages-fr and hunspell-fr. Will compare DVD. If it's not working for Czech, maybe there's another bug, to do with the Czech country code or smth?
Comment 8 Adam Williamson 2012-11-27 06:09:18 EST
Yup, on my French DVD install, man-pages-fr and hunspell-fr were missing. So in French the bug is exactly as I thought it was.
Comment 9 Kamil Páral 2012-11-27 08:42:06 EST
Adam, I can confirm your findings. With DVD Czech, hunspell-cs is not installed, but it is installed when I do "yum reinstall hunspell". When I do "yum install libreoffice", libreoffice-langpack-cs is correctly installed. With netinst Czech, hunspell-cs is installed out of the box. I have no idea why it was not the case in comment 4. Maybe I did some mistake.
Comment 10 Adam Williamson 2012-12-03 13:38:12 EST
Discussed at 2012-12-03 blocker review meeting: http://meetbot.fedoraproject.org/fedora-qa/2012-12-03/f18final-blocker-review-1.2.2012-12-03-17.25.log.txt . This is accepted as a blocker per criterion cited in comment #2 (note that web use is in the critpath, and Firefox translations are affected). This acceptance is based on our belief that a simple 'yum update firefox' (for e.g.) after install will not cause the translation to be added - I'm currently testing this. If it does cause the translation to be added, we may re-evaluate the blocker status for this bug.
Comment 11 Adam Williamson 2012-12-03 14:23:34 EST
I confirmed that a simple 'yum update firefox' after installing with this bug does not appear to add the firefox translation to the package set. So the above comment stands, this is a blocker.
Comment 12 Daniel Mach 2012-12-06 07:11:34 EST
This needs pungi and yum to be patched. I think I've got patches for both. Yum patch is attached to bug#881756. I'll provide pungi patches today or tomorrow at latest (need to review my git first).
Comment 13 Jaroslav Reznik 2012-12-06 07:27:54 EST
Dan, thanks - I'm going to ping yum guys to include the patch. And this bug should be split to two bugs for Fedora - yum and pungi one.
Comment 14 Tim Flink 2012-12-06 11:36:18 EST
I just tried a test compose with a pungi built using Daniel's patches but I'm getting a TB pretty early on in the process: Start: chroot pungi --force --nosource --nodebuginfo -G -C -B -I --force --flavor Fedora --name Fedora --ver 18 -c /tmp/fedora-install-fedora.ks Pungi:INFO: Adding repo fedora Pungi:INFO: URL for repo fedora is ['http://infrastructure.phx2.fedoraproject.org/pub/fedora/linux/development/18/x86_64/os/'] Pungi:INFO: Adding repo siderepo Pungi:INFO: URL for repo siderepo is ['file:///tmp/siderepo/'] Pungi:INFO: Getting sacks for arches ['x86_64', 'noarch', 'src'] fedora | 4.2 kB 00:00 siderepo | 2.9 kB 00:00 !!! Pungi:WARNING: The --nobase kickstart option is no longer supported; Please add @base to the kickstart if you want it in. Warning: Reusing existing destination directory. Traceback (most recent call last): File "/usr/bin/pungi", line 264, in <module> main() File "/usr/bin/pungi", line 102, in main mypungi.getPackageObjects() File "/usr/lib/python2.7/site-packages/pypungi/__init__.py", line 608, in getPackageObjects if "core" not in self.ksparser.handler.packages.groupList: File "/usr/lib/python2.7/site-packages/pykickstart/parser.py", line 252, in __cmp__ if self.name < other.name: AttributeError : 'str' object has no attribute 'name' The warning about @base is a little annoying. I'm not using --nobase and I did add @base and @core to the ks. I'm using pykickstart-1.99.22-1.fc18 on the builder and did upgrade yum to yum-3.4.3-47.fc18
Comment 15 Daniel Mach 2012-12-06 13:09:47 EST
Thanks for cathing that; here's a new build: http://koji.fedoraproject.org/koji/taskinfo?taskID=4764307
Comment 16 Tim Flink 2012-12-06 16:22:31 EST
(In reply to comment #15) > Thanks for cathing that; here's a new build: > http://koji.fedoraproject.org/koji/taskinfo?taskID=4764307 Updated to this build and got a different TB: Start: chroot pungi --force --nosource --nodebuginfo -G -C -B -I --force --flavor Fedora --name Fedora --ver 18 -c /tmp/fedora-install-fedora.ks Pungi:INFO: Adding repo fedora Pungi:INFO: URL for repo fedora is ['http://infrastructure.phx2.fedoraproject.org/pub/fedora/linux/development/18/x86_64/os/'] Pungi:INFO: Adding repo siderepo Pungi:INFO: URL for repo siderepo is ['file:///tmp/siderepo/'] Pungi:INFO: Getting sacks for arches ['x86_64', 'noarch', 'src'] fedora | 4.2 kB 00:00 siderepo | 2.9 kB 00:00 !!! Warning: Reusing existing destination directory. Traceback (most recent call last): File "/usr/bin/pungi", line 264, in <module> main() File "/usr/bin/pungi", line 102, in main mypungi.getPackageObjects() File "/usr/lib/python2.7/site-packages/pypungi/__init__.py", line 621, in getPackageObjects searchlist.extend(self.getPackagesFromGroup(group)) File "/usr/lib/python2.7/site-packages/pypungi/__init__.py", line 550, in getPackagesFromGroup pkgs = self.ayum.bestPackagesFromList(pkgs, arch=self.ayum.compatarch) AttributeError : 'PungiYum' object has no attribute 'compatarch' Finish: chroot pungi --force --nosource --nodebuginfo -G -C -B -I --force --flavor Fedora --name Fedora --ver 18 -c /tmp/fedora-install-fedora.ks
Comment 17 Fedora Update System 2012-12-07 05:10:11 EST
yum-3.4.3-47.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/FEDORA-2012-19846/yum-3.4.3-47.fc18
Comment 18 Daniel Mach 2012-12-07 09:16:38 EST
I've fixed it but found a single dependency incorrectly resolved when I tested the pungi build on RHEL. I'll need to write new tests to cover this corner-case. I also realized there might be a workaround. You could achieve all this by using the original pungi (pungi-2.12-1.fc18) and patched yum (yum-3.4.3-47.fc18) and updating the langpack patterns in f18 kickstart (see the # Langpacks section in /usr/share/spin-kickstarts/fedora-install-fedora.ks).
Comment 19 Tim Flink 2012-12-10 17:08:46 EST
(In reply to comment #18) > I also realized there might be a workaround. > You could achieve all this by using the original pungi (pungi-2.12-1.fc18) > and patched yum (yum-3.4.3-47.fc18) and updating the langpack patterns in > f18 kickstart (see the # Langpacks section in > /usr/share/spin-kickstarts/fedora-install-fedora.ks). OK, but I don't quite understand what you mean about updating the langpack patters in the kickstart. Can you be a bit more specific?
Comment 20 Fedora Update System 2012-12-11 00:54:03 EST
yum-3.4.3-47.fc18 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report.
Comment 21 Daniel Mach 2012-12-11 02:50:15 EST
(In reply to comment #19) > (In reply to comment #18) > > I also realized there might be a workaround. > > You could achieve all this by using the original pungi (pungi-2.12-1.fc18) > > and patched yum (yum-3.4.3-47.fc18) and updating the langpack patterns in > > f18 kickstart (see the # Langpacks section in > > /usr/share/spin-kickstarts/fedora-install-fedora.ks). > > OK, but I don't quite understand what you mean about updating the langpack > patters in the kickstart. Can you be a bit more specific? /usr/share/spin-kickstarts/fedora-install-fedora.ks # Langpacks aspell-* autocorr-* eclipse-nls-* hunspell-* hyphen-* calligra-l10n-* kde-l10n-* kde-i18n-* libreoffice-langpack-* man-pages-* mythes-* If there are some other langpacks to be pulled into composes, then they have to be added here (AFAIK there are more langpacks in comps than these).
Comment 22 Kamil Páral 2012-12-11 04:56:59 EST
We still need to verify, setting ON_QA.
Comment 23 Bill Nottingham 2012-12-11 14:08:15 EST
Those langpacks should be all the ones *for things that are pulled onto the DVD*.
Comment 24 Bill Nottingham 2012-12-11 14:11:22 EST
I lied. I'll add gimp-help-*.
Comment 25 Tim Flink 2012-12-12 18:15:30 EST
I built a smoke image with yum-3.4.3-47 and the changes to fedora-install-fedora.ks (and a few other changes that shouldn't affect this bug) http://dl.fedoraproject.org/pub/alt/qa/20121212_f18-smoke6/
Comment 26 Adam Williamson 2012-12-12 18:41:45 EST
Small problem with adding gimp-help-*: it don't fit. Each gimp-help package is approx. 30MB: Name : gimp-help-de Arch : noarch Version : 2.8.0 Release : 3.fc18 Size : 31 M and there are 20 languages. So smoke6 DVD, built with this change, is 600MB larger than smoke5 DVD, and is now oversize.
Comment 27 Jaroslav Reznik 2012-12-13 08:36:06 EST
Maybe we should be more specific on supported languages (do we actually have a list of "the must" languages supported on install media???) - for gimp - 600 MB just for langpacks is really a lot. So there are two options: a) cut down supported languages (could be a little bit messy in ks as all languages has to be listed but with limited set...) b) be more selective in what langpacks we want on DVD (but it's probably cleaner to install only specific language than selectivly not install langpack for a few packages) c) but it's long term one - do the review of what we already have now and cut it down...
Comment 28 Bill Nottingham 2012-12-13 12:16:31 EST
So, yeah, gimp-help is way too big for the ~150MB of space we have available. I've removed it and we will just have a langpack gap with gimp on the DVD. If it wasn't the last minute, we could discuss more nuanced cutting of other groups or content.
Comment 29 Adam Williamson 2012-12-13 18:46:17 EST
So, with a smoke6 French install, I get hunspell-fr and man-pages-fr . So the bug seems fixed. There are a few nitty issues in the langpacks themselves - the openoffice one looks like it needs updating to libreoffice, and the Firefox langpack packages no longer seem to exist, they seem to be back in the main firefox package. But I believe this bug is fixed.