Created attachment 339506 [details] Full mergerepos.log Description of problem: Not sure if this is the right component since it's caused by mergerepos, though the traceback is coming from yum Version-Release number of selected component (if applicable): koji-builder-1.3.1-1.fc10.noarch yum-3.2.21-2.fc10.noarch How reproducible: Add dist-f7 repos for the release and updates Steps to Reproduce: 1. koji add-tag dist-f7-updates 2. koji add-external-repo -t dist-f7-updates dist-f7-release http://www.kanarip.com/archive/fedora/releases/7/Everything/\$arch/os/ 3. koji add-external-repo -t dist-f7-updates dist-f7-updates http://www.kanarip.com/archive/fedora/updates/7/\$arch/ Let koji run mergerepos on these. Actual results: Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 241, in <module> main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 236, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 216, in write_metadata mdgen.doPkgMetadata() File "/usr/lib/python2.5/site-packages/createrepo/__init__.py", line 332, in doPkgMetadata self.writeMetadataDocs(packages) File "/usr/lib/python2.5/site-packages/createrepo/__init__.py", line 472, in writeMetadataDocs self.primaryfile.write(po.xml_dump_primary_metadata()) File "/usr/lib/python2.5/site-packages/yum/packages.py", line 943, in xml_dump_primary_metadata msg += misc.to_unicode(self._dump_format_items()) File "/usr/lib/python2.5/site-packages/yum/packages.py", line 809, in _dump_format_items msg += self._dump_pco('provides') UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 38: ordinal not in range(128) Additional info: The full mergerepos.log is attached
mergerepos and this version of koji should most likely require yum 3.2.22 Can you upgrade your version of yum and retest this?
reopen this if you can provide the info in question. thanks
Sorry for the delayed response. It seems yum-3.2.22 is not available for Fedora 10, is that correct?
It's in koji. http://koji.fedoraproject.org/koji/buildinfo?buildID=101778 check it out there - it'll be in updates-testing "shortly"
any change on this? I think we fixed it up - but I verification is nice.
I've upgraded to F-11 meanwhile, not seeing the issue there. Maybe one of the people in CC: can confirm the issue is resolved, but for now I'm closing this bug.
It occurred on F-11 now, too: koji-builder-1.3.1-2.fc11.noarch yum-3.2.23-3.fc11.noarch == Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 241, in <module> main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 236, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 216, in write_metadata mdgen.doPkgMetadata() File "/usr/lib/python2.6/site-packages/createrepo/__init__.py", line 364, in doPkgMetadata self.writeMetadataDocs(packages) File "/usr/lib/python2.6/site-packages/createrepo/__init__.py", line 527, in writeMetadataDocs self.primaryfile.write(po.xml_dump_primary_metadata()) File "/usr/lib/python2.6/site-packages/yum/packages.py", line 1015, in xml_dump_primary_metadata msg += misc.to_unicode(self._dump_format_items()) File "/usr/lib/python2.6/site-packages/yum/packages.py", line 894, in _dump_format_items msg += self._dump_pco('provides') File "/usr/lib/python2.6/site-packages/yum/packages.py", line 919, in _dump_pco msg += pcostring UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 32: ordinal not in range(128)
It seems i386 fails on aspell-ca all the time, whereas x86_64 fails on rsyslog-mysql.
Digging a little further on x86_64, I added a statement on line 919 of /usr/lib/python2.6/site-packages/yum/packages.py: pcostring += "/>\n" + print "%r" % pcostring msg += pcostring The following is the result: == ' <rpm:entry name="libcamel-1.2.so.14()(64bit)"/>\n' u' <rpm:entry name="evolution-webcal" flags="EQ" epoch="0" ver="2.26.1"/>\n' u' <rpm:entry name="evolution-webcal" flags="LT" epoch="0" ver="2.24.0"/>\n' 1591/13929 - evolution-data-server-2.26.1-1.fc11.x86_64 u' <rpm:entry name="hunspell-st" flags="EQ" epoch="0" ver="0.20060123" rel="2.fc11"/>\n' 1592/13929 - hunspell-st-0.20060123-2.fc11.noarch u' <rpm:entry name="rsyslog-mysql(x86-64)" flags="EQ" epoch="0" ver="3.21.11" rel="1.fc11"/>\n' u' <rpm:entry name="rsyslog-mysql" flags="EQ" epoch="0" ver="3.21.11" rel="1.fc11"/>\n' ' <rpm:entry name="ommysql.so()(64bit)"/>\n' 1593/13929 - rsyslog-mysql-3.21.11-1.fc11.x86_64 u' <rpm:entry name="ipa-pgothic-fonts" flags="EQ" epoch="0" ver="003.01" rel="2.fc11"/>\n' ' <rpm:entry name="font(ipap\xe3\x82\xb4\xe3\x82\xb7\xe3\x83\x83\xe3\x82\xaf)"/>\n' Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 241, in <module> main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 236, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 216, in write_metadata mdgen.doPkgMetadata() File "/usr/lib/python2.6/site-packages/createrepo/__init__.py", line 364, in doPkgMetadata self.writeMetadataDocs(packages) File "/usr/lib/python2.6/site-packages/createrepo/__init__.py", line 527, in writeMetadataDocs self.primaryfile.write(po.xml_dump_primary_metadata()) File "/usr/lib/python2.6/site-packages/yum/packages.py", line 1016, in xml_dump_primary_metadata msg += misc.to_unicode(self._dump_format_items()) File "/usr/lib/python2.6/site-packages/yum/packages.py", line 894, in _dump_format_items msg += self._dump_pco('provides') File "/usr/lib/python2.6/site-packages/yum/packages.py", line 920, in _dump_pco msg += pcostring UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 32: ordinal not in range(128) ==
The packages I have to block in order to be able to generate a full merged repo are: - apanov-edrip-fonts - apanov-heuristica-fonts - baekmuk-ttf-fonts - hanazono-fonts - ipa-mincho-fonts - ipa-pgothic-fonts - ipa-pmincho-fonts - sazanami-fonts - vlgothic-fonts - wqy-zenhei-fonts - ipa-gothic-fonts Apparently, something in RPM has a non-utf8/non-unicode lang(some-japanese-characters) foo which causes yum.packages._dump_pco() to fail.
(In reply to comment #10) > Apparently, something in RPM has a non-utf8/non-unicode > lang(some-japanese-characters) foo which causes yum.packages._dump_pco() to > fail. We now export the font names font files declared in rpm metadata for font autoinstallation to work. They can be complex (non ASCII) Unicode. Though IIRC fc-query exports valid UTF-8, something else must be corrupting this data later
I've hit this same issuing running kojira/koji-builders using F11. The newRepo task trying to run is for dist-f12, using a local mirror of yesterday's rawhide. I had to block the following packages: un-core-fonts wqy-zenhei-fonts hanazono-fonts vlgothic-fonts sazanami-fonts ipa-mincho-fonts ipa-pgothic-fonts ipa-pmincho-fonts ipa-gothic-fonts baekmuk-ttf-fonts apanov-edrip-fonts apanov-heuristica-fonts
These two patches (which have been applied to yum upstream) should fix the problems with unicode Provides: causing mergerepos to fail: http://lists.baseurl.org/pipermail/yum-devel/2009-August/005714.html http://lists.baseurl.org/pipermail/yum-devel/2009-August/005719.html
(In reply to comment #13) > These two patches (which have been applied to yum upstream) should fix the > problems with unicode Provides: causing mergerepos to fail: > > http://lists.baseurl.org/pipermail/yum-devel/2009-August/005714.html > > http://lists.baseurl.org/pipermail/yum-devel/2009-August/005719.html I applied the second listed here locally and it does indeed seem to clear up the mergerepo failures I was seeing. I did not need to block packages any longer.
(In reply to comment #14) > (In reply to comment #13) > > These two patches (which have been applied to yum upstream) should fix the > > problems with unicode Provides: causing mergerepos to fail: > > > > http://lists.baseurl.org/pipermail/yum-devel/2009-August/005714.html > > > > http://lists.baseurl.org/pipermail/yum-devel/2009-August/005719.html > > I applied the second listed here locally and it does indeed seem to clear up > the mergerepo failures I was seeing. I did not need to block packages any > longer. Correcting myself, both patches are needed. I ran into other issues with a later newRepo task. The last line in the first patch should be ignored though.
both patches are in yum in f11 and f12/rawhide