Bug 963698

We don't have a proper Final release criteria when it comes to third-party packages, but it's a so common operation that it should warrant at least a blocker bug discussion. Nominating.

Expected impact: All languages that have a non-ascii string returned from yum backend. Probably a lot of languages.

This might be the same bug with a different traceback:
http://lists.fedoraproject.org/pipermail/test/2013-May/115495.html

This reproducer illustrates the problem:

Save this as gpg-lang-test.py:

#!/usr/bin/python

import os
import yum
from yum.packages import YumLocalPackage

yb = yum.YumBase()

for attrname in ("gpgcheck", "repo_gpgcheck", "localpkg_gpgcheck"):
    if hasattr(yb.conf, attrname):
        setattr(yb.conf, attrname, True)

po = YumLocalPackage(ts=yb.rpmdb.readOnlyTS(), filename="/home/hughsie/Downloads/rpmfusion-free-release-branched.noarch.rpm")
try:
    yb._checkSignatures([po], None)
except yum.Errors.YumGPGCheckError, e:
    print str(e)
print "all working"

running it like this "sudo LANG=cs_CZ.UTF-8 ./gpg-lang-test.py" gives:

UnicodeEncodeError: 'ascii' codec can't encode character u'\u0159' in position 2: ordinal not in range(128)

Even doing this "print unicode(e, 'utf-8', errors='replace')" produces:

TypeError: coercing to Unicode: need string or buffer, YumGPGCheckError found

Which seems to suggest something like this is required?

diff --git a/backends/yum/yumBackend.py b/backends/yum/yumBackend.py
index 8e04999..289f10f 100755
--- a/backends/yum/yumBackend.py
+++ b/backends/yum/yumBackend.py
@@ -44,6 +44,7 @@ from yum.callbacks import *
 from yum.misc import prco_tuple_to_string, unique
 from yum.packages import YumLocalPackage, parsePackages
 from yum.packageSack import MetaSack
+from yum.Errors import YumBaseError
 import rpmUtils
 import exceptions
 import types
@@ -120,7 +121,9 @@ def sigquit(signum, frame):
     sys.exit(1)
 
 def _to_unicode(txt, encoding='utf-8'):
-    if isinstance(txt, basestring):
+    if isinstance(txt, YumBaseError):
+        txt = unicode(txt)
+    elif isinstance(txt, basestring):
         if not isinstance(txt, unicode):
             txt = unicode(txt, encoding, errors='replace')
     return txt

This certainly solves the bug for me, but I'm kinda wondering why YumBaseError.__str__() would return anything other than unicode.

def _to_unicode(txt, encoding='utf-8'):
    if isinstance(txt, str):
        # decode str to unicode
        return unicode(txt, encoding, errors='replace')
    # exception instances
    return unicode(txt)

I don't think we need isinstance(txt, YumBaseError) check there.
This is simpler, and works as expected in all 4 cases:

>>> _to_unicode(u'ěšč')
u'\u011b\u0161\u010d'
>>> _to_unicode('ěšč')
u'\u011b\u0161\u010d'
>>> _to_unicode(yum.Errors.YumBaseError('ěšč'))
u'\u011b\u0161\u010d'
>>> _to_unicode(yum.Errors.YumBaseError(u'ěšč'))
u'\u011b\u0161\u010d'

> but I'm kinda wondering why YumBaseError.__str__() would return anything other than unicode.

__str__() is called when Python needs bytes, and yes, it should always work and not raise encoding errors.  The recommended practice is to do all string formatting in __unicode__() and have __str__() just return self.__unicode__().encode('utf-8').  This implements it.

http://lists.baseurl.org/pipermail/yum-devel/2013-May/010157.html

(In reply to comment #4)
> self.__unicode__().encode('utf-8').  This implements it.
> http://lists.baseurl.org/pipermail/yum-devel/2013-May/010157.html

Awesome. Can you pin me when this gets into a released yum version and I'll drop the hack in PK and dep on the new yum version. Thanks!

How does this relate to https://bugzilla.redhat.com/show_bug.cgi?id=963810 ? Does that PackageKit (PackageKit-0.8.9-1.fc19 ) fix the problem?

Discussed at 2013-05-29 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2013-05-29/f19final-blocker-review-1.2013-05-29-16.02.log.txt . We'd at least like to see the results of testing with the latest PK before making a call on this.

It does not crash for me with PackageKit-0.8.9-1.fc19.x86_64.

Verified fixed with PackageKit-0.8.9-1.fc19.x86_64 and yum-3.4.3-83.fc19.noarch. Closing.

Richard, if you're still waiting for something from Zdeněk, feel free to reopen to track it.