Bug 438580 - Yum UnicodeEncodeError when doing yum update
Summary: Yum UnicodeEncodeError when doing yum update
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Fedora
Classification: Fedora
Component: yum
Version: rawhide
Hardware: i386
OS: Linux
low
medium
Target Milestone: ---
Assignee: Seth Vidal
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-03-22 12:28 UTC by Adrian "Adi1981" P.
Modified: 2014-01-21 23:02 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2008-04-14 19:59:24 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
yum strace (9.48 KB, application/octet-stream)
2008-04-10 23:04 UTC, Adrian "Adi1981" P.
no flags Details

Description Adrian "Adi1981" P. 2008-03-22 12:28:21 UTC
Description of problem:

When doing yum update (daily updated rawhide) i've received:

yum update --exclude=xorg\* --exclude=linuxwacom --skip-broken
/.../
(38/40): selinux-policy-3 100% |=========================| 430 kB    00:05
(39/40): yum-versionlock- 100% |=========================| 8.8 kB    00:00
(40/40): cairo-devel-1.5. 100% |=========================| 158 kB    00:03
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 236, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 192, in main
    return exFatal(e)
  File "/usr/share/yum-cli/yummain.py", line 63, in exFatal
    logger.critical('\n\n%s', str(e))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-2:
ordinal not in range(128)

Second run gone smoothly without any problems.

Version-Release number of selected component (if applicable):
rpm -q yum
yum-3.2.13-1.fc9.noarch
rpm -q python
python-2.5.1-23.fc9.i386

How reproducible:
Randomly

Steps to Reproduce:
n/a
  
Actual results:
Traceback and exit with error code

Expected results:
Clean update

Additional info:
LANG=pl_PL.UTF-8

Comment 1 Adrian "Adi1981" P. 2008-03-30 14:12:39 UTC
I'm receiving this errors now quite often, means once per every 2-3 yum updates,
and usually it depends on what package is being updated - probably some of
packages have some characters? that yum can't handle. So sometimes yum update is
just impossible - it always finishes then with: 

File "/usr/share/yum-cli/yummain.py", line 63, in exFatal
    logger.critical('\n\n%s', str(e))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0142' in position
30: ordinal not in range(128)

As packages are already downloaded, manually rpm -U from yum cache folder helps.

Comment 2 Adrian "Adi1981" P. 2008-04-08 05:29:59 UTC
Problem still exist on newest version of yum

(17/18): rhpxl-1.3-1.fc9.i386.rpm                        |  97 kB     00:00
(18/18): libgweather-2.22.1.1-1.fc9.i386.rpm             | 1.8 MB     00:06
ostrzeżenie: rpmts_HdrFromFdno: Nagłówek V3 Podpis DSA: NOKEY, key ID a109b1ec
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 236, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 192, in main
    return exFatal(e)
  File "/usr/share/yum-cli/yummain.py", line 63, in exFatal
    logger.critical('\n\n%s', str(e))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0142' in position
30: ordinal not in range(128)

$ rpm -q yum
yum-3.2.14-2.fc9.noarch

Comment 3 Seth Vidal 2008-04-08 05:43:12 UTC
This is NOT a complete patch but if you can recreate the error/problem apply
this patch and see what it outputs, please.

http://skvidal.fedorapeople.org/patches/yum-exit-error-unicode.patch

Comment 4 Adrian "Adi1981" P. 2008-04-09 10:18:32 UTC
Looks like it works ok, but still need more updates to test it well.

Comment 5 Adrian "Adi1981" P. 2008-04-09 10:25:29 UTC
uhhh... i was too fast. Just crashed it again.

Comment 7 Adrian "Adi1981" P. 2008-04-09 15:15:44 UTC
unfortunately still no luck:
/.../
(10/12): libxml2-devel-2.6.32-1.fc9.i386.rpm             | 2.2 MB     00:23
(11/12): gnome-desktop-2.22.1-2.fc9.i386.rpm             | 975 kB     00:11
(12/12): libxslt-1.1.23-1.fc9.i386.rpm                   | 528 kB     00:09
ostrzeżenie: rpmts_HdrFromFdno: Nagłówek V3 Podpis DSA: NOKEY, key ID a109b1ec
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 236, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 192, in main
    return exFatal(e)
  File "/usr/share/yum-cli/yummain.py", line 63, in exFatal
    logger.critical('\n\n%s', str(e))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0142' in position
30: ordinal not in range(128)
[root@localhost Pobrane]# rpm -q yum
yum-3.2.14-3.fc9.noarch

But in this ver. crash with "yum info glibc" is eliminated (there was no BZ for it).

Comment 8 Seth Vidal 2008-04-10 15:46:18 UTC
When you got the above error were you redirecting the output to a file or other
program?


Comment 9 Johan Cwiklinski 2008-04-10 16:08:21 UTC
For myself, nope, a simple "yum update" or "yum-complete-transaction" give this
error.

Comment 10 Adrian "Adi1981" P. 2008-04-10 16:21:24 UTC
No, only "yum update" directly from console (xterm, yakuake), without any
redirections. Here's full log from today's update:
http://www.wklej.org/id/1efe34a808

Comment 12 Adrian "Adi1981" P. 2008-04-10 20:02:26 UTC
Still the same.

Comment 13 James Antill 2008-04-10 20:19:15 UTC
 Can you try changing:

/usr/share/yum-cli/output.py:scriptout

...to:

    def scriptout(self, package, msgs):
            if True:
                sys.stdout.write(to_unicode(msgs))
            else:
                sys.stdout.write(msgs)

...ie. remove the isatty() test.


Comment 14 James Antill 2008-04-10 20:20:16 UTC
and that should be:

    def scriptout(self, package, msgs):
         if msgs:
            if True:
                sys.stdout.write(to_unicode(msgs))
            else:
                sys.stdout.write(msgs)

...sorry, accidentally deleted a line.


Comment 16 Adrian "Adi1981" P. 2008-04-10 23:04:46 UTC
Created attachment 302085 [details]
yum strace

Well, it still didn't fix the problem, but i think i've found guilty. I've made
"strace yum update" and this gives me interesting info before traceback about
missing gpg key from livna repo:

gettimeofday({1207867139, 691711}, NULL) = 0
stat64("/etc/pki/rpm-gpg/RPM-GPG-KEY-livna", 0xbfd87c28) = -1 ENOENT (No such
file or directory)
write(2, "Traceback (most recent call last"..., 35Traceback (most recent call
last):

After importing gpg key from livna, update was successfully. But without this
key i will always got this traceback. Also --nogpg option makes the trick. So
in fact it looks that the problem is not in encoding, but in handling GPG keys
as i think. It would be nice if yum could tell user that gpg key is missing
rather than show useless (for normal user) traceback without any info about GPG
key.

I have attached last lines from output of strace and yum install amule (from
livna) with, and without GPG key imported

Comment 17 James Antill 2008-04-11 05:29:45 UTC
 Yum does allow you to import keys, in most locale's ... can you remove the
livna key again and try changing /usr/lib/python2.5/site-packages/yum/i18n.py from:



    import gettext
    t = gettext.translation('yum')
    _ =  t.ugettext

...to:

    import gettext
    t = gettext.translation('yum')
    _ =  t.gettext

...ie. remove the u as the first character from the last line.


Comment 18 Adrian "Adi1981" P. 2008-04-11 13:36:51 UTC
This gives me only another error:

# yum install amule
Wczytane wtyczki: aliases, allowdowngrade, changelog, downloadonly, fedorakmod,
                : kernel-module, list-data, merge-conf, presto, priorities,
                : security, tsflags, verify, versionlock
Setting up and reading Presto delta metadata
No Presto metadata available for livna-development
No Presto metadata available for koji
No Presto metadata available for livna
No Presto metadata available for rawhide
Wykluczanie pakietów na globalnej liście wykluczonych pakietów
Finished
Reading version lock configuration
Ustawianie procesu instalacji
Analizowanie argumentów instalacji pakietów
Rozwiązywanie zależności
--> Wykonywanie sprawdzania transakcji
---> Pakiet amule.i386 0:2.1.3-3.lvn9 zostanie zaktualizowany
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 236, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 152, in main
    (result, resultmsgs) = base.buildTransaction()
  File "/usr/lib/python2.5/site-packages/yum/__init__.py", line 624, in
buildTransaction
    (rescode, restring) = self.resolveDeps()
  File "/usr/lib/python2.5/site-packages/yum/depsolve.py", line 649, in resolveDeps
    CheckDeps, checkinstalls, checkremoves, missing = self._resolveRequires(errors)
  File "/usr/lib/python2.5/site-packages/yum/depsolve.py", line 720, in
_resolveRequires
    _("Checking deps for %s") %(txmbr,))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 16: ordinal
not in range(128)

Comment 19 Adrian "Adi1981" P. 2008-04-11 13:40:31 UTC
But chaning in one from 2 above lines, ie.
t = gettext.translation('yum')

to
t = ugettext.translation('yum')

gives error about gpg key missing:
Dependencies Resolved

=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 amule                   i386       2.1.3-3.lvn9     livna-development  4.1 M

Transaction Summary
=============================================================================
Install      1 Package(s)
Update       0 Package(s)
Remove       0 Package(s)

Total size: 4.1 M
Downloading Packages:
Downloading DeltaRPMs:
Rebuilding rpms from deltarpms
ostrzeżenie: rpmts_HdrFromFdno: Nagłówek V3 Podpis DSA: NOKEY, key ID a109b1ec


GPG key retrieval failed: [Errno 5] OSError: [Errno 2] Nie ma takiego pliku ani
katalogu: '/etc/pki/rpm-gpg/RPM-GPG-KEY-livna'
[root@localhost yum]# 

Nie ma takiego pliku ani katalogu: = There's no such file or directory:

Comment 20 James Antill 2008-04-11 14:03:28 UTC
Yeh, doing ugettext.translation() is an error which will switch translations off
for yum.

 Ok, thanks for testing. I'll see if I can come up with another fix.


Comment 21 James Antill 2008-04-11 23:38:38 UTC
 Ok, we're pretty sure this one will fix it and if not there's a config. option
to turn the i18n stuff off now:

http://koji.fedoraproject.org/packages/yum/3.2.14/6.fc9/noarch/yum-3.2.14-6.fc9.noarch.rpm

Comment 22 Adrian "Adi1981" P. 2008-04-12 00:48:43 UTC
well, i must unfortunately disappoint you :(
I've already tried this build ~ 2 hours ago,and this build didn't fix issue, but
worst, it produced few new issues. One is i think known issue from version 3.12
afair - yum doesn't ask if we want to complete transaction or not, and it's just
quitting with message like "canceled on user request" (in log, in polish, it's
massage Zamykanie na polecenie użytkownika):


[root@localhost Paczki]# yum reinstall amule
Wczytane wtyczki: aliases, allowdowngrade, changelog, downloadonly, fedorakmod,
                : kernel-module, list-data, merge-conf, presto, priorities,
                : security, tsflags, verify, versionlock
Ustawianie procesu ponownej instalacji
Setting up and reading Presto delta metadata
livna-development                                        | 2.1 kB     00:00
No Presto metadata available for livna-development
koji                                                     | 2.4 kB     00:00
No Presto metadata available for koji
livna                                                    | 2.1 kB     00:00
No Presto metadata available for livna
rawhide                                                  | 2.4 kB     00:00
No Presto metadata available for rawhide
Wykluczanie pakietów na globalnej liście wykluczonych pakietów
Finished
Reading version lock configuration
Rozwiązywanie zależności
--> Wykonywanie sprawdzania transakcji
---> Pakiet amule.i386 0:2.1.3-3.lvn9 zostanie zaktualizowany
---> Pakiet amule.i386 0:2.1.3-3.lvn9 zostanie usunięty
--> Zakończono rozwiązywanie zależności

Rozwiązano zależności

=============================================================================
 Pakiet                  Architektura  Wersja           Repozytorium      Rozmiar
=============================================================================
Instalowanie:
 amule                   i386       2.1.3-3.lvn9     livna-development  4.1 M
Usuwanie:
 amule                   i386       2.1.3-3.lvn9     installed          10 M

Podsumowanie transakcji
=============================================================================
Instalowanie       1 pakietów
Aktualizowanie     0 pakietów
Usuwanie           1 pakietów

Całkowity rozmiar pobierania: 4.1 M
Zamykanie na polecenie użytkownika
Zakończono!
[root@localhost Paczki]#

and second issue we get when we'll typ just "yum" command, it produces following
backtrace:

[root@localhost Paczki]# yum
Wczytane wtyczki: aliases, allowdowngrade, changelog, downloadonly, fedorakmod,
                : kernel-module, list-data, merge-conf, presto, priorities,
                : security, tsflags, verify, versionlock
Musisz podać jakieś polecenie
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 236, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 91, in main
    base.getOptionsConfig(args)
  File "/usr/share/yum-cli/cli.py", line 209, in getOptionsConfig
    self.parseCommands() # before we return check over the base command + args
  File "/usr/share/yum-cli/cli.py", line 230, in parseCommands
    self.usage()
  File "/usr/share/yum-cli/cli.py", line 899, in usage
    self.optparser.print_help()
  File "/usr/lib/python2.5/optparse.py", line 1648, in print_help
    file.write(self.format_help().encode(encoding, "replace"))
  File "/usr/lib/python2.5/codecs.py", line 303, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 193:
ordinal not in range(128)

This error is caused by lack of messages.mo file which, as i believe, should be
yum.mo file since i don't have messages.mo in system (should i have it ?).
Here's result of strace:
stat64("/usr/share/locale/pl_PL.UTF8/LC_MESSAGES/messages.mo", 0xbfc0a398) = -1
ENOENT (No such file or directory)
stat64("/usr/share/locale/pl_PL/LC_MESSAGES/messages.mo", 0xbfc0a398) = -1
ENOENT (No such file or directory)
stat64("/usr/share/locale/pl.UTF8/LC_MESSAGES/messages.mo", 0xbfc0a398) = -1
ENOENT (No such file or directory)
stat64("/usr/share/locale/pl/LC_MESSAGES/messages.mo", 0xbfc0a398) = -1 ENOENT
(No such file or directory)
stat64("/usr/share/locale/pl_PL.UTF8/LC_MESSAGES/messages.mo", 0xbfc0a5e8) = -1
ENOENT (No such file or directory)
stat64("/usr/share/locale/pl_PL/LC_MESSAGES/messages.mo", 0xbfc0a5e8) = -1
ENOENT (No such file or directory)
stat64("/usr/share/locale/pl.UTF8/LC_MESSAGES/messages.mo", 0xbfc0a5e8) = -1
ENOENT (No such file or directory)
stat64("/usr/share/locale/pl/LC_MESSAGES/messages.mo", 0xbfc0a5e8) = -1 ENOENT
(No such file or directory)
brk(0x9080000)                          = 0x9080000
write(2, "Traceback (most recent call last"..., 35Traceback (most recent call last):
) = 35

And of course result of reinstalling amule without gpg key from livna with
forcing yum to install it with -y:

[root@localhost LC_MESSAGES]# yum reinstall amule -y -d1

=============================================================================
 Pakiet                  Architektura  Wersja           Repozytorium      Rozmiar
=============================================================================
Instalowanie:
 amule                   i386       2.1.3-3.lvn9     livna-development  4.1 M
Usuwanie:
 amule                   i386       2.1.3-3.lvn9     installed          10 M

Podsumowanie transakcji
=============================================================================
Instalowanie       1 pakietów
Aktualizowanie     0 pakietów
Usuwanie           1 pakietów

Całkowity rozmiar pobierania: 4.1 M
ostrzeżenie: rpmts_HdrFromFdno: Nagłówek V3 Podpis DSA: NOKEY, key ID a109b1ec
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 236, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 192, in main
    return exFatal(e)
  File "/usr/share/yum-cli/yummain.py", line 63, in exFatal
    logger.critical('\n\n%s', str(e))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0142' in position
30: ordinal not in range(128)

Btw. what is the option to turn i18n off? man yum.conf didn't help with it. 

Comment 23 Adrian "Adi1981" P. 2008-04-12 01:06:15 UTC
[root@localhost share]# rpm -q yum
yum-3.2.14-6.fc9.noarch

I've reverted yum to 3.2.11 and 3.2.12, and this problem with gpg also exist
there, 3.2.11 gives:

Total size: 4.1 M
Is this ok [y/N]: y
Downloading Packages:
Downloading DeltaRPMs:
Rebuilding rpms from deltarpms
ostrzeżenie: rpmts_HdrFromFdno: Nagłówek V3 Podpis DSA: NOKEY, key ID a109b1ec

GPG key retrieval failed: [Errno 5] OSError: [Errno 2] Nie ma takiego pliku ani
katalogu: '/etc/pki/rpm-gpg/RPM-GPG-KEY-livna'


and 3.2.12 gives traceback like in 3.2.14. Maybe it'll be useful also, that
livna *.repo file was added manually, without importing gpg keys. If there are
some else informations that can be useful to fix his problem, let me know.

Comment 24 Adrian "Adi1981" P. 2008-04-14 19:55:55 UTC
With -7 everything works fine here. Problem with importing GPG key was due to
different name of the key, yum was searching for
/etc/pki/rpm-gpg/RPM-GPG-KEY-livna, not RPM-LIVNA-GPG-KEY which i had on my
system. After changing name of the key yum have successfully imported key and
for now everything works fine here atm. And if there's no proper gpg kkey we've
got now clear error about missing key, not backtraces. I think i can close bug. 

Thanks for your great work!


Note You need to log in before you can comment on or make changes to this bug.