Bug 244080

Summary: description with non-ASCII characters causes yum to barf
Product: [Fedora] Fedora Reporter: Sjoerd Mullender <sjoerd>
Component: yumAssignee: James Antill <james.antill>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: low Docs Contact:
Priority: low    
Version: 7CC: allane, cristian.ciupitu, jaroslaw.gorny, katzj, matt.wette
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: 3.2.1-1.fc7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-06-28 14:43:10 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sjoerd Mullender 2007-06-13 17:41:36 UTC
Description of problem:
If text to be printed in an RPM file contains a non-ASCII character and yum's
output is redirected to a pipe or file, yum crashes.

Version-Release number of selected component (if applicable):
yum-3.2.0-1.fc7

How reproducible:
Always (with the right RPM).  E.g. the RPM for aterm-1.0.0-7.fc7 contains an
ACUTE ACCENT (#00B4).

Steps to Reproduce:
1. yum info aterm | less   OR   yum info aterm > file
2.
3.
  
Actual results:
# yum info aterm > @i
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 94, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 264, in doCommands
    return self.yum_cli_commands[self.basecmd].doCommand(self, self.basecmd,
self.extcmds)
  File "/usr/share/yum-cli/yumcommands.py", line 165, in doCommand
    base.listPkgs(ypl.available, 'Available Packages', basecmd)
  File "/usr/share/yum-cli/output.py", line 102, in listPkgs
    self.infoOutput(pkg)
  File "/usr/share/yum-cli/output.py", line 73, in infoOutput
    print _("Description:\n%s") % pkg.description
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb4' in position
109: ordinal not in range(128)

The file @i contains everything up to the line with the non-ASCII character.

Expected results:
Nice output.

Additional info:
My locale ($LANG) is en_US.UTF-8.

Perhaps this is a bug in (in this case) aterm if RPMs aren't allowed to contain
non-ASCII?

Comment 1 Jeremy Katz 2007-06-13 17:57:04 UTC
Fixed upstream for yum 3.2.1

Comment 2 Jeremy Katz 2007-06-13 18:30:35 UTC
*** Bug 242341 has been marked as a duplicate of this bug. ***

Comment 3 Jeremy Katz 2007-06-18 21:17:21 UTC
*** Bug 244571 has been marked as a duplicate of this bug. ***

Comment 4 Fedora Update System 2007-06-21 22:03:35 UTC
yum-3.2.1-1.fc7 has been pushed to the Fedora 7 testing repository.  If problems still persist, please make note of it in this bug report.

Comment 5 Fedora Update System 2007-06-28 01:48:42 UTC
yum-3.2.1-1.fc7 has been pushed to the Fedora 7 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 6 Vladimir Mosgalin 2007-06-28 09:52:58 UTC
New yum behaves even worse, now it crashes at first available description and it
doesn't even matter if the output is redirected to file. In other words, "yum
info" stopped worked under any conditions.

$ yum info
Loading "installonlyn" plugin
Loading "fedorakmod" plugin
Installed Packages
Name   : ConsoleKit
Arch   : x86_64
Version: 0.2.1
Release: 2.fc7
Size   : 118 k
Repo   : installed
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 94, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 266, in doCommands
    return self.yum_cli_commands[self.basecmd].doCommand(self, self.basecmd,
self.extcmds)
  File "/usr/share/yum-cli/yumcommands.py", line 164, in doCommand
    base.listPkgs(ypl.installed, 'Installed Packages', basecmd)
  File "/usr/share/yum-cli/output.py", line 102, in listPkgs
    self.infoOutput(pkg)
  File "/usr/share/yum-cli/output.py", line 72, in infoOutput
    print _("Summary: %s") % pkg.summary.encode("UTF-8")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal
not in range(128)

My guess is "ConsoleKit" description is in utf-8 encoded string already. Trying
to "encode" it to utf-8 is meaningless operation, only unicode string can be
"encoded".

At least some hack like

if isinstance(pkg.summary, unicode):
 print _("Summary: %s") % pkg.summary.encode("UTF-8")
else:
 print _("Summary: %s") % pkg.summary # assuming it's in utf-8, not some other
obscure encoding

is needed, but the problem should be fought at its roots, so that there is a
guarantee that pkg.summary is always a unicode string.

Comment 7 Sjoerd Mullender 2007-06-28 10:26:40 UTC
A simple way to reproduce the new problem is

yum info checkpolicy

The problem is that the first line of the description contains the Unicode
equivalent of (R) which is presumably what causes the crash.

Comment 8 Jeremy Katz 2007-06-28 14:43:10 UTC
Original case is fixed here, the other case is being tracked with bug 245445

Comment 9 Allan Engelhardt 2007-07-14 19:11:41 UTC
This is still an issue with yum-3.2.1-1.fc7 and the ticket should be re-opened.
 Also reported as Bug 248078 .

On Fedora 7 x86_64:

# rpm -q yum
yum-3.2.1-1.fc7
# yum info cups.x86_64
Loading "fastestmirror" plugin
Loading "kernel-module" plugin
Loading "installonlyn" plugin
Loading mirror speeds from cached hostfile
livna                     100% |=========================| 2.1 kB    00:00     
fedora                    100% |=========================| 2.1 kB    00:00     
updates-debuginfo         100% |=========================| 1.9 kB    00:00     
fedora-debuginfo          100% |=========================| 1.9 kB    00:00     
updates                   100% |=========================| 1.9 kB    00:00     
moertel-community         100% |=========================|  951 B    00:00     
Installed Packages
Name   : cups
Arch   : x86_64
Epoch  : 1
Version: 1.2.11
Release: 2.fc7
Size   : 8.8 M
Repo   : installed
Summary: Common Unix Printing System

Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 94, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 266, in doCommands
    return self.yum_cli_commands[self.basecmd].doCommand(self, self.basecmd,
self.extcmds)
  File "/usr/share/yum-cli/yumcommands.py", line 164, in doCommand
    base.listPkgs(ypl.installed, 'Installed Packages', basecmd)
  File "/usr/share/yum-cli/output.py", line 102, in listPkgs
    self.infoOutput(pkg)
  File "/usr/share/yum-cli/output.py", line 73, in infoOutput
    print _("Description:\n%s") % pkg.description.encode("UTF-8")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 75: ordinal
not in range(128)

No updates in testing:

# yum --enablerepo=updates-testing update yum
Loading "fastestmirror" plugin
Loading "kernel-module" plugin
Loading "installonlyn" plugin
Loading mirror speeds from cached hostfile
Setting up Update Process
Could not find update match for yum
No Packages marked for Update/Obsoletion


Comment 10 Allan Engelhardt 2007-07-14 21:04:22 UTC
(Proposed patch attached to Bug 248078)

Comment 11 Seth Vidal 2007-07-24 20:20:46 UTC
*** Bug 249462 has been marked as a duplicate of this bug. ***