Bug 1234967

Summary: yum list installed: UnicodeDecodeError
Product: Red Hat Enterprise Linux 7 Reporter: Zack Cerza <zcerza>
Component: yumAssignee: Michal Domonkos <mdomonko>
Status: CLOSED ERRATA QA Contact: Eva Mrakova <emrakova>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 7.1CC: emrakova, james.antill, ksrot, mdomonko, vakulkar, vmukhame, zcerza
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: yum-3.4.3-138.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1337054 (view as bug list) Environment:
Last Closed: 2016-11-04 05:28:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1295396    
Attachments:
Description Flags
patch none

Description Zack Cerza 2015-06-23 15:27:42 UTC
Description of problem:
I'm seeing lots of yum crashes during Ceph testing.


Version-Release number of selected component (if applicable):
yum-3.4.3-125.el7.noarch


[ubuntu@magna117 ~]$ sudo yum list installed
Loaded plugins: priorities, product-id, subscription-manager
Installed Packages
NetworkManager.x86_64            1:1.0.0-14.git20150121.b4ea599c.el7
                                                             @anaconda/7.1
NetworkManager-config-server.x86_64
                                 1:1.0.0-14.git20150121.b4ea599c.el7
                                                             @anaconda/7.1
NetworkManager-libnm.x86_64      1:1.0.0-14.git20150121.b4ea599c.el7
                                                             @anaconda/7.1
NetworkManager-team.x86_64       1:1.0.0-14.git20150121.b4ea599c.el7
                                                             @anaconda/7.1
NetworkManager-tui.x86_64        1:1.0.0-14.git20150121.b4ea599c.el7
                                                             @anaconda/7.1
Red_Hat_Enterprise_Linux-Release_Notes-7-en-US.noarch
                                 1.0-9.el7                   @anaconda/7.1
SDL.x86_64                       1.2.15-11.el7               @rhel-7-server-rpms
acl.x86_64                       2.2.51-12.el7               @anaconda/7.1
agg.x86_64                       2.5-18.el7                  @rhel-7-server-rpms
aic94xx-firmware.noarch          30-6.el7                    @anaconda/7.1
alsa-firmware.noarch             1.0.28-2.el7                @anaconda/7.1
alsa-lib.x86_64                  1.0.28-2.el7                @anaconda/7.1
alsa-tools-firmware.x86_64       1.0.27-4.el7                @anaconda/7.1
ant.noarch                       1.9.2-9.el7                 @rhel-7-server-rpms
apr.x86_64                       1.4.8-3.el7                 @rhel-7-server-rpms
apr-devel.x86_64                 1.4.8-3.el7                 @rhel-7-server-rpms
apr-util.x86_64                  1.5.2-6.el7                 @rhel-7-server-rpms
apr-util-devel.x86_64            1.5.2-6.el7                 @rhel-7-server-rpms
at.x86_64                        3.1.13-17.el7_0.1           @rhel-7-server-rpms
atk.x86_64                       2.8.0-4.el7                 @rhel-7-server-rpms
atlas.x86_64                     3.10.1-7.el7                @rhel-7-server-rpms
attr.x86_64                      2.4.46-12.el7               @rhel-7-server-rpms
audit.x86_64                     2.4.1-5.el7                 @anaconda/7.1
audit-libs.x86_64                2.4.1-5.el7                 @anaconda/7.1
authconfig.x86_64                6.2.8-9.el7                 @anaconda/7.1
autoconf.noarch                  2.69-11.el7                 @rhel-7-server-rpms
autogen-libopts.x86_64           5.18-5.el7                  @rhel-7-server-rpms
automake.noarch                  1.13.4-3.el7                @rhel-7-server-rpms
avahi-autoipd.x86_64             0.6.31-14.el7               @anaconda/7.1
avahi-libs.x86_64                0.6.31-14.el7               @anaconda/7.1
basesystem.noarch                10.0-7.el7                  @anaconda/7.1
bash.x86_64                      4.2.46-12.el7               @anaconda/7.1
bc.x86_64                        1.06.95-13.el7              @rhel-7-server-rpms
bind-libs-lite.x86_64            32:9.9.4-18.el7             @anaconda/7.1
bind-license.noarch              32:9.9.4-18.el7             @anaconda/7.1
binutils.x86_64                  2.23.52.0.1-30.el7          @anaconda/7.1
biosdevname.x86_64               0.6.1-2.el7                 @anaconda/7.1
bison.x86_64                     2.7-4.el7                   @rhel-7-server-rpms
blas.x86_64                      3.4.2-4.el7                 @rhel-7-server-rpms
blktrace.x86_64                  1.0.5-6.el7                 @rhel-7-server-rpms
boost-program-options.x86_64     1.53.0-23.el7               @rhel-7-server-rpms
boost-system.x86_64              1.53.0-23.el7               @rhel-7-server-rpms
boost-thread.x86_64              1.53.0-23.el7               @rhel-7-server-rpms
btrfs-progs.x86_64               3.16.2-1.el7                @anaconda/7.1
byacc.x86_64                     1.9.20130304-3.el7          @rhel-7-server-rpms
bzip2.x86_64                     1.0.6-12.el7                @rhel-7-server-rpms
bzip2-libs.x86_64                1.0.6-12.el7                @anaconda/7.1
ca-certificates.noarch           2014.1.98-72.el7            @anaconda/7.1
cairo.x86_64                     1.12.14-6.el7               @rhel-7-server-rpms
celt051.x86_64                   0.5.1.3-8.el7               @rhel-7-server-rpms
Traceback (most recent call last):
  File "/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 367, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 174, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 571, in doCommands
    return self.yum_cli_commands[self.basecmd].doCommand(self, self.basecmd, self.extcmds)
  File "/usr/share/yum-cli/yumcommands.py", line 693, in doCommand
    '=' : clir, 'not in' : clie})
  File "/usr/share/yum-cli/output.py", line 954, in listPkgs
    highlight=highlight, columns=columns)
  File "/usr/share/yum-cli/output.py", line 729, in simpleList
    print self.fmtColumns(columns, text_width=len)
  File "/usr/share/yum-cli/output.py", line 706, in fmtColumns
    return msg % tuple(data)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 2: ordinal not in range(128)

Comment 2 Vasu Kulkarni 2015-06-23 20:51:43 UTC
Please provide a fix or workaround for this issue, this is causing lot of automated Ceph test run failures and I am randomly hitting this issue very often.

Comment 3 Valentina Mukhamedzhanova 2015-06-25 13:28:18 UTC
Could you please provide a reproducer?

Comment 4 Valentina Mukhamedzhanova 2015-06-25 13:32:27 UTC
Also the output of the following commands would be helpful:

# rpm -qa
# yum-config-manager

If you don't have yum-config-manager installed, it's provided by yum-utils package.

Comment 5 Zack Cerza 2015-06-25 15:17:18 UTC
Unfortunately, I don't have a reproducer. Since yum is crashing while trying to print a package name, I don't know which one it's failing on.

rpm -qa:
http://pastebin.test.redhat.com/292799

yum-config-manager:
http://pastebin.test.redhat.com/292801

Comment 6 Karel Srot 2015-06-25 15:29:34 UTC
Maybe it is
ceph-debuginfo-0.80.9-255.gd0f9c5f.el7
which is next in the list but I am not able to find this particular rpm.

Comment 7 Karel Srot 2015-06-25 15:33:00 UTC
maybe try listing packages one by one

for P in `rpm -qa`; do yum info $P || break; done

Comment 8 Zack Cerza 2015-06-25 17:08:48 UTC
I ran:
for P in `rpm -qa`; do echo "CHECKING $P"; yum info $P || break; done

It stopped here:
CHECKING gpg-pubkey-fd431d51-4ae0493b
Loaded plugins: priorities, product-id, subscription-manager
Repo rhel-7-server-extras-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/4709113451064542264-key.pem
Repo rhel-7-server-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/4709113451064542264-key.pem
Repo rhel-7-server-optional-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/4709113451064542264-key.pem
Error: No matching Packages to list

The ceph-debuginfo package is from one of our Ceph upstream testing repos

Comment 9 Karel Srot 2015-06-25 18:19:02 UTC
OK, that didn't work wel.. My point was to find the particular package that is causing the problem. You would be probably able to come up with a better method. What about running
  ceph-debuginfo-0.80.9-255.gd0f9c5f.el7
directly?

> The ceph-debuginfo package is from one of our Ceph upstream testing repos

Probably, but as I said, I was not able to find this particular RPM.

Comment 10 Zack Cerza 2015-06-25 18:25:45 UTC
Looks like we have our culprit. But none of the repo names in /etc/yum.repos.d/*.repo contain invalid characters.

$ yum info ceph-debuginfo-0.80.9-255.gd0f9c5f.el7
Loaded plugins: priorities, product-id, subscription-manager
Repo rhel-7-server-extras-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/4709113451064542264-key.pem
Repo rhel-7-server-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/4709113451064542264-key.pem
Repo rhel-7-server-optional-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/4709113451064542264-key.pem
Installed Packages
Name        : ceph-debuginfo
Arch        : x86_64
Version     : 0.80.9
Release     : 255.gd0f9c5f.el7
Size        : 1.8 G
Repo        : installed
From repo   : ��B
Summary     : Debug information for package ceph
URL         : http://ceph.com/
License     : GPL-2.0
Description : This package provides debug information for package ceph.
            : Debug information is useful when developing applications that use
            : this package or when debugging this package.

Comment 11 Karel Srot 2015-06-25 18:52:28 UTC
This is strange. Yum doesn't let me to use repo with strange characters. I have no idea how such repo could be used.

Actually, I did a mistake and had typed 'yum info' instead of 'yum list'.
I guess that 'yum list ceph-debuginfo-0.80.9-255.gd0f9c5f.el7' results in traceback then.

Comment 12 Valentina Mukhamedzhanova 2015-06-29 10:07:00 UTC
Please list the output of
# yum list available --show-duplicates | grep ceph-debuginfo

Comment 13 Valentina Mukhamedzhanova 2015-06-29 10:09:55 UTC
You can probably work around the problem by setting 'from_repo' key to a valid value for ceph-debuginfo:
# yumdb set from_repo valid_repo_name ceph-debuginfo

Comment 14 Valentina Mukhamedzhanova 2015-06-29 10:13:38 UTC
(In reply to Karel Srot from comment #11)
> This is strange. Yum doesn't let me to use repo with strange characters. I
> have no idea how such repo could be used.

It's possible that the repo had a valid name, but yumdb data got corrupted for some reason.

Comment 16 Zack Cerza 2015-07-22 20:11:44 UTC
That workaround would probably help, but for now we're using rpm to remove ceph-debuginfo instead of yum.

Even so, yum crashing doesn't seem like a good thing at all.

Comment 19 Michal Domonkos 2016-02-01 15:50:07 UTC
Created attachment 1120167 [details]
patch

Comment 20 Michal Domonkos 2016-02-18 13:42:41 UTC
PR created upstream:
https://github.com/rpm-software-management/yum/pull/4

Comment 27 errata-xmlrpc 2016-11-04 05:28:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2397.html