Bug 1332012 - UnicodeEncodeError: 'ascii' codec can't encode characters in position 342-343: ordinal not in range(128)
Summary: UnicodeEncodeError: 'ascii' codec can't encode characters in position 342-343...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 23
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1279204 1334023 1337215 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-01 07:48 UTC by Dov Grobgeld
Modified: 2016-05-23 11:20 UTC (History)
10 users (show)

Fixed In Version: dnf-1.1.9-1.fc23
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-05-22 02:25:09 UTC


Attachments (Terms of Use)

Description Dov Grobgeld 2016-05-01 07:48:42 UTC
Description of problem:

Trying to do `dnf update fails with the error`:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 342-343: ordinal not in range(128)

Version-Release number of selected component (if applicable):

$ rpm -q -f /usr/lib/python3.4/site-packages/dnf/cli/output.py
python3-dnf-1.1.8-1.fc23.noarch

How reproducible:

It probably depends on the exact packages that I have installed.

Steps to Reproduce:
1. ?? (Install a certain set of packages)
2. dnf update
3.

Actual results:

Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/dnf/yum/rpmtrans.py", line 431, in callback
    self._scriptStop(bytes, total, h);
  File "/usr/lib/python3.4/site-packages/dnf/yum/rpmtrans.py", line 573, in _scriptStop
    self._scriptout()
  File "/usr/lib/python3.4/site-packages/dnf/yum/rpmtrans.py", line 219, in _scriptout
    display.scriptout(msgs)
  File "/usr/lib/python3.4/site-packages/dnf/cli/output.py", line 2195, in scriptout
    sys.stdout.write(ucd(msgs))
  File "/usr/lib/python3.4/site-packages/dnf/i18n.py", line 44, in write
    self.stream.write(s)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 342-343: ordinal not in range(128)
FATAL ERROR: python callback ??? failed, aborting!


Expected results:

No crash.

Additional info:

The following patch solves the problem. I suggest that all stdout write statements be embedded in a similar encode/decode statement to prevent dnf crashing even if a package name or description contain international characters that are not compatible with output terminal. To retain the original behaviour perhaps add an additional try/catch statement with the original print() in the try part and the encode/decode asciification in the except part.

...lib/python3.4/site-packages/dnf/cli/output.py:2310:

< sys.stdout.write(ucd(msgs))
--
> sys.stdout.write(ucd(msgs).encode('ascii','ignore').decode('ascii'))

Comment 1 Igor Gnatenko 2016-05-02 11:41:06 UTC
Thanks for the report,

could you provide name of package & source where you downloaded it.. If you don't know which package exactly caused this error it should be written in transaction before crash.

Thanks!

Comment 2 Dov Grobgeld 2016-05-02 13:11:15 UTC
The last row before the crash was:

  Upgrading   : VirtualBox-5.0-5.0.20_106931_fedora22-1.x86_64           77/177 

My main concern, though, is not fixing the packages, but protect dnf from crashing due to non-compatible codecs.

Comment 3 Honza Silhan 2016-05-09 11:28:39 UTC
Fixed in the upstream

PR: https://github.com/rpm-software-management/dnf/pull/484

Comment 4 Honza Silhan 2016-05-09 11:31:03 UTC
*** Bug 1279204 has been marked as a duplicate of this bug. ***

Comment 5 Michal Luscon 2016-05-09 11:47:02 UTC
*** Bug 1334023 has been marked as a duplicate of this bug. ***

Comment 6 Dov Grobgeld 2016-05-09 12:24:48 UTC
Great that this was fixed! Note that pull request 484 fixes only one of multiple places in the dnf code that print to stdout that needs to be protected. Other places are:

.../dnf/i18n.py:44:  self.stream.write(s)
.../dnf/i18n.py:104:  print(ucstring, end='')

These are the cause of e.g. 1279204 1334023 and probably other bug reports.

Comment 7 Fedora Update System 2016-05-19 11:08:43 UTC
dnf-plugins-core-0.1.21-1.fc24 dnf-1.1.9-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-1efd7bc386

Comment 8 Fedora Update System 2016-05-19 11:10:45 UTC
dnf-1.1.9-1.fc23 dnf-plugins-core-0.1.21-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-bb69db975e

Comment 9 Fedora Update System 2016-05-21 01:33:21 UTC
dnf-1.1.9-1.fc24, dnf-plugins-core-0.1.21-1.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-1efd7bc386

Comment 10 Fedora Update System 2016-05-21 02:27:37 UTC
dnf-1.1.9-1.fc23, dnf-plugins-core-0.1.21-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-bb69db975e

Comment 11 Fedora Update System 2016-05-22 02:24:34 UTC
dnf-1.1.9-1.fc23, dnf-plugins-core-0.1.21-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 12 Michal Luscon 2016-05-23 11:20:10 UTC
*** Bug 1337215 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.