Bug 1332012 - UnicodeEncodeError: 'ascii' codec can't encode characters in position 342-343: ordinal not in range(128)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 342-343...
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: dnf (Show other bugs)
23
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: packaging-team-maint
Fedora Extras Quality Assurance
:
: 1279204 1334023 1337215 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-05-01 03:48 EDT by Dov Grobgeld
Modified: 2016-05-23 07:20 EDT (History)
10 users (show)

See Also:
Fixed In Version: dnf-1.1.9-1.fc23
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-05-21 22:25:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Dov Grobgeld 2016-05-01 03:48:42 EDT
Description of problem:

Trying to do `dnf update fails with the error`:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 342-343: ordinal not in range(128)

Version-Release number of selected component (if applicable):

$ rpm -q -f /usr/lib/python3.4/site-packages/dnf/cli/output.py
python3-dnf-1.1.8-1.fc23.noarch

How reproducible:

It probably depends on the exact packages that I have installed.

Steps to Reproduce:
1. ?? (Install a certain set of packages)
2. dnf update
3.

Actual results:

Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/dnf/yum/rpmtrans.py", line 431, in callback
    self._scriptStop(bytes, total, h);
  File "/usr/lib/python3.4/site-packages/dnf/yum/rpmtrans.py", line 573, in _scriptStop
    self._scriptout()
  File "/usr/lib/python3.4/site-packages/dnf/yum/rpmtrans.py", line 219, in _scriptout
    display.scriptout(msgs)
  File "/usr/lib/python3.4/site-packages/dnf/cli/output.py", line 2195, in scriptout
    sys.stdout.write(ucd(msgs))
  File "/usr/lib/python3.4/site-packages/dnf/i18n.py", line 44, in write
    self.stream.write(s)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 342-343: ordinal not in range(128)
FATAL ERROR: python callback ??? failed, aborting!


Expected results:

No crash.

Additional info:

The following patch solves the problem. I suggest that all stdout write statements be embedded in a similar encode/decode statement to prevent dnf crashing even if a package name or description contain international characters that are not compatible with output terminal. To retain the original behaviour perhaps add an additional try/catch statement with the original print() in the try part and the encode/decode asciification in the except part.

...lib/python3.4/site-packages/dnf/cli/output.py:2310:

< sys.stdout.write(ucd(msgs))
--
> sys.stdout.write(ucd(msgs).encode('ascii','ignore').decode('ascii'))
Comment 1 Igor Gnatenko 2016-05-02 07:41:06 EDT
Thanks for the report,

could you provide name of package & source where you downloaded it.. If you don't know which package exactly caused this error it should be written in transaction before crash.

Thanks!
Comment 2 Dov Grobgeld 2016-05-02 09:11:15 EDT
The last row before the crash was:

  Upgrading   : VirtualBox-5.0-5.0.20_106931_fedora22-1.x86_64           77/177 

My main concern, though, is not fixing the packages, but protect dnf from crashing due to non-compatible codecs.
Comment 3 Honza Silhan 2016-05-09 07:28:39 EDT
Fixed in the upstream

PR: https://github.com/rpm-software-management/dnf/pull/484
Comment 4 Honza Silhan 2016-05-09 07:31:03 EDT
*** Bug 1279204 has been marked as a duplicate of this bug. ***
Comment 5 Michal Luscon 2016-05-09 07:47:02 EDT
*** Bug 1334023 has been marked as a duplicate of this bug. ***
Comment 6 Dov Grobgeld 2016-05-09 08:24:48 EDT
Great that this was fixed! Note that pull request 484 fixes only one of multiple places in the dnf code that print to stdout that needs to be protected. Other places are:

.../dnf/i18n.py:44:  self.stream.write(s)
.../dnf/i18n.py:104:  print(ucstring, end='')

These are the cause of e.g. 1279204 1334023 and probably other bug reports.
Comment 7 Fedora Update System 2016-05-19 07:08:43 EDT
dnf-plugins-core-0.1.21-1.fc24 dnf-1.1.9-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-1efd7bc386
Comment 8 Fedora Update System 2016-05-19 07:10:45 EDT
dnf-1.1.9-1.fc23 dnf-plugins-core-0.1.21-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-bb69db975e
Comment 9 Fedora Update System 2016-05-20 21:33:21 EDT
dnf-1.1.9-1.fc24, dnf-plugins-core-0.1.21-1.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-1efd7bc386
Comment 10 Fedora Update System 2016-05-20 22:27:37 EDT
dnf-1.1.9-1.fc23, dnf-plugins-core-0.1.21-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-bb69db975e
Comment 11 Fedora Update System 2016-05-21 22:24:34 EDT
dnf-1.1.9-1.fc23, dnf-plugins-core-0.1.21-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
Comment 12 Michal Luscon 2016-05-23 07:20:10 EDT
*** Bug 1337215 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.