Bug 1308994

Summary: After installing a package from local unicode directory a traceback is shown (UnicodeEncodeError)
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: dnfAssignee: Jaroslav Mracek <jmracek>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: low    
Version: 23CC: jmracek, jsilhan, mluscon, mmraka, packaging-team-maint, pnemade, vmukhame
Target Milestone: ---Keywords: i18n, Reopened, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: dnf-1.1.10-1.fc24 dnf-1.1.10-1.fc23 dnf-1.1.10-1.fc25 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1368340 (view as bug list) Environment:
Last Closed: 2016-08-19 19:51:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1368340    
Attachments:
Description Flags
Full terminal output and traceback none

Description Miro Hrončok 2016-02-16 16:44:11 UTC 
Created attachment 1127634 [details]
Full terminal output and traceback

Description of problem:
I have my locales set to cs_CZ.utf8. When I download some files, they end up in ~/Stažené which means ~/Downloads

When I install downloaded packages, I do something like:

    # dnf install ./Stažené/nosync-1.0-3.fc23.i686.rpm

The installation works fine, but after the installation, a traceback is shown.

Version-Release number of selected component (if applicable):
dnf-1.1.6-2.fc23.noarch

How reproducible:
Only when some dependent packages are installed as well.

Steps to Reproduce:
1. Download an rpm, I've used i686 of nosync http://koji.fedoraproject.org/koji/buildinfo?buildID=724570
2. Place that RPM to ~/Stažené
3. Make sure dependencies are not already satisfied (e.g. for nosync make sure glibc.i686 is missing)
4. dnf install ./Stažené/nosync-1.0-3.fc23.i686.rpm

Actual results:
Package installs fine, brings some deps.
Traceback is shown after "You can remove cached packages by executing 'dnf clean packages'." message is shown.


Expected results:
Package installs fine, brings some deps.
No tracebacks.

Additional info:

Full terminal output and traceback attached.

You are writing Unicode data (the command I put on command line) to this object:

    <_io.TextIOWrapper name='/var/lib/dnf/yumdb/n/[longhash]-nosync-1.0-3.fc23-i686/command_line.tmp' mode='w' encoding='ANSI_X3.4-1968'>

And it blows up. I was unable to reproduce this from Python 3 shell, because if I do (as you do):

    fo = dnf.yum.rpmsack._open_no_umask(fn + '.tmp', 'w')

fo is <_io.TextIOWrapper name='...tmp' mode='w' encoding='UTF-8'>

At this point, my investigation ends. No idea, why your _io.TextIOWrappers are ANSI_X3.4-1968.
Comment 1 Honza Silhan 2016-02-22 12:33:13 UTC 
thanks for the report. We'll fix this.
Comment 2 Miro Hrončok 2016-02-25 14:31:03 UTC 
*** Bug 1312021 has been marked as a duplicate of this bug. ***
Comment 3 Fedora Admin XMLRPC Client 2016-07-08 09:39:26 UTC 
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 4 Jaroslav Mracek 2016-07-28 08:05:52 UTC 
The patch is here and is waiting for review: https://github.com/rpm-software-management/dnf/pull/550
Comment 5 Igor Gnatenko 2016-07-29 14:56:43 UTC 
Merged as part of DNF 2.0 release.
Comment 6 Michael Mráka 2016-08-02 11:30:27 UTC 
# ./bin/dnf-2 install ekiga
Dependencies resolved.
================================================================================
 Package                   Arch     Version                     Repository
                                                                           Size
================================================================================
Installing:
 ekiga                     x86_64   4.0.1-23.fc23               fedora    8.1 M
Installing dependencies:
...
Is this ok [y/N]: y
Downloading Packages:
...
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Traceback (most recent call last):
  File "./bin/dnf-2", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/root/dnf.git/dnf/cli/main.py", line 170, in user_main
    errcode = main(args)
  File "/root/dnf.git/dnf/cli/main.py", line 60, in main
    return _main(base, args)
  File "/root/dnf.git/dnf/cli/main.py", line 119, in _main
    ret = resolving(cli, base)
  File "/root/dnf.git/dnf/cli/main.py", line 146, in resolving
    base.do_transaction(display=displays)
  File "/root/dnf.git/dnf/cli/cli.py", line 221, in do_transaction
    super(BaseCli, self).do_transaction(display)
  File "/root/dnf.git/dnf/base.py", line 639, in do_transaction
    self._run_transaction(cb=cb)
  File "/root/dnf.git/dnf/base.py", line 685, in _run_transaction
    rpmdbv = self.sack._rpmdb_version(self._yumdb)
  File "/root/dnf.git/dnf/sack.py", line 90, in _rpmdb_version
    if 'checksum_type' in ydbi and 'checksum_data' in ydbi:
  File "/root/dnf.git/dnf/yum/rpmsack.py", line 311, in __contains__
    x = self.get(attr)
  File "/root/dnf.git/dnf/yum/rpmsack.py", line 338, in get
    res = self._read(attr)
  File "/root/dnf.git/dnf/yum/rpmsack.py", line 265, in _read
    fo, e = _iopen(fn)
  File "/root/dnf.git/dnf/yum/rpmsack.py", line 59, in _iopen
    ret = open(*args, encoding='utf-8')
TypeError: 'encoding' is an invalid keyword argument for this function

# git log dnf/yum/rpmsack.py
commit 63ab30c520d9fb7d83496f3616e454c363d4bacc
Author: Jaroslav Mracek <jmracek>
Date:   Thu Jul 28 09:54:42 2016 +0200

    Fix handling with unicode directory (RhBug:1308994)
    
    It fixes handling of local rpm from unicode directory.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1308994
    
    Closes: #550
    Approved by: ignatenkobrain
Comment 7 Igor Gnatenko 2016-08-02 11:34:28 UTC 
Uh oh, py2.
Comment 8 Jaroslav Mracek 2016-08-02 12:06:57 UTC 
I will make fix for fix. Sorry forget py2.
Comment 9 Jaroslav Mracek 2016-08-02 13:32:19 UTC 
I create new pull-request that hopefully fix the problem: https://github.com/rpm-software-management/dnf/pull/555
Comment 10 Miro Hrončok 2016-08-02 22:39:44 UTC 
Also, this was reported against Fedora 23. Could you please backport the fix?
Comment 11 Fedora Update System 2016-08-18 15:19:41 UTC 
dnf-1.1.10-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-267f843cda
Comment 12 Fedora Update System 2016-08-18 15:20:53 UTC 
dnf-1.1.10-1.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-8caa619a9e
Comment 13 Fedora Update System 2016-08-18 15:21:54 UTC 
dnf-1.1.10-1.fc24 dnf-plugins-core-0.1.21-3.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-878284e0d3
Comment 14 Fedora Update System 2016-08-18 16:53:06 UTC 
dnf-1.1.10-1.fc25, dnf-plugins-core-0.1.21-4.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-8caa619a9e
Comment 15 Fedora Update System 2016-08-19 00:22:50 UTC 
dnf-1.1.10-1.fc23, dnf-plugins-core-0.1.21-3.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-267f843cda
Comment 16 Fedora Update System 2016-08-19 00:58:01 UTC 
dnf-1.1.10-1.fc24, dnf-plugins-core-0.1.21-3.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-878284e0d3
Comment 17 Fedora Update System 2016-08-19 19:50:56 UTC 
dnf-1.1.10-1.fc24, dnf-plugins-core-0.1.21-3.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.
Comment 18 Fedora Update System 2016-08-23 15:20:04 UTC 
dnf-1.1.10-1.fc23, dnf-plugins-core-0.1.21-3.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
Comment 19 Fedora Update System 2016-08-27 10:21:06 UTC 
dnf-1.1.10-1.fc25, dnf-plugins-core-0.1.21-4.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.