Bug 1277895

Summary: [abrt] dnf: option_parser.py:255:get_usage:UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)
Product: [Fedora] Fedora Reporter: Frank Büttner <bugzilla>
Component: dnf-plugin-system-upgradeAssignee: Will Woods <wwoods>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: high    
Version: 21CC: alex.go4more, as.maps, bigant, bugzilla, bugzilla, covex, fedoraproject, jsilhan, marcotijuana, mluscon, packaging-team-maint, pnemade, res-1, vmukhame, wwoods, zbyszek
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
URL: https://retrace.fedoraproject.org/faf/reports/bthash/51e199a9a2c4ec35b4cbdc4eb9cb4c503ff6b7d0
Whiteboard: abrt_hash:d9f3e0bdda611798f15e1f5cb93f2a9c8b7112ad
Fixed In Version: dnf-plugin-system-upgrade-0.7.1-1.fc22 dnf-plugin-system-upgrade-0.7.1-1.fc23 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-10 03:51:25 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
File: backtrace
none
File: environ none

Description Frank Büttner 2015-11-04 10:21:32 UTC
Description of problem:
dnf system-upgrade download --releasever=22 --best

Version-Release number of selected component:
dnf-0.6.4-7.fc21

Additional info:
reporter:       libreport-2.3.0
cmdline:        /usr/bin/python -OO /bin/dnf migrate --help
executable:     /bin/dnf
kernel:         4.1.10-100.fc21.x86_64
runlevel:       N 5
type:           Python
uid:            0

Truncated backtrace:
option_parser.py:255:get_usage:UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)

Traceback (most recent call last):
  File "/bin/dnf", line 36, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python2.7/site-packages/dnf/cli/main.py", line 185, in user_main
    errcode = main(args)
  File "/usr/lib/python2.7/site-packages/dnf/cli/main.py", line 84, in main
    return _main(base, args)
  File "/usr/lib/python2.7/site-packages/dnf/cli/main.py", line 115, in _main
    cli.configure(map(ucd, args))
  File "/usr/lib/python2.7/site-packages/dnf/cli/cli.py", line 981, in configure
    self.optparser.usage = self.optparser.get_usage()
  File "/usr/lib/python2.7/site-packages/dnf/cli/option_parser.py", line 255, in get_usage
    usage += "%-25s %s\n" % (name, summary)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)

Local variables in innermost frame:
group: u'plugin'
name: u'system-upgrade'
grp: u'plugin'
self: OptionParser(prog='dnf', usage=None, description=None, version=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler=u'error', add_help=False)
summary: 'System f\xc3\xbcr die Aktualisierung auf eine neue Release-Version vorbereiten'
usage: u'dnf [options] COMMAND\n\nListe der Hauptbefehle\n\nautoremove                \ncheck-update              Auf verf\xfcgbare Paket-Aktualisierungen \xfcberpr\xfcfen\nclean                     Gespeicherte Daten entfernen\ndistro-sync               Installierte Pakete mit den neuesten verf\xfcgbaren\nVersionen abgleichen\ndowngrade                 Ein Paket zur\xfccksetzen\ngroup                     Gruppeninformation anzeigen oder verwenden\nhelp                      Eine kurze Verwendungsinformation anzeigen\nhistory                   \xdcbertragungsverlauf anzeigen oder verwenden\ninfo                      Details zu einem Paket oder einer Gruppe von Paketen\nanzeigen\ninstall                   Ein Paket oder Pakete auf Ihrem System installieren\nlist                      Pakete oder Paketgruppen auflisten\nmakecache                 Den Metadaten-Zwischenspeicher erzeugen\nprovides                  Ein Paket suchen, das den gegebenen Wert bereitstellt\nreinstall                 Paket neu installieren\nremove                    Ein Paket oder Pakete auf Ihrem System entfernen\nrepolist                  Die eingerichteten Paketquellen anzeigen\nrepository-packages       Befehle auf alle Pakete in einer angegebenen\nPaketquelle anwenden\nsearch                    Nach Paket-Details f\xfcr die gegebene\nZeichenkette suchen\nupdateinfo                Erkl\xe4rungen zu Paketen anzeigen\nupgrade                   Ein Paket oder Pakete auf Ihrem System aktualisieren\nupgrade-to                Ein Paket auf Ihrem System auf die angegebene Version aktualisieren\n\nListe der Plugin-Befehle\n\nbuilddep                  Install build dependencies for .src.rpm, .nosrc.rpm or .spec file\nconfig-manager            manage dnf configuration options and repositories\ncopr                      Interact with Copr repositories.\ndebuginfo-install         install debuginfo packages\ndownload                  Download package to current directory\nkickstart                 Install packages defined in a kickstart file on your system\nneeds-restarting          determine updated binaries that need restarting\nplayground                Interact with Playground repository.\nrepoquery                 search for packages matching keyword\nreposync                  download all packages from remote repo\n'
desc: {u'main': u'Liste der Hauptbefehle', u'plugin': u'Liste der Plugin-Befehle'}

Comment 1 Frank Büttner 2015-11-04 10:21:38 UTC
Created attachment 1089549 [details]
File: backtrace

Comment 2 Frank Büttner 2015-11-04 10:21:40 UTC
Created attachment 1089550 [details]
File: environ

Comment 3 Frank Büttner 2015-11-04 10:22:14 UTC
Unable to upgrade to F22

Comment 4 Frank Büttner 2015-11-04 10:23:29 UTC
Another user experienced a similar problem:

call fedup

reporter:       libreport-2.3.0
cmdline:        /usr/bin/python -OO /usr/bin/dnf system-upgrade
executable:     /usr/bin/dnf
kernel:         4.1.10-100.fc21.x86_64
package:        dnf-0.6.4-7.fc21
reason:         option_parser.py:255:get_usage:UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)
runlevel:       N 5
type:           Python
uid:            0

Comment 5 Frank Büttner 2015-11-04 13:10:36 UTC
After some testing it seems an general problem on F21.
An simple "dnf" without anything will crash.
But call it with "LANG=C dnf" it will work.
It looks for me, that the unicode part of it is broken.

Comment 6 Adam Pribyl 2015-11-05 08:33:49 UTC
This is a localization problem - please ask the maintainer of your language to fix the translation.

Comment 7 Frank Büttner 2015-11-05 08:38:13 UTC
Thank, can you tell me where are responsible for the German translation?

Comment 8 Adam Pribyl 2015-11-05 13:53:16 UTC
These pages should help:
https://fedoraproject.org/wiki/L10N_Teams
https://fedora.zanata.org/language/view/de

Comment 9 Frank Büttner 2015-11-05 17:09:36 UTC
Hi Roman,
can you take a look on it, because you one of the German translation team?
Thanks.

Comment 10 Frank Büttner 2015-11-06 19:43:07 UTC
I found out, that the last fedup update breaks it.
python2-dnf-plugin-system-upgrade-0.5.0-1.fc21.noarch
dnf-plugin-system-upgrade-0.5.0-1.fc21.noarch

Comment 11 Roman Spirgi 2015-11-06 21:11:36 UTC
Frank, we can't reproduce it in F23, the mentioned dnf command works and shows a help text:
$ dnf
Sie müssen irgendeinen Befehl eingeben
usage: dnf [options] COMMAND

Not sure how we could improve translations here.

Comment 12 Frank Büttner 2015-11-06 22:11:36 UTC
It only occurs on F21 after the replace of fedup by the both packages of #10.
So an update from F21 to F22 or F23 is now impossible, because dnf crash now.
After remove both dnf will work again.

Comment 13 Frank Büttner 2015-11-07 10:48:27 UTC
An test on an F21 VM with
LANG=C dnf system-upgrade download --releasever=22
LANG=C dnf system-upgrade reboot
Will put the system in an boot loop.
Boot it with sysremd.unit=rescue and look the the journal:
journalctl --since 2015-11-06|grep dnf
Will show the error of the initial report.
So there are something broken in the  "dnf-plugin-system-upgrade" module.

Comment 14 Andreas Schöneck 2015-11-09 10:14:30 UTC
I am on 
Description:	Fedora release 22 (Twenty Two)

And I get this when just calling (in order to get help)

$ sudo dnf system-upgrade

Traceback (most recent call last):
  File "/bin/dnf", line 36, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python2.7/site-packages/dnf/cli/main.py", line 198, in user_main
    errcode = main(args)
  File "/usr/lib/python2.7/site-packages/dnf/cli/main.py", line 84, in main
    return _main(base, args)
  File "/usr/lib/python2.7/site-packages/dnf/cli/main.py", line 117, in _main
    cli.configure(map(ucd, args))
  File "/usr/lib/python2.7/site-packages/dnf/cli/cli.py", line 1033, in configure
    self.command.configure(self.base.extcmds)
  File "/usr/lib/python2.7/site-packages/dnf-plugins/system_upgrade.py", line 414, in configure
    self.opts = self.parse_args(args)
  File "/usr/lib/python2.7/site-packages/dnf-plugins/system_upgrade.py", line 398, in parse_args
    dnf.cli.commands.err_mini_usage(self.cli, self.cli.base.basecmd)
  File "/usr/lib/python2.7/site-packages/dnf/cli/commands/__init__.py", line 52, in err_mini_usage
    txt = cli.cli_commands["help"]._makeOutput(cmd)
  File "/usr/lib/python2.7/site-packages/dnf/cli/commands/__init__.py", line 1020, in _makeOutput
    help_output += "\n\n%s" % summary
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)


This in fact seems to be a problem with the locales:

$ LANG=C sudo dnf system-upgrade
 Mini usage:

system-upgrade [OPTIONS] [download --releasever=VERSION|reboot|clean|log]

Prepare system for upgrade to a new release

alias: fedup

So one might be able to reproduce using

$ LANG=de_DE sudo dnf system-upgrade

Comment 15 Andreas Schöneck 2015-11-09 10:20:03 UTC
Have a look this GitHub issue for reference:
https://github.com/rpm-software-management/dnf-plugin-system-upgrade/issues/32

Comment 16 Honza Silhan 2015-11-09 14:43:54 UTC
This is a dup of bug 1262082. I don't wanna mark it as duplicate because this could be backported to f21. AFAIk for f21 there's only dnf-plugin-system-upgrade-0.5.

Comment 17 Jens Petersen 2015-11-12 02:23:03 UTC
(In reply to Frank Büttner from comment #13)
> An test on an F21 VM with
> LANG=C dnf system-upgrade download --releasever=22
> LANG=C dnf system-upgrade reboot
> Will put the system in an boot loop.

This happened to me from F22 too. :-(

I still haven't managed to rescue my machine. ;-(

After running "dnf system-upgrade clean" in the rescue env
it now boots to "Started firewalld" but just hangs there.
No network in the rescue environment.

Comment 18 Will Woods 2015-11-12 15:05:36 UTC
(In reply to Jan Silhan from comment #16)
> This is a dup of bug 1262082. I don't wanna mark it as duplicate because
> this could be backported to f21. 

Bug 1262082 is a dnf bug, though. Why reassign this to dnf-plugin-system-upgrade? What needs to be backported here?

> AFAIk for f21 there's only
> dnf-plugin-system-upgrade-0.5.

The only difference between 0.5.x and 0.7.x is that 0.5.x switches off plymouth and sends upgrade progress messages to stdout. All the i18n/unicode stuff is unchanged.

Comment 19 Frank Büttner 2015-11-14 11:16:29 UTC
As an dirty workaround, https://bugzilla.redhat.com/show_bug.cgi?id=1278031#c10
will work.

Comment 20 Will Woods 2015-11-16 16:28:09 UTC
I feel like this is probably the same root cause as bug 1278031 - jsilhan, does this mean that I need a patch like this to fix it?

 class SystemUpgradeCommand(dnf.cli.Command):
     # pylint: disable=unused-argument
     aliases = ('system-upgrade', 'fedup')
-    summary = _("Prepare system for upgrade to a new release")
+    summary = dnf.i18n.ucd(_("Prepare system for upgrade to a new release"))
     # NOTE: upgrade isn't meant to be invoked by users, so it's not in usage
     usage = "[%s] [download --releasever=%s|reboot|clean|log]" % (
         dnf.i18n.ucd(_("OPTIONS")), dnf.i18n.ucd(_("VERSION")))

Comment 21 Honza Silhan 2015-11-20 11:07:41 UTC
With your commit [1], this should be considered as fixed (with dnf-1.1.4 being installed) and could be closed.

[1] https://github.com/rpm-software-management/dnf-plugin-system-upgrade/commit/4bbbf1ec0c3f387249d30dd103ff4d002c17f574

Comment 22 Will Woods 2015-11-20 16:58:10 UTC
Got it. Will close this with the next dnf-plugin-system-upgrade build.

Comment 23 Fedora End Of Life 2015-12-02 15:49:59 UTC
Fedora 21 changed to end-of-life (EOL) status on 2015-12-01. Fedora 21 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 24 Fedora Update System 2015-12-07 20:39:17 UTC
dnf-plugin-system-upgrade-0.7.1-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-6b41346335

Comment 25 Fedora Update System 2015-12-08 22:58:24 UTC
dnf-plugin-system-upgrade-0.7.1-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update dnf-plugin-system-upgrade'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-45efbec1fe

Comment 26 Fedora Update System 2015-12-08 23:51:46 UTC
dnf-plugin-system-upgrade-0.7.1-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update dnf-plugin-system-upgrade'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-6b41346335

Comment 27 Fedora Update System 2015-12-10 03:51:15 UTC
dnf-plugin-system-upgrade-0.7.1-1.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

Comment 28 Rudolf E. Steiner 2015-12-10 09:35:28 UTC
Will there be also an update for Fedora 21?

Comment 29 Will Woods 2015-12-10 19:51:14 UTC
No. It's EOL, and we can't push updates for it anymore.

I may backport the changes and build packages anyway, but it's probably easier to just use the workaround listed in the Common Bugs page:

https://fedoraproject.org/wiki/Common_F23_bugs#System-upgrade_does_not_work_under_Chinese_and_Japanese_.28and_probably_some_other.29_locales

Comment 30 Alex. H. F. 2015-12-12 17:46:23 UTC
This is totally annoying.

You leave F21 users without a reasonable way to upgrade!

I am not sure, if I can trust FEDORA development any more.

IF I AM NOT ABLE TO UPGRADE MY 3 MACHINES RUNNING F21, I WILL SEARCH FOR OTHER DISTRIBUTION, WHERE RESPONSIBILITY HAS MORE VALUE!!!

Ridiculous...

Comment 31 Adam Pribyl 2015-12-12 18:48:07 UTC
I understand your frustration, but is it that hard to use 

LANG=C dnf system-upgrade download --releasever=22

instead just

dnf system-upgrade download --releasever=22

You may also improve:
https://fedoraproject.org/wiki/DNF_system_upgrade

Comment 32 Frank Büttner 2015-12-12 19:10:04 UTC
Don't use it!!
It will break your system.
First you have to modify the systemd script, or the system will hang in an boot loop. See comment #17.
Only comment #19 will do an correct update to F22 or F23.

Comment 33 Alex. H. F. 2015-12-12 19:58:15 UTC
I changed locale to EN.

Let's see if you will still keep an user, or stay alone playing development...

I'll keep you informed.

Comment 34 Roman Spirgi 2015-12-12 20:03:17 UTC
Just my 2 cents, on my F21 machine I did change the system language to English, processed the upgrade, switched language back, done.

Comment 35 Fedora Update System 2015-12-16 14:36:25 UTC
dnf-plugin-system-upgrade-0.7.1-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 36 Steve 2015-12-27 13:33:18 UTC
Please update the package also for F21.

Comment 37 Zbigniew Jędrzejewski-Szmek 2015-12-27 16:16:53 UTC
(In reply to Steve from comment #36)
> Please update the package also for F21.

F21 is EOL, we cannot release updates for F21 anymore.

Comment 38 Alex. H. F. 2015-12-27 19:22:30 UTC
Hi Steve,

No way. They like better to leave you looking for a solution at yourself.


What you must do, is to change the "locale" to EN.
After that you can upgrade to F22.

From this point, you will be back to their support...

I upgraded two machines this way.

NOW I AM UNSURE, IF I CAN RELY STILL ON QUALITY FOR OTHER LANGUAGES, AND SWITCH BACK TO MY LANGUAGE "DE"...

As by Fedora/redhat developmetn, I suppose I SHOULD NOT DO.