Bug 854959 - Exception handling in Anaconda doesn't work if dump contains non-ascii characters
Summary: Exception handling in Anaconda doesn't work if dump contains non-ascii charac...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-meh
Version: 18
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Vratislav Podzimek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedNTH RejectedBlocker
: 859163 868373 872599 (view as bug list)
Depends On:
Blocks: F18Beta-accepted, F18BetaFreezeExcept
TreeView+ depends on / blocked
 
Reported: 2012-09-06 11:49 UTC by Vratislav Podzimek
Modified: 2012-11-02 19:29 UTC (History)
12 users (show)

Fixed In Version: python-meh-0.17-1.fc18
Doc Type: Bug Fix
Doc Text:
Clone Of: 854688
Environment:
Last Closed: 2012-11-01 20:31:39 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Vratislav Podzimek 2012-09-06 11:49:18 UTC
+++ This bug was initially created as a clone of Bug #854688 +++

Description of problem:
If you try to do Live installation (DVD not tested) using Czech language, installer hangs at "preparing to install". I don't see a traceback in logs, but Vratislav Podzimek says there is some storage traceback caused by wrong locale. Instead of using locale cs_CZ.utf8, anaconda incorrectly sets 'cs' locale and then something crashes (gettext?).

It probably also influences other languages, but with a quick search I haven't found them. But some languages crash sooner due to bug 853609. It could have the same cause - wrongly set locale.

Version-Release number of selected component (if applicable):
F18 Alpha TC5 Live GNOME

How reproducible:
always

Steps to Reproduce:
1. boot TC5 Live GNOME
2. select czech language
3. proceed with installation

--- Additional comment from kparal on 2012-09-05 10:50:14 EDT ---

Created attachment 610078 [details]
anaconda.log

--- Additional comment from kparal on 2012-09-05 10:50:18 EDT ---

Created attachment 610079 [details]
program.log

--- Additional comment from kparal on 2012-09-05 10:50:22 EDT ---

Created attachment 610080 [details]
storage.log

--- Additional comment from kparal on 2012-09-05 10:51:16 EDT ---

Let's discuss at blocker bug meeting.

--- Additional comment from clumens on 2012-09-05 11:21:58 EDT ---

Can you run liveinst from a terminal and see what the error messages look like?

--- Additional comment from kparal on 2012-09-05 12:05:34 EDT ---

Great call. I kinda assumed it would be somewhere in the logs (why it isn't?). Three tracebacks in the process!

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pyanaconda/exception.py", line 59, in run_handleException_on_idle
    obj)
  File "/usr/lib/python2.7/site-packages/meh/handler.py", line 101, in handleException
    fd.write(self.exnText)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfd' in position 1059: ordinal not in range(128)


Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pyanaconda/ui/gui/spokes/welcome.py", line 175, in on_selection_changed
    self.retranslate()
  File "/usr/lib/python2.7/site-packages/pyanaconda/ui/gui/spokes/welcome.py", line 256, in retranslate
    NormalSpoke.retranslate(self)
  File "/usr/lib/python2.7/site-packages/pyanaconda/ui/gui/__init__.py", line 364, in retranslate
    xlated = _(before)
  File "/usr/lib/python2.7/site-packages/pyanaconda/ui/gui/__init__.py", line 28, in <lambda>
    _ = lambda x: gettext.ldgettext("anaconda", x)
  File "/usr/lib/python2.7/gettext.py", line 543, in ldgettext
    return t.lgettext(message)
  File "/usr/lib/python2.7/gettext.py", line 368, in lgettext
    return tmsg.encode(locale.getpreferredencoding())
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u017e' in position 22: ordinal not in range(256)


Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pyanaconda/threads.py", line 87, in run
    threading.Thread.run(self, *args, **kwargs)
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/site-packages/pyanaconda/install.py", line 62, in doInstall
    turnOnFilesystems(storage)
  File "/usr/lib/python2.7/site-packages/pyanaconda/storage/__init__.py", line 196, in turnOnFilesystems
    storage.doIt()
  File "/usr/lib/python2.7/site-packages/pyanaconda/storage/__init__.py", line 359, in doIt
    self.devicetree.processActions()
  File "/usr/lib/python2.7/site-packages/pyanaconda/storage/devicetree.py", line 323, in processActions
    action.execute()
  File "/usr/lib/python2.7/site-packages/pyanaconda/storage/deviceaction.py", line 419, in execute
    msg = _("Creating %(type)s on %(device)s") % {"type": self.device.format.type, "device": self.device.path}
  File "/usr/lib/python2.7/site-packages/pyanaconda/storage/deviceaction.py", line 35, in <lambda>
    _ = lambda x: gettext.ldgettext("anaconda", x)
  File "/usr/lib/python2.7/gettext.py", line 543, in ldgettext
    return t.lgettext(message)
  File "/usr/lib/python2.7/gettext.py", line 368, in lgettext
    return tmsg.encode(locale.getpreferredencoding())
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u0159' in position 5: ordinal not in range(256)

--- Additional comment from kparal on 2012-09-05 12:06:13 EDT ---

Created attachment 610089 [details]
terminal output

--- Additional comment from kparal on 2012-09-05 12:07:55 EDT ---

So it is as Vratislav said - locale is set to incorrect value and because of that gettext uses latin1 to decode translations. That makes some characters appear incorrect and apparently it probably also causes the installation hang.

--- Additional comment from clumens on 2012-09-05 12:10:57 EDT ---

A traceback in exception handling (that's the first one you're seeing there) means exception handling itself is going to be all screwed up.  So, that's why you're not getting logs or a dialog.

--- Additional comment from awilliam on 2012-09-05 13:08:40 EDT ---

Discussed at 2012-09-05 blocker review meeting. We noted this is likely to be considered a dupe of 853609, but for now we'll leave them separate just in case things turn out to be more complex, and this is accepted as a blocker per criterion "The installer must boot and run on all primary architectures, with all system firmware types that are common on those architectures, from default live image, DVD, and boot.iso install media when written to an optical disc and when written to a USB stick with at least one of the officially supported methods" in the case of selecting certain languages.

Comment 1 Vratislav Podzimek 2012-09-06 11:56:27 UTC
As the following traceback shows:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pyanaconda/exception.py", line 59, in run_handleException_on_idle
    obj)
  File "/usr/lib/python2.7/site-packages/meh/handler.py", line 101, in handleException
    fd.write(self.exnText)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfd' in position 1059: ordinal not in range(128)

exception handling (and thus bug reporting) in Anaconda fails, if exception dump info contains non-ascii characters. I'm proposing this as a F18Alpha blocker, per this criteria:
> The installer must be able to report failures to Bugzilla and local disk, with appropriate information included

Patch has been sent to anaconda-patches-list.

Comment 2 Tim Flink 2012-09-07 17:05:17 UTC
Discussed at the 2012-09-06 Fedora 18 alpha go/no-go meeting. Rejected as a blocker for F18 alpha but accepted as NTH. While it is aconditional violation of the following F18 alpha release criterion:

The installer must be able to report failures to Bugzilla and local disk, with appropriate information included.

this only happens for non-ascii strings which doesn't seem to be easy enough to hit to warrant blocker status.

Comment 3 Fedora Update System 2012-09-11 11:48:53 UTC
python-meh-0.16-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-meh-0.16-1.fc18

Comment 4 Kamil Páral 2012-09-11 13:54:44 UTC
Vratislav, can you please add a reproducer how to make sure this is fixed? The Czech language bug in the initial screen was already fixed by a different patch, so I don't know how to test this.

Comment 5 Kamil Páral 2012-09-11 14:35:42 UTC
I at least tried to boot Alpha RC2 Live, update python-meh and perform installation. Everything worked OK, so there hopefully isn't any serious regression.

Comment 6 Fedora Update System 2012-09-11 19:11:17 UTC
Package python-meh-0.16-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-meh-0.16-1.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-13802/python-meh-0.16-1.fc18
then log in and leave karma (feedback).

Comment 7 Vratislav Podzimek 2012-09-12 12:14:20 UTC
(In reply to comment #4)
> Vratislav, can you please add a reproducer how to make sure this is fixed?
> The Czech language bug in the initial screen was already fixed by a
> different patch, so I don't know how to test this.
You can download and install python-meh-0.16-1 RPM from [1],
install it and run simple testing script [2]

[1] http://koji.fedoraproject.org/koji/buildinfo?buildID=353634
[2] http://vpodzime.fedorapeople.org/meh_test.py

Comment 8 Kamil Páral 2012-09-12 13:10:07 UTC
If I run the script with the updated package, I receive anaconda-style error window "Unknown error has occured". Is that correct?

Comment 9 Vratislav Podzimek 2012-09-12 14:07:01 UTC
(In reply to comment #8)
> If I run the script with the updated package, I receive anaconda-style error
> window "Unknown error has occured". Is that correct?
Yes, this means, it has successfully over the problematic part of dumping non-ascii characters.

Comment 10 Kamil Páral 2012-09-12 14:20:41 UTC
Verified then.

Comment 11 Fedora Update System 2012-09-17 22:40:55 UTC
python-meh-0.16-1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 12 Vratislav Podzimek 2012-09-21 08:40:54 UTC
Unfortunately, it appears, that there are cases where changes made in python-meh-0.16-1 don't work. Reopening this bug.

Comment 13 Vratislav Podzimek 2012-09-21 08:41:18 UTC
*** Bug 859163 has been marked as a duplicate of this bug. ***

Comment 14 Vratislav Podzimek 2012-09-21 09:59:34 UTC
The original patch fixed the case when there was some unicode string (with Python's unicode type), but didn't fix the case when there was unicode string encoded and stored as common string.
New patch posted to anaconda-patches list.

Comment 15 Adam Williamson 2012-09-26 19:44:07 UTC
Discussed at 2012-09-26 blocker review meeting: http://meetbot.fedoraproject.org/fedora-qa/2012-09-26/f18-beta-blocker-review-1.2012-09-26-16.03.log.txt . Especially since this is partly fixed already, we think it's still not significant enough to make blocker status even for Beta, but we accept it as NTH. (it's a partial violation of the "The installer must be able to report failures to Bugzilla and local disk, with appropriate information included" criterion, but we don't think the infringing cases are significant enough to qualify).

vratislav, has the fix made its way into 18.9?

Comment 16 Vratislav Podzimek 2012-09-27 08:18:11 UTC
(In reply to comment #15)
> Discussed at 2012-09-26 blocker review meeting:
> http://meetbot.fedoraproject.org/fedora-qa/2012-09-26/f18-beta-blocker-
> review-1.2012-09-26-16.03.log.txt . Especially since this is partly fixed
> already, we think it's still not significant enough to make blocker status
> even for Beta, but we accept it as NTH. (it's a partial violation of the
> "The installer must be able to report failures to Bugzilla and local disk,
> with appropriate information included" criterion, but we don't think the
> infringing cases are significant enough to qualify).
> 
> vratislav, has the fix made its way into 18.9?
Unfortunately not, it still hangs on the anaconda-patches. I'll try to urge somebody from the team to do the review.

Comment 17 Fedora Update System 2012-10-09 16:40:17 UTC
python-meh-0.17-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-meh-0.17-1.fc18

Comment 18 Fedora Update System 2012-10-10 05:04:31 UTC
Package python-meh-0.17-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-meh-0.17-1.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-15784/python-meh-0.17-1.fc18
then log in and leave karma (feedback).

Comment 19 Vratislav Podzimek 2012-10-10 19:47:52 UTC
Updated script for testing available at:
http://vpodzime.fedorapeople.org/meh_test.py

If everything goes well, python-meh's "An unexpected error..." window appears and the text entry contains correct object dump with all non-ascii characters in both unicode_str and encoded_unicode_str

Comment 20 Kamil Páral 2012-10-11 08:50:34 UTC
(In reply to comment #19)
> Updated script for testing available at:
> http://vpodzime.fedorapeople.org/meh_test.py
> 
> If everything goes well, python-meh's "An unexpected error..." window
> appears and the text entry contains correct object dump with all non-ascii
> characters in both unicode_str and encoded_unicode_str

Yup, that works.

Comment 21 Vratislav Podzimek 2012-10-22 16:38:33 UTC
*** Bug 868373 has been marked as a duplicate of this bug. ***

Comment 22 Adam Williamson 2012-11-01 20:31:39 UTC
Update was pushed stable, closing.

Comment 23 Vratislav Podzimek 2012-11-02 19:29:46 UTC
*** Bug 872599 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.