Bug 694928

Summary: "import decimal" fails in Turkish locale with "KeyError: 'ROUND_CEiLiNG'"
Product: [Fedora] Fedora Reporter: M KÜRŞAT YALÇIN <mehyal>
Component: pythonAssignee: Dave Malcolm <dmalcolm>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rawhideCC: awilliam, dmalcolm, gholms, ivazqueznet, jonathan, jonathansteffan, mehyal, ozgur.erdgd, tflink, vanmeeuwen+fedora
Target Milestone: ---Keywords: CommonBugs
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard: AcceptedNTH anaconda_trace_hash:80eb1eb045206697dc3aad00eaea44141198d858efe6b8535f1ddf05f0d714b7 https://fedoraproject.org/wiki/Common_F15_bugs#turkish-partition
Fixed In Version: python-2.7.1-7.fc15 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-04-21 22:20:19 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 657621    
Description Flags
Attached traceback automatically from anaconda.
Attached traceback automatically from anaconda.
Attached traceback automatically from anaconda. none

Description M KÜRŞAT YALÇIN 2011-04-08 18:57:27 EDT
The following was filed automatically by anaconda:
anaconda 14.22 exception report
Traceback (most recent call first):
  File "/usr/lib/python2.7/decimal.py", line 3712, in <module>
    val = globals()[globalname]
  File "/usr/lib/python2.7/site-packages/pyanaconda/iw/partition_gui.py", line 37, in <module>
    from decimal import Decimal
  File "/usr/lib/python2.7/site-packages/pyanaconda/gui.py", line 1241, in setScreen
    loaded = imp.load_module(moduleName, *found)
  File "/usr/lib/python2.7/site-packages/pyanaconda/gui.py", line 1191, in nextClicked
    self.setScreen ()
KeyError: 'ROUND_CEiLiNG'
Comment 1 M KÜRŞAT YALÇIN 2011-04-08 18:57:43 EDT
Created attachment 490881 [details]
Attached traceback automatically from anaconda.
Comment 2 M KÜRŞAT YALÇIN 2011-04-08 18:59:37 EDT
Created attachment 490884 [details]
Attached traceback automatically from anaconda.
Comment 3 M KÜRŞAT YALÇIN 2011-04-08 19:02:49 EDT
Created attachment 490885 [details]
Attached traceback automatically from anaconda.
Comment 4 Dave Malcolm 2011-04-11 17:10:45 EDT
Note to self: LANG appears to be tr_TR.UTF-8

Code in question seems to be here:

  3709  # get rounding method function:
  3710  rounding_functions = [name for name in Decimal.__dict__.keys()
  3711                                      if name.startswith('_round_')]
  3712  for name in rounding_functions:
  3713      # name is like _round_half_even, goes to the global ROUND_HALF_EVEN value.
  3714      globalname = name[1:].upper()
  3715      val = globals()[globalname]
  3716      Decimal._pick_rounding_function[val] = name
  3718  del name, val, globalname, rounding_functions

from 2.7.1, although this has the "val = globals()[globalname]" at line 3715
Comment 5 Dave Malcolm 2011-04-11 17:12:24 EDT
  ['_round_ceiling', '_round_half_up', '_round_half_even', '_round_half_down', '_round_05up', '_round_floor', '_round_down', '_round_up']



Looks like "upper()" is doing something unexpected by the decimal.py module within this locale.
Comment 6 Dave Malcolm 2011-04-11 17:20:48 EDT
How does Anaconda actually set the locale?

I tried reproducing this using
 $ LANG=tr_TR.UTF-8 python -c 'print "round_ceiling".upper()'
in the environment, but wasn't able to reproduce.
Comment 7 Dave Malcolm 2011-04-11 18:11:34 EDT
Here's a minimal recipe for reproducing the string conversion that triggers this:

$ LC_ALL=tr_TR.UTF-8 python -c 'import locale; locale.setlocale(locale.LC_ALL, ""); print "round_ceiling".upper()'

Here's a minimal reproducer for the "import" failure:

$ LC_ALL=tr_TR.UTF-8 python -c 'import locale; locale.setlocale(locale.LC_ALL, ""); import decimal'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python2.7/decimal.py", line 3715, in <module>
    val = globals()[globalname]
KeyError: 'ROUND_CEiLiNG'
Comment 8 Dave Malcolm 2011-04-11 18:19:14 EDT
Or even shorter:
  python -c 'import locale; locale.setlocale(locale.LC_ALL, "tr_TR.UTF-8"); print "round_ceiling".upper()'

This has actually already been reported before, as bug 251491

Similar bug reported here for IronPython:
  http://lists.ironpython.com/pipermail/users-ironpython.com/2007-  October/005865.html

Other instances:
Comment 9 Dave Malcolm 2011-04-11 18:20:05 EDT
A nasty possible workaround:
  import decimal
within the entrypoint to anaconda, before calling locale.setlocale()
Comment 10 Dave Malcolm 2011-04-11 18:23:06 EDT
Also mentioned here:
The interpretation of "upper" and "lower" for unicode is here:
Comment 11 Dave Malcolm 2011-04-11 18:27:18 EDT
A robust fix for this for now may be to convert the str instances to unicode first, then call upper on them, then convert back to str.

This would work since upper() for a unicode instance is locale-independent as per:
(though there seems to have been some debate there).
Comment 12 Dave Malcolm 2011-04-11 18:33:56 EDT
Note to self: works with python 3, as the symbols are already unicode internally.
Comment 13 Dave Malcolm 2011-04-11 18:47:53 EDT
Reported upstream as http://bugs.python.org/issue11830
Comment 14 Dave Malcolm 2011-04-12 11:41:35 EDT
Fixed upstream in 2.7 branch.

Fix committed/pushed to "master" for F16:
  * Tue Apr 12 2011 David Malcolm <dmalcolm@redhat.com> - 2.7.1-7
  - fix "import decimal" in the Turkish locale (patch 131; rhbz#694928)

Working on fixing config snafu on my machine so I can build it in Koji
Comment 15 Dave Malcolm 2011-04-12 12:08:31 EDT
Building python-2.7.1-7.fc16 into master:

I also committed/pushed to "f15"; building python-2.7.1-7.fc15 there as:
Comment 16 Dave Malcolm 2011-04-13 17:18:43 EDT
From my reading of this,  any import of Anaconda's pyanaconda.iw.partition_gui module whilst in the Turkish locale will cause a failure of the installer.

This code is referenced by gui
    "partition" : ("partition_gui", "PartitionWindow"),
and it looks like doing so is mandatory for all installs.

pyanaconda/storage/size.py also imports the "decimal" module.

So it looks likely that this bug will block all installations in the Turkish locale.

Added to "F15Blocker" and marked with "CommonBugs".
Comment 17 Fedora Update System 2011-04-13 17:29:34 EDT
python-2.7.1-7.fc15 has been submitted as an update for Fedora 15.
Comment 18 Adam Williamson 2011-04-13 17:30:07 EDT
Tested for F15 Beta purposes: I confirm this bug exists in F15 Beta, but you only hit it if you go to the custom partitioning screen; if you don't use that screen (not using it is the click-straight-through case, btw) you dodge the bug.

Proposing as nice-to-have for final (it doesn't quite hit blocker criteria, IMO), we will document for Beta. The fix will certainly get in for final anyway if it's submitted now.
Comment 19 Fedora Update System 2011-04-13 20:35:45 EDT
Package python-2.7.1-7.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-2.7.1-7.fc15'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).
Comment 20 Fedora Update System 2011-04-21 22:20:13 EDT
python-2.7.1-7.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 21 Tim Flink 2011-04-25 12:11:16 EDT
Discussed at the 2011-04-15 blocker bug review meeting (updating the bug in case it is re-opened).

This causes problems in anaconda and possibly other packages but doesn't clearly hit any release criteria - accepted as F15 NTH.