Bug 198971 - case conversion not locale safe in logging library
Summary: case conversion not locale safe in logging library
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python
Version: rawhide
Hardware: All
OS: Linux
medium
high
Target Milestone: ---
Assignee: Mihai Ibanescu
QA Contact: Brock Organ
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2006-07-15 06:53 UTC by Sertaç Ö. Yıldız
Modified: 2007-11-30 22:11 UTC (History)
1 user (show)

Fixed In Version: 2.4.3-14
Clone Of:
Environment:
Last Closed: 2006-07-21 12:50:43 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Test case for the broken SysLog handler (388 bytes, text/x-python)
2006-07-17 17:46 UTC, Mihai Ibanescu
no flags Details

Description Sertaç Ö. Yıldız 2006-07-15 06:53:30 UTC
Description of problem:
In turkish locale, lowercase of I is ı, not i (which is lowercase of İ).
Using string.lower() is not safe in turkish locale, as the following traceback
demonstrates:

Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: zip                          ######################### [1/8]
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in ?
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 180, in main
    base.doTransaction()
  File "/usr/share/yum-cli/cli.py", line 680, in doTransaction
    self.runTransaction(cb=cb)
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 435, in
runTransaction
    errors = self.ts.run(cb.callback, '')
  File "/usr/share/yum-cli/callback.py", line 143, in callback
    self.logger.info(msg)
  File "/usr/lib/python2.4/logging/__init__.py", line 969, in info
    apply(self._log, (INFO, msg, args), kwargs)
  File "/usr/lib/python2.4/logging/__init__.py", line 1079, in _log
    self.handle(record)
  File "/usr/lib/python2.4/logging/__init__.py", line 1089, in handle
    self.callHandlers(record)
  File "/usr/lib/python2.4/logging/__init__.py", line 1126, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python2.4/logging/__init__.py", line 642, in handle
    self.emit(record)
  File "/usr/lib/python2.4/logging/handlers.py", line 648, in emit
    string.lower(record.levelname)),
  File "/usr/lib/python2.4/logging/handlers.py", line 623, in encodePriority
    priority = self.priority_names[priority]
KeyError: 'Info'


Version-Release number of selected component (if applicable):
python-2.4.3-11.FC6
yum-2.9.3-1


How reproducible:
always

Steps to Reproduce:
1. set locale to tr_TR.UTF-8
2. use yum to install/update packages
3.
  
Actual results:
traceback above

Expected results:


Additional info:
Trying to update core system packages (like libc) breaks the system seriously,
hence the high severity.

Comment 1 Mihai Ibanescu 2006-07-17 03:02:37 UTC
I looked a bit at the code, but I couldn't figure out where the problem is. At
first glance, it appears that yum is trying to translate something that it
shouldn't. If you can pinpoint the problem before I get to it, and it turns out
to be in python, I'll be happy to report it upstream and/or patch python.

Comment 2 Mihai Ibanescu 2006-07-17 03:45:57 UTC
I can't quite see why the string would be in Turkish to begin with, it should be
a  simple ASCII string from the internal python module.

Comment 3 Sertaç Ö. Yıldız 2006-07-17 05:45:44 UTC
The problem is that the string "INFO" is passed to string.lower() and assumed 
to be equal to "info" and passed as a key to dict. Try this simple test case:

import string, locale
s= "INFO"
print string.lower(s),
locale.setlocale(locale.LC_ALL, '')
print string.lower(s)

The output is "info info" in most locale settings, but "info Info" in some
unlucky ones.

Comment 4 Mihai Ibanescu 2006-07-17 17:46:12 UTC
Created attachment 132561 [details]
Test case for the broken SysLog handler

This works:
python /tmp/logging-broken.py

This doesn't:
LANG=tr_TR.UTF-8 python /tmp/logging-broken.py

Comment 6 Mihai Ibanescu 2006-07-18 14:43:16 UTC
FWIW, u"INFO".lower() == "info" even in the Turkish locale, maybe that's the way
to force a comparison using the C locale after you've changed it to something else.

This link has some suggestions for what NOT to do:

http://docs.python.org/lib/node323.html

Comment 7 Mihai Ibanescu 2006-07-21 12:50:43 UTC
Fixed upstream in subversion transaction 50740

http://svn.python.org/view/python/trunk/Lib/logging/handlers.py?rev=50740&r1=47121&r2=50740&makepatch=1&diff_format=u

Fix added to python 2.4.3, should appear in 2.4.3-14


Note You need to log in before you can comment on or make changes to this bug.