Bug 198971 - case conversion not locale safe in logging library
case conversion not locale safe in logging library
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: python (Show other bugs)
rawhide
All Linux
medium Severity high
: ---
: ---
Assigned To: Mihai Ibanescu
Brock Organ
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-07-15 02:53 EDT by Sertaç Ö. Yıldız
Modified: 2007-11-30 17:11 EST (History)
1 user (show)

See Also:
Fixed In Version: 2.4.3-14
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-07-21 08:50:43 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Test case for the broken SysLog handler (388 bytes, text/x-python)
2006-07-17 13:46 EDT, Mihai Ibanescu
no flags Details

  None (edit)
Description Sertaç Ö. Yıldız 2006-07-15 02:53:30 EDT
Description of problem:
In turkish locale, lowercase of I is ı, not i (which is lowercase of İ).
Using string.lower() is not safe in turkish locale, as the following traceback
demonstrates:

Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: zip                          ######################### [1/8]
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in ?
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 180, in main
    base.doTransaction()
  File "/usr/share/yum-cli/cli.py", line 680, in doTransaction
    self.runTransaction(cb=cb)
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 435, in
runTransaction
    errors = self.ts.run(cb.callback, '')
  File "/usr/share/yum-cli/callback.py", line 143, in callback
    self.logger.info(msg)
  File "/usr/lib/python2.4/logging/__init__.py", line 969, in info
    apply(self._log, (INFO, msg, args), kwargs)
  File "/usr/lib/python2.4/logging/__init__.py", line 1079, in _log
    self.handle(record)
  File "/usr/lib/python2.4/logging/__init__.py", line 1089, in handle
    self.callHandlers(record)
  File "/usr/lib/python2.4/logging/__init__.py", line 1126, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python2.4/logging/__init__.py", line 642, in handle
    self.emit(record)
  File "/usr/lib/python2.4/logging/handlers.py", line 648, in emit
    string.lower(record.levelname)),
  File "/usr/lib/python2.4/logging/handlers.py", line 623, in encodePriority
    priority = self.priority_names[priority]
KeyError: 'Info'


Version-Release number of selected component (if applicable):
python-2.4.3-11.FC6
yum-2.9.3-1


How reproducible:
always

Steps to Reproduce:
1. set locale to tr_TR.UTF-8
2. use yum to install/update packages
3.
  
Actual results:
traceback above

Expected results:


Additional info:
Trying to update core system packages (like libc) breaks the system seriously,
hence the high severity.
Comment 1 Mihai Ibanescu 2006-07-16 23:02:37 EDT
I looked a bit at the code, but I couldn't figure out where the problem is. At
first glance, it appears that yum is trying to translate something that it
shouldn't. If you can pinpoint the problem before I get to it, and it turns out
to be in python, I'll be happy to report it upstream and/or patch python.
Comment 2 Mihai Ibanescu 2006-07-16 23:45:57 EDT
I can't quite see why the string would be in Turkish to begin with, it should be
a  simple ASCII string from the internal python module.
Comment 3 Sertaç Ö. Yıldız 2006-07-17 01:45:44 EDT
The problem is that the string "INFO" is passed to string.lower() and assumed 
to be equal to "info" and passed as a key to dict. Try this simple test case:

import string, locale
s= "INFO"
print string.lower(s),
locale.setlocale(locale.LC_ALL, '')
print string.lower(s)

The output is "info info" in most locale settings, but "info Info" in some
unlucky ones.
Comment 4 Mihai Ibanescu 2006-07-17 13:46:12 EDT
Created attachment 132561 [details]
Test case for the broken SysLog handler

This works:
python /tmp/logging-broken.py

This doesn't:
LANG=tr_TR.UTF-8 python /tmp/logging-broken.py
Comment 6 Mihai Ibanescu 2006-07-18 10:43:16 EDT
FWIW, u"INFO".lower() == "info" even in the Turkish locale, maybe that's the way
to force a comparison using the C locale after you've changed it to something else.

This link has some suggestions for what NOT to do:

http://docs.python.org/lib/node323.html
Comment 7 Mihai Ibanescu 2006-07-21 08:50:43 EDT
Fixed upstream in subversion transaction 50740

http://svn.python.org/view/python/trunk/Lib/logging/handlers.py?rev=50740&r1=47121&r2=50740&makepatch=1&diff_format=u

Fix added to python 2.4.3, should appear in 2.4.3-14

Note You need to log in before you can comment on or make changes to this bug.