Bug 466011 - Bug in unicode handling between python-sqlobject-0.9.7 and MySQL-python-1.2.1-1
Bug in unicode handling between python-sqlobject-0.9.7 and MySQL-python-1.2.1-1
Status: CLOSED CURRENTRELEASE
Product: Fedora EPEL
Classification: Fedora
Component: python-sqlobject (Show other bugs)
el5
All Linux
medium Severity medium
: ---
: ---
Assigned To: Luke Macken
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-10-07 14:04 EDT by Toshio Ernie Kuratomi
Modified: 2016-09-19 22:38 EDT (History)
5 users (show)

See Also:
Fixed In Version: 0.9.7-2
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-10-17 20:39:40 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Toshio Ernie Kuratomi 2008-10-07 14:04:54 EDT
Description of problem:
When using SQLObject 0.9.7 on EL-5 (with MySQL-python-1.2.1-1) Unicode errors are thrown when unicode input is given.


How reproducible:
Everytime

Steps to Reproduce:
1. Run transifex-0.2 on an EL-5 box
2. Attempt to submit something while logged in with a username that uses unicode characters.
3.
  
Actual results:
Traceback

Expected results:
upload will be submitted

Additional info:
The root cause of this is that SQLObject-0.9.7 and above is passing the query string to the MySQL bindings as a byte string and the MySQL bindings are expecting a unicode string.  In Fedora-9 this is fixed in the MySQL bindings by checking whether unicode or a byte string were given.

In SQLObject-0.9.2, there is a check in mysql/mysqlconnection.py that checks if the version of Mysql is greater than X.Y and if so transforms the query string to unicode prior to submitting it to the backend.

We have several choices for resolving this:

* open a bug against EL-5's MySQL-python
* return to python-sqlobject-0.9.2
* forward port the detection of MySQL-python version and subsequent transformation to python-sqlobject-0.9.7

Looking at the code, this appears to affect 0.10.2 as well.
Comment 1 Toshio Ernie Kuratomi 2008-10-07 14:12:47 EDT
Relevant section of: sqlobject/mysql/mysqlconnection.py

class MySQLConnection(DBAPI):
[...]
    def __init__(self, db, user, password='', host='localhost', port=0, **kw):
[...]
        if MySQLdb.version_info[:3] >= (1, 2, 1):
            # need to add: and MySQLdb.version_info[:3] < (1, 2, 2):
            self.need_unicode = True
        else:
            self.need_unicode = False
[...]
    def _executeRetry(self, conn, cursor, query):
[...]
                if self.need_unicode and not isinstance(query, unicode):
                    try:
                        query = unicode(query, self.encoding)
                    except UnicodeError:
                        pass                        
                return cursor.execute(query)
Comment 2 Toshio Ernie Kuratomi 2008-10-07 22:03:31 EDT
- query = unicode(query, self.encoding)
+ query = unicode(query, self.dbEncoding)
Comment 3 Toshio Ernie Kuratomi 2008-10-07 22:25:56 EDT
Hotpatched this on app4 in fedora infrastructure.  This seems to allow transifex-0.2 to work.
Comment 4 Toshio Ernie Kuratomi 2008-10-17 20:20:42 EDT
Patch under discussion on upstream mailing list and applied toour EL-5 package.
Comment 5 andred 2009-02-19 23:15:57 EST
(In reply to comment #2)
> - query = unicode(query, self.encoding)
> + query = unicode(query, self.dbEncoding)

This modification doesn't work in all cases. 

class MySQLConnection(DBAPI):
[...]
    def __init__(self, db, user, password='', host='localhost', port=0, **kw):
[...]
        if "charset" in kw:
            self.dbEncoding = self.kw["charset"] = col.popKey(kw, "charset")
        else:
            self.dbEncoding = None

self.dbEncoding will be None if charset isn't passed in with the kw's. This then causes

[...]
    def _executeRetry(self, conn, cursor, query):
[...]
                if self.need_unicode and not isinstance(query, unicode):
                    try:
                        query = unicode(query, self.dbEncoding)
                except UnicodeError:
 

to throw a TypeError which isn't caught by the exception handler.

TypeError: unicode() argument 2 must be string, not None

I have run into this using TurboGears / sqlobject to connect to a MySQL DB while attempting to support RHEL for a custom project.

Note You need to log in before you can comment on or make changes to this bug.