Bug 725726 - XMLRPC: Methods traceback when data contain Unicode strings
Summary: XMLRPC: Methods traceback when data contain Unicode strings
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: TCMS
Classification: Other
Component: Application
Version: 3.5
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: ---
Assignee: Yang Ren
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks: 593666
TreeView+ depends on / blocked
 
Reported: 2011-07-26 11:48 UTC by Petr Šplíchal
Modified: 2016-06-01 01:42 UTC (History)
10 users (show)

Fixed In Version: TCMS 3.6.2
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-09-26 13:20:48 UTC
Embargoed:


Attachments (Terms of Use)

Description Petr Šplíchal 2011-07-26 11:48:24 UTC
Description of problem:

Fetching a user whose name contains Unicode characters results in
a traceback. This worked fine with Nitrate 3.4.

Version-Release number of selected component (if applicable):
Nitrate 3.5.0

Steps to Reproduce:
User.get(2117)
  
Actual results:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "nitrate.py", line 120, in request_with_cookies
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: "UnicodeEncodeError: 'ascii' codec can't encode character u'\\u0160' in position 0: ordinal not in range(128)">

Comment 1 Petr Šplíchal 2011-07-27 09:50:26 UTC
Seems the problem is not limited to the User object. Today I've
come across the following traceback while trying to fetch test
case data:

TestCase.get(12345)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "nitrate.py", line 120, in request_with_cookies
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: "UnicodeEncodeError: 'ascii' codec can't encode character u'\\u2013' in position 378: ordinal not in range(128)">

Comment 2 Yuguang Wang 2011-07-27 11:50:27 UTC
Seems works ok for me:
In [1]: from nitrate import NitrateKerbXmlrpc

In [2]: n = NitrateKerbXmlrpc('https://tcms-stage.englab.bne.redhat.com/xmlrpc/')

In [3]: n.server.User.get(2117)
Out[3]: 
{'date_joined': '2009-12-02 09:40:59',
 'email': 'psplicha',
 'first_name': 'Petr',
 'id': '2117',
 'is_active': 'True',
 'is_staff': 'True',
 'is_superuser': 'False',
 'last_login': '2011-07-26 17:40:51',
 'last_name': 'Splichal',
 'username': 'psplicha'}

In [4]: n = NitrateKerbXmlrpc('https://tcms.engineering.redhat.com/xmlrpc/')

In [5]: n.server.User.get(2117)
Out[5]: 
{'date_joined': '2009-12-02 09:40:59',
 'email': 'psplicha',
 'first_name': 'Petr',
 'id': '2117',
 'is_active': 'True',
 'is_staff': 'True',
 'is_superuser': 'False',
 'last_login': '2011-07-27 19:37:14',
 'last_name': 'Splichal',
 'username': 'psplicha'}


(In reply to comment #0)
> Description of problem:
> 
> Fetching a user whose name contains Unicode characters results in
> a traceback. This worked fine with Nitrate 3.4.
> 
> Version-Release number of selected component (if applicable):
> Nitrate 3.5.0
> 
> Steps to Reproduce:
> User.get(2117)
> 
> Actual results:
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
>     return self.__send(self.__name, args)
>   File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
>     verbose=self.__verbose
>   File "nitrate.py", line 120, in request_with_cookies
>     return self._parse_response(h.getfile(), sock)
>   File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
>     return u.close()
>   File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
>     raise Fault(**self._stack[0])
> xmlrpclib.Fault: <Fault 1: "UnicodeEncodeError: 'ascii' codec can't encode
> character u'\\u0160' in position 0: ordinal not in range(128)">

Comment 3 Yuguang Wang 2011-07-27 11:51:13 UTC
I'll see what's wrong in the backends.
Thanks for reporting.

(In reply to comment #1)
> Seems the problem is not limited to the User object. Today I've
> come across the following traceback while trying to fetch test
> case data:

Comment 4 Petr Šplíchal 2011-07-27 12:15:59 UTC
(In reply to comment #2)
> Seems works ok for me:
> 
> {'date_joined': '2009-12-02 09:40:59',
>  'email': 'psplicha',
>  'first_name': 'Petr',
>  'id': '2117',
>  'is_active': 'True',
>  'is_staff': 'True',
>  'is_superuser': 'False',
>  'last_login': '2011-07-26 17:40:51',
>  'last_name': 'Splichal',
>  'username': 'psplicha'}

I've changed my surname to ASCII only to be able to run my
scripts. Just use "Šplíchal" as the surname for an example.

Comment 5 Petr Šplíchal 2011-08-19 10:09:15 UTC
Any update about fixing this unpleasant regression?

Comment 6 Yuguang Wang 2011-08-19 10:41:40 UTC
@Petr,
Sorry, was busy with other stuffs recently.
I'll take it. There're some unexpected behaviors in rpc functions in 3.5.
(In reply to comment #5)
> Any update about fixing this unpleasant regression?

Comment 7 Jan Ščotka 2011-08-19 11:05:20 UTC
Hi,
I've found same bug with UTF formating:
 TP#1815 - system-config-printer / General: <Fault 1: "UnicodeEncodeError: 'ascii' codec can't encode character u'\\u201c' in position 6: ordinal not in range(128)">


I've found also another bug, in same function,
>>> TP#4333 - cups / General: <Fault 1: 'DoesNotExist: User matching query does not exist.'>

I don't know what case this problem, but I know that it is caused by testplan 4333
so it is reproducible very easy.

   Thanks&Regards
   Honza

Comment 8 Ales Zelinka 2011-08-29 08:14:06 UTC
Please fix this, it is hampering our ability to process test-tier runs (=critical process) as fast as needed.

Comment 9 Danqing Li 2011-09-02 07:08:04 UTC
Hi, This bug to be fixed in 3.6 version.

Comment 10 Petr Šplíchal 2011-09-02 08:03:48 UTC
This is a critical regression for us. What is the ETA of 3.6
please? I expected this issue to be fixed in 3.5 already.

Comment 11 Yuguang Wang 2011-09-05 07:21:39 UTC
Code frozen in two weeks. Depends on the acceptance testing result, 3.6 would be online after that.
Sorry for inconvenience.

Comment 12 Jan Ščotka 2011-09-13 14:05:37 UTC
another problem found, propably connect with UTF:
        cases = self.tcms.TestRun.get_test_cases(self.id)
XML-RPC returns:
  XML-RPC fault: DoesNotExist: User matching query does not exist.


when used 
https://tcms.engineering.redhat.com/run/26826/
It blocks my testing.

   Regards
   Honza

Comment 13 Jan Ščotka 2011-09-13 14:52:44 UTC
created bug for more verbose tracebacks in XML-RPC interface:
https://bugzilla.redhat.com/show_bug.cgi?id=737986

Comment 14 Chaobin Tang 2011-09-27 07:27:01 UTC
The 3.6 hotfix release should now be alive on production with this issue fixed.

Comment 15 Petr Šplíchal 2011-09-27 15:31:29 UTC
Issue mentioned in comment #0 seems to be fixed. However fetching TC#12345 still fails:

>>> TestCase.get(12345)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "nitrate.py", line 120, in request_with_cookies
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: 'DoesNotExist: User matching query does not exist.'>

Comment 16 Chaobin Tang 2011-09-28 03:02:51 UTC
In this thread we are having two different problems, let's create a new bug regarding to humanizing traceback information, and close this one.

Comment 17 Petr Šplíchal 2011-09-29 07:47:02 UTC
I believe the "better tracebacks" issue should be already covered
by bug 737986. Anyway, what's actual problem here? Why, when
trying to fetch a test case data I get an error message regarding
non-existent user?

Comment 18 Petr Šplíchal 2011-11-03 14:06:35 UTC
Another month passed without any feedback. Can we get status and
ETA on this urgent regression?

Comment 19 Chaobin Tang 2011-11-04 11:23:38 UTC
This goes fixed in the coming version, thus to be alive in three weeks to come.

Comment 20 Xin Gao 2011-11-24 07:18:41 UTC
Verify version and result:
3.6.2 on englab --> PASSED

Actual results:
>>> from nitrate import NitrateKerbXmlrpc
>>> n=NitrateKerbXmlrpc('https://tcms.englab.nay.redhat.com/xmlrpc/')
>>> n.server.User.get(2117)
{'username': 'psplicha', 'first_name': 'Petr', 'last_name': 'Splichal', 'is_active': True, 'id': 2117, 'is_superuser': False, 'is_staff': True, 'last_login': '2011-09-29 15:47:24', 'groups': [1, 3], 'user_permissions': [], 'email': 'psplicha', 'date_joined': '2009-12-02 09:40:59'}


Note You need to log in before you can comment on or make changes to this bug.