Bug 725726 - XMLRPC: Methods traceback when data contain Unicode strings
XMLRPC: Methods traceback when data contain Unicode strings
Status: CLOSED CURRENTRELEASE
Product: TCMS
Classification: Other
Component: Application (Show other bugs)
3.5
Unspecified Unspecified
urgent Severity urgent
: ---
: ---
Assigned To: Yang Ren
tools-bugs
: Regression
Depends On:
Blocks: 593666
  Show dependency treegraph
 
Reported: 2011-07-26 07:48 EDT by Petr Šplíchal
Modified: 2016-05-31 21:42 EDT (History)
10 users (show)

See Also:
Fixed In Version: TCMS 3.6.2
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-09-26 09:20:48 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Petr Šplíchal 2011-07-26 07:48:24 EDT
Description of problem:

Fetching a user whose name contains Unicode characters results in
a traceback. This worked fine with Nitrate 3.4.

Version-Release number of selected component (if applicable):
Nitrate 3.5.0

Steps to Reproduce:
User.get(2117)
  
Actual results:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "nitrate.py", line 120, in request_with_cookies
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: "UnicodeEncodeError: 'ascii' codec can't encode character u'\\u0160' in position 0: ordinal not in range(128)">
Comment 1 Petr Šplíchal 2011-07-27 05:50:26 EDT
Seems the problem is not limited to the User object. Today I've
come across the following traceback while trying to fetch test
case data:

TestCase.get(12345)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "nitrate.py", line 120, in request_with_cookies
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: "UnicodeEncodeError: 'ascii' codec can't encode character u'\\u2013' in position 378: ordinal not in range(128)">
Comment 2 Yuguang Wang 2011-07-27 07:50:27 EDT
Seems works ok for me:
In [1]: from nitrate import NitrateKerbXmlrpc

In [2]: n = NitrateKerbXmlrpc('https://tcms-stage.englab.bne.redhat.com/xmlrpc/')

In [3]: n.server.User.get(2117)
Out[3]: 
{'date_joined': '2009-12-02 09:40:59',
 'email': 'psplicha@redhat.com',
 'first_name': 'Petr',
 'id': '2117',
 'is_active': 'True',
 'is_staff': 'True',
 'is_superuser': 'False',
 'last_login': '2011-07-26 17:40:51',
 'last_name': 'Splichal',
 'username': 'psplicha'}

In [4]: n = NitrateKerbXmlrpc('https://tcms.engineering.redhat.com/xmlrpc/')

In [5]: n.server.User.get(2117)
Out[5]: 
{'date_joined': '2009-12-02 09:40:59',
 'email': 'psplicha@redhat.com',
 'first_name': 'Petr',
 'id': '2117',
 'is_active': 'True',
 'is_staff': 'True',
 'is_superuser': 'False',
 'last_login': '2011-07-27 19:37:14',
 'last_name': 'Splichal',
 'username': 'psplicha'}


(In reply to comment #0)
> Description of problem:
> 
> Fetching a user whose name contains Unicode characters results in
> a traceback. This worked fine with Nitrate 3.4.
> 
> Version-Release number of selected component (if applicable):
> Nitrate 3.5.0
> 
> Steps to Reproduce:
> User.get(2117)
> 
> Actual results:
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
>     return self.__send(self.__name, args)
>   File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
>     verbose=self.__verbose
>   File "nitrate.py", line 120, in request_with_cookies
>     return self._parse_response(h.getfile(), sock)
>   File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
>     return u.close()
>   File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
>     raise Fault(**self._stack[0])
> xmlrpclib.Fault: <Fault 1: "UnicodeEncodeError: 'ascii' codec can't encode
> character u'\\u0160' in position 0: ordinal not in range(128)">
Comment 3 Yuguang Wang 2011-07-27 07:51:13 EDT
I'll see what's wrong in the backends.
Thanks for reporting.

(In reply to comment #1)
> Seems the problem is not limited to the User object. Today I've
> come across the following traceback while trying to fetch test
> case data:
Comment 4 Petr Šplíchal 2011-07-27 08:15:59 EDT
(In reply to comment #2)
> Seems works ok for me:
> 
> {'date_joined': '2009-12-02 09:40:59',
>  'email': 'psplicha@redhat.com',
>  'first_name': 'Petr',
>  'id': '2117',
>  'is_active': 'True',
>  'is_staff': 'True',
>  'is_superuser': 'False',
>  'last_login': '2011-07-26 17:40:51',
>  'last_name': 'Splichal',
>  'username': 'psplicha'}

I've changed my surname to ASCII only to be able to run my
scripts. Just use "Šplíchal" as the surname for an example.
Comment 5 Petr Šplíchal 2011-08-19 06:09:15 EDT
Any update about fixing this unpleasant regression?
Comment 6 Yuguang Wang 2011-08-19 06:41:40 EDT
@Petr,
Sorry, was busy with other stuffs recently.
I'll take it. There're some unexpected behaviors in rpc functions in 3.5.
(In reply to comment #5)
> Any update about fixing this unpleasant regression?
Comment 7 Jan Ščotka 2011-08-19 07:05:20 EDT
Hi,
I've found same bug with UTF formating:
 TP#1815 - system-config-printer / General: <Fault 1: "UnicodeEncodeError: 'ascii' codec can't encode character u'\\u201c' in position 6: ordinal not in range(128)">


I've found also another bug, in same function,
>>> TP#4333 - cups / General: <Fault 1: 'DoesNotExist: User matching query does not exist.'>

I don't know what case this problem, but I know that it is caused by testplan 4333
so it is reproducible very easy.

   Thanks&Regards
   Honza
Comment 8 Ales Zelinka 2011-08-29 04:14:06 EDT
Please fix this, it is hampering our ability to process test-tier runs (=critical process) as fast as needed.
Comment 9 Danqing Li 2011-09-02 03:08:04 EDT
Hi, This bug to be fixed in 3.6 version.
Comment 10 Petr Šplíchal 2011-09-02 04:03:48 EDT
This is a critical regression for us. What is the ETA of 3.6
please? I expected this issue to be fixed in 3.5 already.
Comment 11 Yuguang Wang 2011-09-05 03:21:39 EDT
Code frozen in two weeks. Depends on the acceptance testing result, 3.6 would be online after that.
Sorry for inconvenience.
Comment 12 Jan Ščotka 2011-09-13 10:05:37 EDT
another problem found, propably connect with UTF:
        cases = self.tcms.TestRun.get_test_cases(self.id)
XML-RPC returns:
  XML-RPC fault: DoesNotExist: User matching query does not exist.


when used 
https://tcms.engineering.redhat.com/run/26826/
It blocks my testing.

   Regards
   Honza
Comment 13 Jan Ščotka 2011-09-13 10:52:44 EDT
created bug for more verbose tracebacks in XML-RPC interface:
https://bugzilla.redhat.com/show_bug.cgi?id=737986
Comment 14 Chaobin Tang 2011-09-27 03:27:01 EDT
The 3.6 hotfix release should now be alive on production with this issue fixed.
Comment 15 Petr Šplíchal 2011-09-27 11:31:29 EDT
Issue mentioned in comment #0 seems to be fixed. However fetching TC#12345 still fails:

>>> TestCase.get(12345)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "nitrate.py", line 120, in request_with_cookies
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: 'DoesNotExist: User matching query does not exist.'>
Comment 16 Chaobin Tang 2011-09-27 23:02:51 EDT
In this thread we are having two different problems, let's create a new bug regarding to humanizing traceback information, and close this one.
Comment 17 Petr Šplíchal 2011-09-29 03:47:02 EDT
I believe the "better tracebacks" issue should be already covered
by bug 737986. Anyway, what's actual problem here? Why, when
trying to fetch a test case data I get an error message regarding
non-existent user?
Comment 18 Petr Šplíchal 2011-11-03 10:06:35 EDT
Another month passed without any feedback. Can we get status and
ETA on this urgent regression?
Comment 19 Chaobin Tang 2011-11-04 07:23:38 EDT
This goes fixed in the coming version, thus to be alive in three weeks to come.
Comment 20 Xin Gao 2011-11-24 02:18:41 EST
Verify version and result:
3.6.2 on englab --> PASSED

Actual results:
>>> from nitrate import NitrateKerbXmlrpc
>>> n=NitrateKerbXmlrpc('https://tcms.englab.nay.redhat.com/xmlrpc/')
>>> n.server.User.get(2117)
{'username': 'psplicha', 'first_name': 'Petr', 'last_name': 'Splichal', 'is_active': True, 'id': 2117, 'is_superuser': False, 'is_staff': True, 'last_login': '2011-09-29 15:47:24', 'groups': [1, 3], 'user_permissions': [], 'email': 'psplicha@redhat.com', 'date_joined': '2009-12-02 09:40:59'}

Note You need to log in before you can comment on or make changes to this bug.