Bug 715596

Summary: XMLRPC driver does not allow None values
Product: [Other] TCMS Reporter: Petr Šplíchal <psplicha>
Component: ApplicationAssignee: jianchen <jianchen>
Status: CLOSED CURRENTRELEASE QA Contact: tools-bugs <tools-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.4.1CC: aigao, azelinka, jianchen, junzhang, nli, ohudlick, ryang, tools-bugs, vchen, xchu, xgao, yuwang, zheliu
Target Milestone: ---   
Target Release: 3.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 3.8.0-3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-09-26 13:21:05 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 593666    
Attachments:
Description Flags
Simple patch allowing None values none

Description Petr Šplíchal 2011-06-23 13:05:42 UTC
Description of problem:

Currently it's not possible to update a field value to "None",
for example:

    TestCaseRun.update(1234, {"notes": None }

gives the following traceback:

    Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
        return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1483, in __request
        allow_none=self.__allow_none)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1132, in dumps
        data = m.dumps(params)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 677, in dumps
        dump(v, write)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 699, in __dump
        f(self, value, write)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 780, in dump_struct
        dump(v, write)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 699, in __dump
        f(self, value, write)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 703, in dump_nil
        raise TypeError, "cannot marshal None unless allow_none is enabled"
    TypeError: cannot marshal None unless allow_none is enabled

This can be easily fixed by providing xmlrpclib.ServerProxy with a
"allow_none = 1" parameter.

Version-Release number of selected component (if applicable):
Nitrate 3.4.1

Additional info:
Simple patch attached.

Comment 1 Petr Šplíchal 2011-06-23 13:07:42 UTC
Created attachment 506233 [details]
Simple patch allowing None values

Simple patch allowing None values

Comment 2 Petr Šplíchal 2011-06-23 13:13:59 UTC
Well, the patch fixes the traceback, however the value on the
server is still not updated. Is there any other way how to update
field value to None?

Comment 3 Yuguang Wang 2011-07-05 02:39:57 UTC
Sorry for late reply.

Do you really need to set notes = None, how about leave it blank.
See codes below:

if form.cleaned_data['notes']:
    tcrs.update(notes = form.cleaned_data['notes'])

Set notes = None, the 'tcrs.update()' will never execute, this is the reason.

Comment 4 Petr Šplíchal 2011-07-07 12:30:34 UTC
I see, using empty string could be a solution here. But similar
problems appear at other places. For example when trying to clear
the default tester for a test run:

    TestRun.update(6757, {'default_tester': None })

This will not clear the default tester id and using and empty
string here is not an option. I believe there should be a clean
way how to set values to None.

Comment 5 Yuguang Wang 2011-07-08 09:46:26 UTC
I also suffer from this 'does not allow none values' while calling rpc.
I'll take it, will do some research on it and update afterwards.

Thanks for reporting.

Comment 6 Petr Šplíchal 2011-09-21 09:22:50 UTC
Test case "arguments" field started to return "None" while before
it returned "" when the field was empty. Is that connected with
implementation of this bug? Is that expected behavior? Thanks.

Comment 7 Yuguang Wang 2011-09-21 10:08:42 UTC
See case 32768 as an example.
from Web, https://tcms.engineering.redhat.com/case/32768/
The fields 'arguments' are displayed as "None"

from rpc, it returns empty string:
>>> n.server.TestCase.get(32768).get('arguments')
''

In db:
mysql> select arguments from test_cases where case_id=32768;
+-----------+
| arguments |
+-----------+
|           |
+-----------+


The 'arguments' field is empty while through Web you'll see 'None' instead of empty.

Comment 8 Petr Šplíchal 2011-09-21 10:57:26 UTC
But this differs from case to case, see bug 740220 for an example
where for one test case '' is returned and for the other 'None'.
In the web interface both show as None.

Comment 9 Chaobin Tang 2011-09-27 09:03:26 UTC
In TCMS 3.6 Hotfix which is already now alive, the driver initialization will set allow_none=True.

Comment 10 Petr Šplíchal 2011-09-27 14:57:26 UTC
Thanks for updating the driver. However, this is only a part of
the problem. We're still unable to update object with a 'None'
value, see comment #4 for an example. This still does not work.

Comment 11 Petr Šplíchal 2012-05-04 13:31:29 UTC
Hi, any progress here?

Comment 12 jianchen 2012-07-18 12:54:30 UTC
This bug was fixed in tcms 3.8.0.
Thanks.

Comment 13 Petr Šplíchal 2012-07-25 09:41:06 UTC
This seems to be working with the example in comment #4 but I am
still not able to update the test case run notes to None. Also
providing an empty string does not have any effect:

>>> TestCaseRun.update(1205413, {"notes": "hmmm" })
[{'case': 'Blah blah blah blah',
'build_id': 1156,
'tested_by_id': 2117,
'environment_id': 0,
'run': 'Test run',
'links': [],
'run_id': 41118,
'notes': 'hmmm',     <------------------- here --------------
'sortkey': 0,
'running_date': None,
'assignee': 'mfranc',
'build': 'unspecified',
'case_id': 35269,
'is_current': False,
'case_run_status': 'IDLE',
'assignee_id': 2739,
'case_run_id': 1205413,
'tested_by': 'psplicha',
'case_text_version': 1,
'case_run_status_id': 1,
'close_date': '2012-07-25 17:27:14'}]

>>> TestCaseRun.update(1205413, {"notes": "" })
[{'case': 'Blah blah blah blah',
'build_id': 1156,
'tested_by_id': 2117,
'environment_id': 0,
'run': 'Test run',
'links': [],
'run_id': 41118,
'notes': 'hmmm',     <------------------- here --------------
'sortkey': 0,
'running_date': None,
'assignee': 'mfranc',
'build': 'unspecified',
'case_id': 35269,
'is_current': False,
'case_run_status': 'IDLE',
'assignee_id': 2739,
'case_run_id': 1205413,
'tested_by': 'psplicha',
'case_text_version': 1,
'case_run_status_id': 1,
'close_date': '2012-07-25 17:27:14'}]

>>> TestCaseRun.update(1205413, {"notes": None })
[{'case': 'Blah blah blah blah',
'build_id': 1156,
'tested_by_id': 2117,
'environment_id': 0,
'run': 'Test run',
'links': [],
'run_id': 41118,
'notes': 'hmmm',     <------------------- here --------------
'sortkey': 0,
'running_date': None,
'assignee': 'mfranc',
'build': 'unspecified',
'case_id': 35269,
'is_current': False,
'case_run_status': 'IDLE',
'assignee_id': 2739,
'case_run_id': 1205413,
'tested_by': 'psplicha',
'case_text_version': 1,
'case_run_status_id': 1,
'close_date': '2012-07-25 17:27:14'}]

Comment 14 jianchen 2012-07-27 05:51:46 UTC
Now TestCaseRun.update() and TestRun.update() suport notes None or ''.
There still some field can't support None or '', we will continuous improvement that.
Thanks,

Comment 15 Petr Šplíchal 2012-07-30 11:09:51 UTC
Thanks for the quick fix. Last thing: I've noticed that when
supplying en empty string the value is converted to None. I'm not
sure whether this would be expected / desired behaviour:

>>> TestCaseRun.update(1205413, {"notes": '' })
[{'case': 'Blah blah blah blah',
'build_id': 1156,
'tested_by_id': 2117,
'environment_id': 0,
'run': 'Test run',
'links': [],
'run_id': 41118,
'notes': None,      <--------------- here ---------
'sortkey': 0,
'running_date': None,
'assignee': 'mfranc',
'build': 'unspecified',
'case_id': 35269,
'is_current': False,
'case_run_status': 'IDLE',
'assignee_id': 2739,
'case_run_id': 1205413,
'tested_by': 'psplicha',
'case_text_version': 1,
'case_run_status_id': 1,
'close_date': '2012-07-25 17:27:14'}]

Comment 16 jianchen 2012-07-31 08:38:48 UTC
(In reply to comment #15)
> Thanks for the quick fix. Last thing: I've noticed that when
> supplying en empty string the value is converted to None. I'm not
> sure whether this would be expected / desired behaviour:
> 
> >>> TestCaseRun.update(1205413, {"notes": '' })
> [{'case': 'Blah blah blah blah',
> 'build_id': 1156,
> 'tested_by_id': 2117,
> 'environment_id': 0,
> 'run': 'Test run',
> 'links': [],
> 'run_id': 41118,
> 'notes': None,      <--------------- here ---------
> 'sortkey': 0,
> 'running_date': None,
> 'assignee': 'mfranc',
> 'build': 'unspecified',
> 'case_id': 35269,
> 'is_current': False,
> 'case_run_status': 'IDLE',
> 'assignee_id': 2739,
> 'case_run_id': 1205413,
> 'tested_by': 'psplicha',
> 'case_text_version': 1,
> 'case_run_status_id': 1,
> 'close_date': '2012-07-25 17:27:14'}]
Hi Petr, i'm just treat empty string equal to Python build-in None type.
It is reasonable that keep user's initial input.
what's your opinion?

Comment 17 jianchen 2012-07-31 09:09:13 UTC
(In reply to comment #16)
> (In reply to comment #15)
> > Thanks for the quick fix. Last thing: I've noticed that when
> > supplying en empty string the value is converted to None. I'm not
> > sure whether this would be expected / desired behaviour:
> > 
> > >>> TestCaseRun.update(1205413, {"notes": '' })
> > [{'case': 'Blah blah blah blah',
> > 'build_id': 1156,
> > 'tested_by_id': 2117,
> > 'environment_id': 0,
> > 'run': 'Test run',
> > 'links': [],
> > 'run_id': 41118,
> > 'notes': None,      <--------------- here ---------
> > 'sortkey': 0,
> > 'running_date': None,
> > 'assignee': 'mfranc',
> > 'build': 'unspecified',
> > 'case_id': 35269,
> > 'is_current': False,
> > 'case_run_status': 'IDLE',
> > 'assignee_id': 2739,
> > 'case_run_id': 1205413,
> > 'tested_by': 'psplicha',
> > 'case_text_version': 1,
> > 'case_run_status_id': 1,
> > 'close_date': '2012-07-25 17:27:14'}]
> Hi Petr, i'm just treat empty string equal to Python build-in None type.
> It is reasonable that keep user's initial input.
> what's your opinion?
i'm means will keep user's input, retain empty string.

Comment 18 Petr Šplíchal 2012-07-31 14:11:49 UTC
Yes, I think '' and 'None' should be treated as different values
and should be retained in their original form, not converted.

Comment 19 Xin Gao 2012-08-03 07:02:31 UTC
Verify 3.8.0-3 on stage --->PASS

Verify steps:
In [11]: tcms.TestCaseRun.update([12345], {'notes': ''})

Actual result:
Out[11]: 
[{'assignee': 'xkuang',
  'assignee_id': 2206,
  'build': '4.2.0.GA_CP06-CR1',
  'build_id': 93,
  'case': 'compat12',
  'case_id': 4617,
  'case_run_id': 12345,
  'case_run_status': 'PASSED',
  'case_run_status_id': 2,
  'case_text_version': 1,
  'close_date': '2009-02-12 09:07:31',
  'environment_id': 118,
  'is_current': True,
  'links': [],
  'notes': '',  <----------- it was retained in original form  -----------
  'run': 'TCK1.4 EAP4.2 CP06',
  'run_id': 758,
  'running_date': None,
  'sortkey': None,
  'tested_by': 'istudens',
  'tested_by_id': 2160}]

Comment 20 Petr Šplíchal 2012-08-03 12:32:54 UTC
Great, thanks for the quick fix!