Bug 826749

Created attachment 587858 [details]
test script

Created attachment 587860 [details]
update python-bugzilla rpms

It looks like it's using the Bug.search RPC call.

I traced through my Python script and got to the __request() function from /usr/lib64/python2.7/xmlrpclib.py

    def __request(self, methodname, params):
        # call a method on the remote server

        request = dumps(params, methodname, encoding=self.__encoding,
                        allow_none=self.__allow_none)
        ...


And the python debugger says:

    (Pdb) methodname
    'Bug.search'
    (Pdb) p params
    ({'f1': 'keywords', 'product': ['Fedora'], 'query_format': 'advanced', 'bug_status': ['NEW', 'ASSIGNED'], 'v1': 'Patch', 'include_fields': ['id', 'component', 'opendate', 'summary'], 'o1': 'substring'},)

If I break right after the query call, the data returned includes:

(Pdb) results[0].__dict__
{'autorefresh': True, 'url': 'https://bugzilla.redhat.com/show_bug.cgi?id=72487', 'short_desc': 'RFE: Add support for disabling the duplicate IP check', 'component': 'initscripts', 'summary': 'RFE: Add support for disabling the duplicate IP check', 'bug_id': 72487, 'bugzilla': <bugzilla.rhbugzilla.RHBugzilla4 object at 0xe9a0d0>, 'components': ['initscripts'], 'id': 72487}


If I then I try to look at the missing opendate, it uses the __getattr__ call to try and fill it in, but fails:

(Pdb) results[0].opendate
*** AttributeError: field opendate not in bugzilla.bugfields


Hmm, what *is* in the bugzilla.bugfields list?

BZ is my instance of the bugzilla class, so:

(Pdb) BZ.bugfields
['content', 'bug_id', 'work_time', 'keywords', 'cf_qa_whiteboard', 'view', 'blocked', 'cf_cust_facing', 'qa_contact', 'everconfirmed', 'bug_severity', 'cf_conditional_nak', 'estimated_time', 'cf_pm_score', 'attachments.mimetype', 'attachments.isobsolete', 'reporter_accessible', 'cf_verified', 'cf_crm', 'cf_qe_conditional_nak', 'attachments.description', 'external_bugzilla.url', 'vcs_commits', 'attach_data.thedata', 'bug_file_loc', 'cf_clone_of', 'ext_bz_bug_map.ext_bz_bug_id', 'status_whiteboard', 'votes', 'delta_ts', 'cf_environment', 'attachments.isprivate', 'version', 'requestees.login_name', 'deadline', 'attachments.ispatch', 'dependson', 'commenter', 'cf_release_notes', 'flagtypes.name', 'priority', 'bug_group', 'cclist_accessible', 'cc', 'setters.login_name', 'longdescs.count', 'see_also', 'assigned_to', 'tag', 'short_desc', 'cf_regression_status', 'cf_partner', 'external_bugzilla.description', 'remaining_time', 'cf_story_points', 'owner_idle_time', 'resolution', 'classification', 'class', 'cf_devel_whiteboard', 'cf_internal_whiteboard', 'alias', 'longdescs.isprivate', 'op_sys', 'target_release', 'percentage_complete', 'attachments.filename', 'reporter', 'cf_type', 'cf_documentation_action', 'rep_platform', 'cf_mount_type', 'creation_ts', 'longdesc', 'attachments.submitter', 'component', 'days_elapsed', 'cf_fixed_in', 'attachstatusdefs.name', 'product', 'target_milestone', 'cf_build_id', 'cf_verified_branch', 'blockedby', 'components', 'status']

There's a lot of fields listed, but opendate is not one of them.

See bug 823981 for a similar problem with cf_last_closed

Hi Jeff,

Reading the bugzilla4 code, it seems like opendate has been renamed to creation_ts.  Can you try that and see if it works?

Cheers,
Don

Ahh, in my frustration yesterday I missed that field.  Yes, that seems to work!  Thanks!  (Why, oh why, were all these field names changed?)

(And the format of the date changed from 'YYYY-mm-dd HH:MM:SS' to 'YYYY.mm.dd HH:MM'.  Argh.)


However, the creation_ts field is still not returned in the initial results.  I put a break right after making the query:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...
query['include_fields'] = [ 'id', 'component', 'creation_ts', 'summary' ]

print "%s Making query" % (timestamp())
results = BZ.query(query)
import pdb; pdb.set_trace()
...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


And there is no creation_ts included in the result:

(Pdb) results[0].__dict__
{'autorefresh': True, 'url': 'https://bugzilla.redhat.com/show_bug.cgi?id=72487', 'short_desc': 'RFE: Add support for disabling the duplicate IP check', 'component': 'initscripts', 'summary': 'RFE: Add support for disabling the duplicate IP check', 'bug_id': 72487, 'bugzilla': <bugzilla.rhbugzilla.RHBugzilla4 object at 0x12d20d0>, 'components': ['initscripts'], 'id': 72487}


It fills in the missing data when I try to use it:

(Pdb) results[0].creation_ts
'2002.08.24 09:25'


But the above command takes nearly a minute to run, and it returns gobs and gobs of data, much more than the one requested field.  The data returned initially has 9 entries in the dictionary:

(Pdb) len(results[0].__dict__)
9

But after getting creation_ts, it balloons to 119!

(Pdb) len(results[0].__dict__)
119


So why is it not returned initially even though it's specified in include_fields?

Hi Jeff,

Sorry about that, I didn't read the nuisances of the mozilla bugzilla4 code correctly (this is soooo annoying that I have to read the upstream code to understand what the xmlrpc is doing).  The field you want is

creation_time

which is translated to creation_ts in the backend of the bugzilla server which also looks for opendate.  But opendate doesn't seem to be allowed in xmlrpc, so I am not sure what it is hoping to translate.  *sigh*

That work in my quick testing, *without* the 50-60 second  Bug.field delay.

Cheers,
Don

creation_time does work much faster than creation_ts

It's strange that creation_time is not listed in BZ.bugfields as a valid field (see comment 4)


I'll re-work my patch.

Thank you for all the help!  I really appreciate it!

if it's not listed and it should, see _getbugfields() so you can add it in.

as reference for fields, you can look here:
http://www.bugzilla.org/docs/tip/en/html/api/Bugzilla/WebService/Bug.html

Changes to my test script from comment 1 include using creation_time in the include_fields along with some string manipulation on the creation_time to convert it to the old format:
  opendate format:      YYYY-mm-dd HH:MM:SS
  creation_time format: YYYYmmddTHH:MM:SS


--- bz-query.py.ORIG	2012-05-31 14:01:41.199815836 -0500
+++ bz-query.py	2012-05-31 14:02:47.178958589 -0500
@@ -38,7 +38,7 @@
 query['f1'] = 'keywords'
 query['o1'] = 'substring'
 query['v1'] = 'Patch'
-query['include_fields'] = [ 'id', 'component', 'opendate', 'summary' ]
+query['include_fields'] = [ 'id', 'component', 'creation_time', 'summary' ]
 
 print "%s Making query" % (timestamp())
 results = BZ.query(query)
@@ -46,5 +46,7 @@
 
 print "%s Printing report" % (timestamp())
 for r in results:
-    output = "%s %s %s %s" % (r.id, r.component, r.opendate, r.summary)
+    opendate = r.creation_time.value.replace('T', ' ')
+    opendate = opendate[:4] + "-" + opendate[4:6] + "-" + opendate[6:]
+    output = "%s %s %s %s" % (r.id, r.component, opendate, r.summary)
     print "%s %s" % (timestamp(), output)

Hi Jeff,

Thanks for those changes.  I'll try to pass them along to Will to get them into upstream python-bugzilla.

Cheers,
Don

Upstream python-bugzilla now aliases opendate -> creation_time and creation_ts -> creation_time, so there's only one value kept internally. I think this will solve the problems reported here.

*** Bug 823981 has been marked as a duplicate of this bug. ***

Fixed in python-bugzilla-0.8.0-1.fc19