Bug 826749 - opendate not returned in include_fields
opendate not returned in include_fields
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: python-bugzilla (Show other bugs)
rawhide
All Linux
high Severity high
: ---
: ---
Assigned To: Will Woods
Fedora Extras Quality Assurance
:
: 823981 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-30 17:54 EDT by Jeff Bastian
Modified: 2013-02-16 10:35 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-02-16 10:35:30 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
test script (1.25 KB, text/plain)
2012-05-30 17:54 EDT, Jeff Bastian
no flags Details
update python-bugzilla rpms (124.29 KB, application/x-gzip)
2012-05-30 17:57 EDT, Jeff Bastian
no flags Details

  None (edit)
Description Jeff Bastian 2012-05-30 17:54:04 EDT
Description of problem:
I have a script using the python-bugzilla module.  My query includes opendate in the include_fields, but opendate is not returned.  The other 3 fields are returned.
   query['include_fields'] = [ 'id', 'component', 'opendate', 'summary' ]

$ ./bz-query.py 
...
AttributeError: field opendate not in bugzilla.bugfields

Version-Release number of selected component (if applicable):
Bugzilla 4.2
python-bugzilla-0.6.3pre-3.fc16.noarch
   -- a local build of the module with the latest patches
      for BZ 4.2 from Aristeu Rozanski Filho

How reproducible:
every time

Steps to Reproduce:
1. create ~/.bugzillarc if necessary
2. run the attached bz-query.py script
  
Actual results:
AttributeError: field opendate not in bugzilla.bugfields

Expected results:
report of open Fedora BZs with Patch keyword

Additional info:
Using the web interface and clicking "Change columns", I can select the "Opened" column which uses the 'opendate' keyword:
   bugzilla.redhat.com/buglist.cgi?...columnlist=bug_severity%2C...%2Copendate
                                                                      ^^^^^^^^
Comment 1 Jeff Bastian 2012-05-30 17:54:49 EDT
Created attachment 587858 [details]
test script
Comment 2 Jeff Bastian 2012-05-30 17:57:44 EDT
Created attachment 587860 [details]
update python-bugzilla rpms
Comment 3 Jeff Bastian 2012-05-30 18:19:53 EDT
It looks like it's using the Bug.search RPC call.

I traced through my Python script and got to the __request() function from /usr/lib64/python2.7/xmlrpclib.py

    def __request(self, methodname, params):
        # call a method on the remote server

        request = dumps(params, methodname, encoding=self.__encoding,
                        allow_none=self.__allow_none)
        ...


And the python debugger says:

    (Pdb) methodname
    'Bug.search'
    (Pdb) p params
    ({'f1': 'keywords', 'product': ['Fedora'], 'query_format': 'advanced', 'bug_status': ['NEW', 'ASSIGNED'], 'v1': 'Patch', 'include_fields': ['id', 'component', 'opendate', 'summary'], 'o1': 'substring'},)
Comment 4 Jeff Bastian 2012-05-30 18:26:27 EDT
If I break right after the query call, the data returned includes:

(Pdb) results[0].__dict__
{'autorefresh': True, 'url': 'https://bugzilla.redhat.com/show_bug.cgi?id=72487', 'short_desc': 'RFE: Add support for disabling the duplicate IP check', 'component': 'initscripts', 'summary': 'RFE: Add support for disabling the duplicate IP check', 'bug_id': 72487, 'bugzilla': <bugzilla.rhbugzilla.RHBugzilla4 object at 0xe9a0d0>, 'components': ['initscripts'], 'id': 72487}


If I then I try to look at the missing opendate, it uses the __getattr__ call to try and fill it in, but fails:

(Pdb) results[0].opendate
*** AttributeError: field opendate not in bugzilla.bugfields


Hmm, what *is* in the bugzilla.bugfields list?

BZ is my instance of the bugzilla class, so:

(Pdb) BZ.bugfields
['content', 'bug_id', 'work_time', 'keywords', 'cf_qa_whiteboard', 'view', 'blocked', 'cf_cust_facing', 'qa_contact', 'everconfirmed', 'bug_severity', 'cf_conditional_nak', 'estimated_time', 'cf_pm_score', 'attachments.mimetype', 'attachments.isobsolete', 'reporter_accessible', 'cf_verified', 'cf_crm', 'cf_qe_conditional_nak', 'attachments.description', 'external_bugzilla.url', 'vcs_commits', 'attach_data.thedata', 'bug_file_loc', 'cf_clone_of', 'ext_bz_bug_map.ext_bz_bug_id', 'status_whiteboard', 'votes', 'delta_ts', 'cf_environment', 'attachments.isprivate', 'version', 'requestees.login_name', 'deadline', 'attachments.ispatch', 'dependson', 'commenter', 'cf_release_notes', 'flagtypes.name', 'priority', 'bug_group', 'cclist_accessible', 'cc', 'setters.login_name', 'longdescs.count', 'see_also', 'assigned_to', 'tag', 'short_desc', 'cf_regression_status', 'cf_partner', 'external_bugzilla.description', 'remaining_time', 'cf_story_points', 'owner_idle_time', 'resolution', 'classification', 'class', 'cf_devel_whiteboard', 'cf_internal_whiteboard', 'alias', 'longdescs.isprivate', 'op_sys', 'target_release', 'percentage_complete', 'attachments.filename', 'reporter', 'cf_type', 'cf_documentation_action', 'rep_platform', 'cf_mount_type', 'creation_ts', 'longdesc', 'attachments.submitter', 'component', 'days_elapsed', 'cf_fixed_in', 'attachstatusdefs.name', 'product', 'target_milestone', 'cf_build_id', 'cf_verified_branch', 'blockedby', 'components', 'status']

There's a lot of fields listed, but opendate is not one of them.
Comment 5 Jeff Bastian 2012-05-31 08:42:31 EDT
See bug 823981 for a similar problem with cf_last_closed
Comment 6 Don Zickus 2012-05-31 09:41:30 EDT
Hi Jeff,

Reading the bugzilla4 code, it seems like opendate has been renamed to creation_ts.  Can you try that and see if it works?

Cheers,
Don
Comment 7 Jeff Bastian 2012-05-31 10:49:25 EDT
Ahh, in my frustration yesterday I missed that field.  Yes, that seems to work!  Thanks!  (Why, oh why, were all these field names changed?)

(And the format of the date changed from 'YYYY-mm-dd HH:MM:SS' to 'YYYY.mm.dd HH:MM'.  Argh.)


However, the creation_ts field is still not returned in the initial results.  I put a break right after making the query:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...
query['include_fields'] = [ 'id', 'component', 'creation_ts', 'summary' ]

print "%s Making query" % (timestamp())
results = BZ.query(query)
import pdb; pdb.set_trace()
...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


And there is no creation_ts included in the result:

(Pdb) results[0].__dict__
{'autorefresh': True, 'url': 'https://bugzilla.redhat.com/show_bug.cgi?id=72487', 'short_desc': 'RFE: Add support for disabling the duplicate IP check', 'component': 'initscripts', 'summary': 'RFE: Add support for disabling the duplicate IP check', 'bug_id': 72487, 'bugzilla': <bugzilla.rhbugzilla.RHBugzilla4 object at 0x12d20d0>, 'components': ['initscripts'], 'id': 72487}


It fills in the missing data when I try to use it:

(Pdb) results[0].creation_ts
'2002.08.24 09:25'


But the above command takes nearly a minute to run, and it returns gobs and gobs of data, much more than the one requested field.  The data returned initially has 9 entries in the dictionary:

(Pdb) len(results[0].__dict__)
9

But after getting creation_ts, it balloons to 119!

(Pdb) len(results[0].__dict__)
119


So why is it not returned initially even though it's specified in include_fields?
Comment 8 Don Zickus 2012-05-31 11:31:41 EDT
Hi Jeff,

Sorry about that, I didn't read the nuisances of the mozilla bugzilla4 code correctly (this is soooo annoying that I have to read the upstream code to understand what the xmlrpc is doing).  The field you want is

creation_time

which is translated to creation_ts in the backend of the bugzilla server which also looks for opendate.  But opendate doesn't seem to be allowed in xmlrpc, so I am not sure what it is hoping to translate.  *sigh*

That work in my quick testing, *without* the 50-60 second  Bug.field delay.

Cheers,
Don
Comment 10 Jeff Bastian 2012-05-31 12:01:31 EDT
creation_time does work much faster than creation_ts

It's strange that creation_time is not listed in BZ.bugfields as a valid field (see comment 4)


I'll re-work my patch.

Thank you for all the help!  I really appreciate it!
Comment 11 Aristeu Rozanski 2012-05-31 12:04:11 EDT
if it's not listed and it should, see _getbugfields() so you can add it in.
Comment 12 Aristeu Rozanski 2012-05-31 12:05:59 EDT
as reference for fields, you can look here:
http://www.bugzilla.org/docs/tip/en/html/api/Bugzilla/WebService/Bug.html
Comment 14 Jeff Bastian 2012-05-31 15:05:27 EDT
Changes to my test script from comment 1 include using creation_time in the include_fields along with some string manipulation on the creation_time to convert it to the old format:
  opendate format:      YYYY-mm-dd HH:MM:SS
  creation_time format: YYYYmmddTHH:MM:SS


--- bz-query.py.ORIG	2012-05-31 14:01:41.199815836 -0500
+++ bz-query.py	2012-05-31 14:02:47.178958589 -0500
@@ -38,7 +38,7 @@
 query['f1'] = 'keywords'
 query['o1'] = 'substring'
 query['v1'] = 'Patch'
-query['include_fields'] = [ 'id', 'component', 'opendate', 'summary' ]
+query['include_fields'] = [ 'id', 'component', 'creation_time', 'summary' ]
 
 print "%s Making query" % (timestamp())
 results = BZ.query(query)
@@ -46,5 +46,7 @@
 
 print "%s Printing report" % (timestamp())
 for r in results:
-    output = "%s %s %s %s" % (r.id, r.component, r.opendate, r.summary)
+    opendate = r.creation_time.value.replace('T', ' ')
+    opendate = opendate[:4] + "-" + opendate[4:6] + "-" + opendate[6:]
+    output = "%s %s %s %s" % (r.id, r.component, opendate, r.summary)
     print "%s %s" % (timestamp(), output)
Comment 15 Don Zickus 2012-05-31 22:01:58 EDT
Hi Jeff,

Thanks for those changes.  I'll try to pass them along to Will to get them into upstream python-bugzilla.

Cheers,
Don
Comment 16 Cole Robinson 2012-12-28 16:02:11 EST
Upstream python-bugzilla now aliases opendate -> creation_time and creation_ts -> creation_time, so there's only one value kept internally. I think this will solve the problems reported here.
Comment 17 Cole Robinson 2012-12-28 16:04:31 EST
*** Bug 823981 has been marked as a duplicate of this bug. ***
Comment 18 Cole Robinson 2013-02-16 10:35:30 EST
Fixed in python-bugzilla-0.8.0-1.fc19

Note You need to log in before you can comment on or make changes to this bug.