I am unable to get this to fail using my own test script with your query parameters. I consistently get back around 4k+ bugs.

The AttributeError message seems like a Python error being generated. The 'blockedby' key name should still work as well with the latest code.

Can you speak with the Python Bugzilla module maintainer to help track down this issue?

Dave

Either tue to the way the xmlrpc interface works, or the way python-bugzilla works (honestly I'm not certain), the query won't actually fail regardless of what fields you ask for.  However, when you go to access the returned bug objects, you'll get an error if you try to look at the blockedby field.

Actually, it looks like when you attempt to access that field (with the 3.4 instance only) python-bugzilla ends up doing another xmlrpc call to attempt to fetch that data, which indicates that it's just not being returned.

Is there any way to get a list of fields that you can actually query for?  Is it possible that 'blockedby' is simply no longer a valid field?

CC'ing wwoods; perhaps he has some idea of what might be going on.

Well I do not see what is going on but have not yet found the solution. What is happening is that in BZ3.4 if the column is undef or empty then it is omitted from the returned data where in BZ3.2 it still returned the empty data. So we can either do 1 or both of 2 things. Have the python-bugzilla code or your script first check to see if the field is defined before using it, or get BZ3.4 to go back to the old behaviour. We may have to do the latter anyway since it probably
breaks other people's scripts somewhere else as well.

I will see what I can do.

Dave

I'll try to fix up my script to check and see how that goes.  I have a feeling that this isn't the only field with a change in behavior, unfortunately.  I'm not sure I can actually fix things above the python-bugzilla level, though, because just attempting to look at fields that weren't returned as a result of the query results in xmlrpc calls back to bugzilla.  Doing thousands of those would not be good for anyone.

I saw the "public beta 2" notice so I figured I'd give my scripts a try (without attempting to hack around the potential unavailability of some of the data), but now I get this backtrace-within-a-backtrace when my initial query runs.  I suppose it might not be related at all; let me know if I should open a separate bug.

Traceback (most recent call last):
  File "./review-stats.py", line 327, in <module>
    (bugs, bugdata) = run_query(bz)              
  File "./review-stats.py", line 111, in run_query
    bugs = filter(lambda b: str(b.bug_id) not in trackers, bz.query(querydata))
  File "/usr/lib/python2.6/site-packages/bugzilla/base.py", line 502, in query 
    r = self._query(query)                                                     
  File "/usr/lib/python2.6/site-packages/bugzilla/rhbugzilla.py", line 414, in _query
    return self._proxy.Bug.search(query)                                             
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__                   
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.6/site-packages/bugzilla/base.py", line 957, in request
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault -32000: 'DBD::mysql::st execute failed: FUNCTION bugs.AliasList does not exist [for Statement "SELECT bugs.bug_id AS bug_id, map_products.name AS product, bugs.creation_ts AS opendate, bugs.delta_ts AS changeddate, bugs.bug_severity AS bug_severity, AliasList(bugs.bug_id) AS alias, map_assigned_to.realname AS assigned_to, map_reporter.realname AS reporter, bugs.bug_status AS bug_status, bugs.resolution AS resolution, map_components.name AS component, BlockedByList(bugs.bug_id) AS blockedby, DependsOnList(bugs.bug_id) AS dependson, bugs.short_desc AS short_desc, bugs.status_whiteboard AS status_whiteboard, bugs.priority AS priority FROM bugs  INNER JOIN profiles AS map_assigned_to ON (bugs.assigned_to = map_assigned_to.userid) INNER JOIN profiles AS map_reporter ON (bugs.reporter = map_reporter.userid) INNER JOIN products AS map_products ON (bugs.product_id = map_products.id) INNER JOIN components AS map_components ON (bugs.component_id = map_components.id) LEFT JOIN flags AS flags_0 ON (bugs.bug_id = flags_0.bug_id ) LEFT JOIN flagtypes AS flagtypes_0 ON (flags_0.type_id = flagtypes_0.id) LEFT JOIN bug_status ON (bug_status.value = bugs.bug_status) LEFT JOIN priority ON (priority.value = bugs.priority) LEFT JOIN bug_group_map  ON bug_group_map.bug_id = bugs.bug_id  WHERE (( bugs.component_id IN (18186,82671,82871) ) AND ( bugs.product_id IN (49) )) AND ((0=0)) AND bugs.creation_ts IS NOT NULL AND ((bug_group_map.group_id IS NULL) OR (bug_group_map.group_id IN (-1))) GROUP BY bugs.bug_id HAVING SUM(CASE WHEN CONCAT(flagtypes_0.name, flags_0.status) IS NOT NULL THEN 1 ELSE 0 END) = SUM(CASE WHEN CONCAT(flagtypes_0.name, flags_0.status) NOT REGEXP \'fedora-review[-+?]\' THEN 1 ELSE 0 END) ORDER BY bug_status.sortkey,bug_status.value,priority.sortkey,priority.value,assigned_to,bug_id"] at Bugzilla/WebService/Bug.pm line 742\n\tBugzilla::WebService::Bug::search(\'Bugzilla::WebService::Bug\', \'HASH(0x11b92980)\') called at /usr/lib/perl5/vendor_perl/5.8.8/SOAP/Lite.pm line 2559\n\teval {...} called at /usr/lib/perl5/vendor_perl/5.8.8/SOAP/Lite.pm line 2547\n\teval {...} called at /usr/lib/perl5/vendor_perl/5.8.8/SOAP/Lite.pm line 2516\n\tSOAP::Server::handle(\'Bugzilla::WebService::Server::XMLRPC=HASH(0x111d5700)\', \'<?xml version=\\\'1.0\\\'?>\\x{a}<methodCall>\\x{a}<methodName>Bug.search</...\') called at /usr/lib/perl5/vendor_perl/5.8.8/SOAP/Transport/HTTP.pm line 327\n\tSOAP::Transport::HTTP::Server::handle(\'Bugzilla::WebService::Server::XMLRPC=HASH(0x111d5700)\') called at /usr/lib/perl5/vendor_perl/5.8.8/SOAP/Transport/HTTP.pm line 423\n\tSOAP::Transport::HTTP::CGI::handle(\'Bugzilla::WebService::Server::XMLRPC=HASH(0x111d5700)\') called at /var/www/html/bugzilla/xmlrpc.cgi line 47\n'>

Sorry bout that. Some of the stored procs did not get restored properly when we refreshed the data on partner-bugzilla.redhat.com. This should be fixed now so please try your testing again.

Dave

Just to make it clear: the bad stuff in comment 6 is indeed cleared up, but the original problem still exists and unfortunately I'm not sure how to work around it in my code.  I either need to hack python-bugzilla somehow or drop back to doing the xmlrpc stuff myself, if I can remember how to do that.

Not that it's on-topic here, but I don't see how to tell via the exposed python-bugzilla interface that the server failed to return anything for a particular field.  All I can do is query the field, which will cause a backtrace.

Created attachment 379602 [details]
v1 of patch to fix problem with null multi-value columns in xmlrpc search 

Hi Dave,

attached is patch that will always add empty string for any NULL value columns returned from the search query and was required in the display column of the xmlrpc search call. Please let me know if it looks good to you.

Thanks,
Noura

Comment on attachment 379602 [details]
v1 of patch to fix problem with null multi-value columns in xmlrpc search 

Works for me for now. Ideally in the future it would be best to look at each column type returned and return the proper empty value. For example for a multi_select column type, then return an empty list [] and for a scalar type return a empty scalar. But for now this fixes the regression and the return results looks the same as 3.2.

Dave

Thanks for the review Dave. committed to svn.

Noura

I can verify that my scripts are currently working with partner-bugzilla. Thanks so much for working this out.