Bug 968317 - [regression] Custom Search "flags(contains any of the strings)" doesn't work as expected
[regression] Custom Search "flags(contains any of the strings)" doesn't work ...
Status: CLOSED UPSTREAM
Product: Bugzilla
Classification: Community
Component: Query/Bug List (Show other bugs)
4.4
Unspecified Unspecified
high Severity high (vote)
: ---
: ---
Assigned To: Simon Green
tools-bugs
: Regression, Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-05-29 09:00 EDT by Ademar Reis
Modified: 2014-10-12 18:50 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-06-04 07:08:44 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ademar Reis 2013-05-29 09:00:21 EDT
Some of my queries stopped working after the upgrade to bugzilla 4.4. After some investigation, I figured out that there's something wrong with the "flags(contains any of the strings)" filter in the custom search / advanced features.

I believe this is also affecting the performance of bugzilla, because the queries are returning way more results than they should and we extensively use flags in many places.

Example of broken query and a fixed version by individually searching for the string:

- I I use two OR'd flags(contains the string)":

2 results as of now (CORRECT)

https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&f1=cf_devel_whiteboard&f2=flagtypes.name&f3=OP&f4=flagtypes.name&f5=flagtypes.name&j3=OR&list_id=1406381&o1=substring&o4=substring&o5=substring&query_format=advanced&v1=developer-ack-6.5&v4=devel_ack%3F&v5=pm_ack%3F

- If I use "flags(contains any of the strings)" for the same two strings:

17 results as of now (BROKEN), QUICK RESPONSE:

https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&f1=cf_devel_whiteboard&f2=flagtypes.name&f3=OP&j3=OR&list_id=1406385&o1=substring&o2=anywordssubstr&query_format=advanced&v1=developer-ack-6.5&v2=devel_ack%3F%2Cpm_ack%3F
Comment 1 Ademar Reis 2013-05-29 09:03:03 EDT
(In reply to Ademar de Souza Reis Jr. from comment #0)s)"

<snip>

> 
> 17 results as of now (BROKEN), QUICK RESPONSE:

Please ignore the "QUICK RESPONSE" in the line above, it's irrelevant.
Comment 2 Ademar Reis 2013-05-29 09:18:36 EDT
BTW, just discovered another workaround: if I use "flags(is equal to any of the strings)" instead of "flags(contains any of the strings)", it works as expected.

I've also noticed that my queries that use "flags(contains any of the strings)", when searching for rhel-6.5.0? and rhel-6.5.0+ combined, are all timing out.

So, again, this is probably affecting performance, because I doubt I'm the only one using "flags(contains any of the strings(rhel6.5.0+,rhel6.5.0?))" or something similar.
Comment 3 Simon Green 2013-05-29 10:17:54 EDT
This is an intentional change from upstream as part of the Bugzilla 4.4 upgrade. The upstream bug is https://bugzilla.mozilla.org/show_bug.cgi?id=828344

  -- simon
Comment 4 Ademar Reis 2013-05-29 11:47:41 EDT
(In reply to Simon Green from comment #3)
> This is an intentional change from upstream as part of the Bugzilla 4.4
> upgrade. The upstream bug is
> https://bugzilla.mozilla.org/show_bug.cgi?id=828344

What a mess... no wonder the updgrade broke plenty of queries and you're having such a hard time with the performance issues. :-(

It would be nice if you could:

1. Refuse to run queries that use "flags(contains any of strings)", or have a consistent behavior in such cases. The way they're working right now, as far as I can test, the filter is ignored *sometimes*, causing the queries to potentially return tons of results and consume lots of resources (that's my case: my queries are timing out).

2. Document such a change as a "known regression" (or not... if they're far too common).

But well, I'll just change my queries to use "is equal to any of the strings", which solves my problem for now.
Comment 5 Simon Green 2013-05-29 20:30:19 EDT
(In reply to Ademar de Souza Reis Jr. from comment #4)
> It would be nice if you could:
> 
> 1. Refuse to run queries that use "flags(contains any of strings)", or have
> a consistent behavior in such cases. The way they're working right now, as
> far as I can test, the filter is ignored *sometimes*

If you are getting two different results from the same query (baring actual changes), I would consider that bug, and one should be filed. The same query should always return the same results.

>, causing the queries to
> potentially return tons of results and consume lots of resources (that's my
> case: my queries are timing out).

The old way was a regression from Bugzilla 4.0 (and Red Hat Bugzilla 3.6). It was fixed upstream in Bugzilla 4.2.6, but we never took that release due to the release of Bugzilla 4.4 upstream.
 
> 2. Document such a change as a "known regression" (or not... if they're far
> too common).

Upstream didn't consider this important enough to put it in their release notes. And it didn't hit my radar due to me not using a query like this.
 
> But well, I'll just change my queries to use "is equal to any of the
> strings", which solves my problem for now.

:-)

  -- simon
Comment 6 Ademar Reis 2013-05-29 21:38:39 EDT
(In reply to Simon Green from comment #5)
> (In reply to Ademar de Souza Reis Jr. from comment #4)
> > It would be nice if you could:
> > 
> > 1. Refuse to run queries that use "flags(contains any of strings)", or have
> > a consistent behavior in such cases. The way they're working right now, as
> > far as I can test, the filter is ignored *sometimes*
> 
> If you are getting two different results from the same query (baring actual
> changes), I would consider that bug, and one should be filed. The same query
> should always return the same results.

The same query always returns the same results (AKA server timeout in my case). What appears to be different is the behavior depending on the terms used in the query. I didn't test it extensively enough, but if you search for bogus flags such as "flags(contains any of the strings(foo,bar))", it will return zero bugs, as one would expect. If you search for "flags(contains any of the strings(devel_ack?,pm_ack?))", you'll get all bugs, with or without such flags.
Comment 7 Simon Green 2013-05-29 23:04:03 EDT
(In reply to Ademar de Souza Reis Jr. from comment #6)
> will return zero bugs, as one would expect. If you search for
> "flags(contains any of the strings(devel_ack?,pm_ack?))", you'll get all
> bugs, with or without such flags.

I'm not seeing this. Can you please let me know the complete buglist.cgi URL and a bug number from the result that you wouldn't expect on the list, so I can investigate this.

Thanks.

  -- simon
Comment 8 Ademar Reis 2013-06-03 09:21:58 EDT
(In reply to Simon Green from comment #7)
> (In reply to Ademar de Souza Reis Jr. from comment #6)
> > will return zero bugs, as one would expect. If you search for
> > "flags(contains any of the strings(devel_ack?,pm_ack?))", you'll get all
> > bugs, with or without such flags.
> 
> I'm not seeing this. Can you please let me know the complete buglist.cgi URL
> and a bug number from the result that you wouldn't expect on the list, so I
> can investigate this.
> 

Second query from comment #0. Constrast it with the first query, as I reported.

And again, I'm afraid of the performance impact of lots of queries that used to work, now returning tons of results and therefore timing out. That's what happened to me: all my queries which filtered by rhel-6.5.0?,rhel-6.5.z? flags apparently started returning *ALL* bugs from the database that matched the other criteria and therefore time out before I could see anything. I wonder if I was the only one.
Comment 9 Simon Green 2013-06-03 18:49:29 EDT
(In reply to Ademar de Souza Reis Jr. from comment #0)
> 17 results as of now (BROKEN), QUICK RESPONSE:

Found the problem. Will file a bug upstream. Once they take it, we will probably do a hot fix, even if upstream don't.

  -- simon
Comment 10 Simon Green 2013-06-03 19:16:05 EDT
https://bugzilla.mozilla.org/show_bug.cgi?id=879055 is the upstream bug. I'm setting the private group on the bug since it mentions fields that are only available internally.

  -- simon
Comment 11 Simon Green 2013-06-04 07:08:44 EDT
Upstream have reviewed and accepted my patch. We will inherit it that way.

  -- simon

Note You need to log in before you can comment on or make changes to this bug.