Bug 458223 - mod_perl & buglist queries OOM apache
mod_perl & buglist queries OOM apache
Status: CLOSED CURRENTRELEASE
Product: Bugzilla
Classification: Community
Component: Query/Bug List (Show other bugs)
3.2
All Linux
medium Severity medium (vote)
: ---
: ---
Assigned To: David Lawrence
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-08-06 23:33 EDT by Kevin Baker
Modified: 2014-12-01 18:09 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-11-24 15:31:58 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Patch to limit amount of rows returned to the user in buglist.cgi and webservices (v1) (8.37 KB, patch)
2008-08-08 14:36 EDT, David Lawrence
no flags Details | Diff
buglist.cgi with KBakers take (49.25 KB, text/plain)
2008-08-08 14:57 EDT, Kevin Baker
no flags Details
Patch to limit amount of rows returned to the user in buglist.cgi and webservices (v1) (7.32 KB, patch)
2008-08-08 16:13 EDT, David Lawrence
kbaker: review-
Details | Diff

  None (edit)
Description Kevin Baker 2008-08-06 23:33:55 EDT
Description of problem:

The webserver is getting hit with calls to buglist.cgi and xmlrpc.cgi that are requesting *all* bugs in bugzilla. This causes apache to balloon to 1.1 Gigs, and we've had reports of processes hitting 6.5Gig (not confirmed if these were buglist.cgi calls)


How reproducible:

https://bz-web2-test.devel.redhat.com/buglist.cgi?query_format=advanced
  
Actual results:

Browser collapses under weight of returned content

Expected results:

Some rational limits are put in place to stop abuse of the web server. Does anyone need to retrieve over 4000 records at a time? Perhaps in xmlrpc but not in buglist. The web page could implement some sort of paging at least. 

Additional information:

- perhaps a standalone xmlrpc server could be put in place.
Comment 1 David Lawrence 2008-08-07 17:58:56 EDT
Taking
Comment 2 David Lawrence 2008-08-08 14:36:01 EDT
Created attachment 313830 [details]
Patch to limit amount of rows returned to the user in buglist.cgi and webservices (v1)

Patch added to limit the number of rows returned to either the web UI or the XMLRPC query methods.

The number is currently 20000 but can be configured in data/params.

I am also adding extra logging lines to display the actual number of rows returned so we can look in the logs to see how often users are getting this error.

Please review
Dave
Comment 3 Kevin Baker 2008-08-08 14:57:39 EDT
Created attachment 313833 [details]
buglist.cgi with KBakers take 

Hi Dave,

I thought you would have put the exit as soon as the query had run so we don't waste anymore memory on reading it into the @buglist @bugid arrays. Here's my take on it.

I like having the limit set via parameter. Makes it easy to tweak
Comment 4 Kevin Baker 2008-08-08 14:59:28 EDT
(In reply to comment #2)

> The number is currently 20000 but can be configured in data/params.

Hrm, I just had a thought that we may want to differentiate between buglist and xmlrpc limits. We may let larger amounts go thru xmlrpc since it doesn't go thru Template toolkit?
Comment 5 David Lawrence 2008-08-08 16:13:21 EDT
Created attachment 313846 [details]
Patch to limit amount of rows returned to the user in buglist.cgi and webservices (v1)

New patch:

Added new buglistrowlimitxmlrpc param for webservice methods so that they can be tuned higher if needed.

Also move the ThrowUserError() earlier since MySQL can actually count the rows selected before they are actually fetched.

Please review.
Dave
Comment 6 David Lawrence 2008-08-08 16:56:40 EDT
Pushed attachment 313846 [details] to live servers so we will see how much useful information we get.
Comment 7 Max Kanat-Alexander 2008-09-03 19:51:38 EDT
Note that extensions should not be using incrementally higher error numbers in Bugzilla::WebService::Constants, as that will conflict with future error numbers. You should add 10000 to your error numbers.
Comment 8 Kevin Baker 2008-09-04 11:20:04 EDT
Comment on attachment 313846 [details]
Patch to limit amount of rows returned to the user in buglist.cgi and webservices (v1)

Looks ok. NAK'd it on the basis of Max's comment.
Comment 9 David Lawrence 2008-09-04 14:11:42 EDT
Updated the fault code to be 10000 in our CVS tree. Should show up in the next scheduled update.
Comment 10 Max Kanat-Alexander 2008-09-04 18:33:24 EDT
I'd also be interested in some input upstream on what exactly is causing your OOMs, particularly the 6GB one (normal buglist.cgi should never never get that large, even with 500,000 bugs).
Comment 11 David Lawrence 2008-11-24 15:31:58 EST
I think this can be closed now as we figured out the performance issue and have remedied it. We will switch over to mod_perl in the near future once we get some other issue resolved non-specific to Bugzilla.

Note You need to log in before you can comment on or make changes to this bug.