Bug 1293011 - beaker-log-delete takes an unreasonable amount of CPU and memory to delete very large jobs
Summary: beaker-log-delete takes an unreasonable amount of CPU and memory to delete ve...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Retired
Component: general
Version: 21
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: 23.0
Assignee: Roman Joost
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-12-19 02:57 UTC by Dan Callaghan
Modified: 2016-07-07 23:09 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-07 23:09:53 UTC
Embargoed:


Attachments (Terms of Use)

Description Dan Callaghan 2015-12-19 02:57:36 UTC
Related to bug 1293007, we are trying to recover from a job which contained two recipes which accidentally reported a very large number of results in an infinite loop (31457 total results in each recipe, 63543 total result logs in each recipe). beaker-log-delete struggles to delete such a large job, because it has load the entire job object graph into memory, including N db roundtrips for each result's logs. For example, right now it has spent 49 CPU minutes and consumed 1511MB virtual, 361MB resident in trying to load the large job and I'm not sure how much longer until it will need...

beaker-log-delete does allow applying a limit on the total number of jobs to delete in one run, but that doesn't help when there is a single job that is very large.

It needs to be smarter about loading the logs, ideally in batches, without requiring the entire job object graph to be fetched from the database. Bonus points if it can query all the logs in O(1) instead of O(N) db roundtrips.

Comment 1 Roman Joost 2016-01-08 01:49:17 UTC
First patch is up:

https://gerrit.beaker-project.org/#/c/4574/1

Comment 2 Roman Joost 2016-03-23 00:30:27 UTC
Dan has fixed regressions found when working on this and his patch on gerrit was merged:

https://gerrit.beaker-project.org/#/c/4719/

Comment 5 Dan Callaghan 2016-07-07 23:09:53 UTC
Beaker 23.0 has been released.


Note You need to log in before you can comment on or make changes to this bug.