Bug 734535 - beaker-transfer query used on scheduler is inefficient.
Summary: beaker-transfer query used on scheduler is inefficient.
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Community
Component: lab controller
Version: 0.7
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified vote
Target Milestone: ---
Assignee: Bill Peck
QA Contact:
URL:
Whiteboard:
: 703449 (view as bug list)
Depends On:
Blocks: 703449
TreeView+ depends on / blocked
 
Reported: 2011-08-30 17:37 UTC by Bill Peck
Modified: 2019-05-22 13:40 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-10-20 06:19:08 UTC


Attachments (Terms of Use)

Description Bill Peck 2011-08-30 17:37:55 UTC
Description of problem:
beaker-transfer defaults to one day to know what it should transfer

need an optional argument to specify other than one day.

Comment 1 Bill Peck 2011-08-31 18:14:09 UTC
Thinking about this some more we should fix this so we don't need to specify the days.  Lets fix the performance issue.

       recipes = Recipe\
                 .query().join(['status'])\
                         .outerjoin(['logs'])\
                         .outerjoin(['tasks','logs'])\
                         .filter(recipe_table.c.finish_time > timedifference)\
                         .filter(task_status_table.c.status.in_(finished))\
                         .filter(or_(log_recipe_table.c.server.like('%s%%' % server),
                                     log_recipe_task_table.c.server.like('%s%%' % server)))


the like statements are very expensive and the column server is a text column which prevents us from adding an index to it.

server column looks like this in the DB:

http://lab-devel.example.com/beaker/logs/recipes/3232

I think we should do the following:

1. rename column server to url
2. add column server with a fixed length (so we can add an index)
3. populate server with just the hostname (lab-devel.example.com)
4. change the query to use server and use = instead of like.
5. add a default limit of 50.  Everytime the script is run it will get the next batch

Comment 3 Raymond Mancy 2011-09-20 04:00:35 UTC
All the logs from this job https://beaker-stage.app.eng.bos.redhat.com/jobs/3810
seem to be transferred just fine.

Comment 4 Raymond Mancy 2012-10-04 06:26:37 UTC
*** Bug 703449 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.