Bug 734535

Summary: beaker-transfer query used on scheduler is inefficient.
Product: [Retired] Beaker Reporter: Bill Peck <bpeck>
Component: lab controllerAssignee: Bill Peck <bpeck>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 0.7CC: bpeck, dcallagh, mcsontos, rmancy, stl
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-10-20 06:19:08 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 703449    

Description Bill Peck 2011-08-30 17:37:55 UTC 
Description of problem:
beaker-transfer defaults to one day to know what it should transfer

need an optional argument to specify other than one day.
Comment 1 Bill Peck 2011-08-31 18:14:09 UTC 
Thinking about this some more we should fix this so we don't need to specify the days.  Lets fix the performance issue.

       recipes = Recipe\
                 .query().join(['status'])\
                         .outerjoin(['logs'])\
                         .outerjoin(['tasks','logs'])\
                         .filter(recipe_table.c.finish_time > timedifference)\
                         .filter(task_status_table.c.status.in_(finished))\
                         .filter(or_(log_recipe_table.c.server.like('%s%%' % server),
                                     log_recipe_task_table.c.server.like('%s%%' % server)))


the like statements are very expensive and the column server is a text column which prevents us from adding an index to it.

server column looks like this in the DB:

http://lab-devel.example.com/beaker/logs/recipes/3232

I think we should do the following:

1. rename column server to url
2. add column server with a fixed length (so we can add an index)
3. populate server with just the hostname (lab-devel.example.com)
4. change the query to use server and use = instead of like.
5. add a default limit of 50.  Everytime the script is run it will get the next batch
Comment 3 Raymond Mancy 2011-09-20 04:00:35 UTC 
All the logs from this job https://beaker-stage.app.eng.bos.redhat.com/jobs/3810
seem to be transferred just fine.
Comment 4 Raymond Mancy 2012-10-04 06:26:37 UTC 
*** Bug 703449 has been marked as a duplicate of this bug. ***