Bug 1062480

Summary: Queries in beakerd processing loops are too slow
Product: [Retired] Beaker Reporter: Raymond Mancy <rmancy>
Component: schedulerAssignee: Raymond Mancy <rmancy>
Status: CLOSED CURRENTRELEASE QA Contact: tools-bugs <tools-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 0.15CC: aigao, asaha, dcallagh, ebaak, llim, qwan, rmancy, xjia
Target Milestone: 0.15.4   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-02-18 01:38:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Raymond Mancy 2014-02-07 02:55:50 UTC
Description of problem:

On large installations, the beakerd processing loops can be quite slow

Version-Release number of selected component (if applicable):


How reproducible:

Always on a large beaker installation

Steps to Reproduce:
1. Set the slow query log to log anything over 15 seconds
2. Queries from beakerd processing loops turn up there
3.

Actual results:

These kind of queries show up:

SELECT count(1) AS count_1 
FROM recipe_set INNER JOIN job ON job.id = recipe_set.job_id 
WHERE job.dirty_version = job.clean_version AND NOT (EXISTS (SELECT 1 
FROM recipe 
WHERE recipe_set.id = recipe.recipe_set_id AND recipe.status != 'Scheduled'));

Expected results:

Not to be there

Additional info:

A simple fix is to add job.deleted IS NULL to the WHERE clause. In my testing, that has brought down the query times from ~15-20 seconds to 4-6 seconds.

Comment 2 Nick Coghlan 2014-02-07 04:37:36 UTC
Target will be to get these queries back to subsecond execution times in 0.16, but a 3x improvement is acceptable for 0.15.4.

Comment 3 Raymond Mancy 2014-02-07 08:02:17 UTC
Just to clarify, those subsecond execution times used to exist when we had top down status updates, and the query was much simpler than then kind of query we now require. 

That's not to say that they cannot be improved upon further than what BZ#1062480 will do, but unfortunately there is no prior query that we can revert back to, to restore the performance we would like.

Comment 4 Raymond Mancy 2014-02-12 10:59:56 UTC
Gerrit review,

http://gerrit.beaker-project.org/#/c/2798/

Comment 7 xjia 2014-02-14 10:18:00 UTC
After smoke test, QE didn't find regression bug for this issue. So move to "VERIFIED"

Comment 8 Dan Callaghan 2014-02-18 01:38:51 UTC
Beaker 0.15.4 has been released.