Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 641385

Summary:	[scheduler]: Setups with lots of images/realms/providers/provider accounts may be slow
Product:	[Retired] CloudForms Cloud Engine	Reporter:	Chris Lalancette <clalance>
Component:	aeolus-conductor	Assignee:	Mo Morsi <mmorsi>
Status:	CLOSED CURRENTRELEASE	QA Contact:	wes hayutin <whayutin>
Severity:	medium	Docs Contact:
Priority:	low
Version:	1.0.0	CC:	cpelland, dajohnso, deltacloud-maint, mmorsi
Target Milestone:	alpha
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-02-09 17:58:11 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Chris Lalancette 2010-10-08 15:13:43 UTC

Description of problem:
Because we are doing several nested loops in condormatic_classads_sync, we end up doing tons of individual SQL queries to get at all of the information in the aggregator database.  If many images/realms/providers/provider_accounts, etc, are defined, this time can add up and take quite a long time to do the sync.  Instead of nested loops we should really do a large join operation to fetch all of the data at once.  While JOINs do take some time, it should definitely be faster than "death by a thousand cuts" of individual SQL queries.

Comment 1 Mo Morsi 2010-11-11 19:19:25 UTC

If I understand whats going on here, we are collecting all tuples of images/realms/hardware profiles/replicated_images for all providers in condormatic_classads_sync and are advertising a UPDATE_STARTD_AD to each.

While we could obtain this via a db cross or outer join, AFAIK in rails / activerecord this is only possible to a limited extend, as the db queries you can perform and data you receive back is pretty tightly tied to your model classes and the relationships tied there.

Regardless I submitted a patch optimizing this a bit, it removes the calls to the sql queries from the nested loops, instead it performs the queries once, assembles the result, and just iterates over that.

https://fedorahosted.org/pipermail/deltacloud-devel/2010-November/003215.html

Comment 2 Dave Johnson 2011-07-29 18:26:46 UTC

Everything still works from what I can tell, is there any specific I should be checking for?  Basically I added a bunch of provider accounts, realms, hwp and launched several images.

Comment 4 Mo Morsi 2011-12-08 14:58:02 UTC

Hrm not sure, full verification might need to wait till we have full-on performance testing w/ aeolus. Regardless if everything still works, I'd be happy w/ just closing this out for the time being.

Comment 5 wes hayutin 2012-01-16 21:53:52 UTC

perf bug.. do not close w/o whayutin's ack

Comment 7 wes hayutin 2012-02-09 17:58:11 UTC

will flesh this out w/ perf testing.. closing