Bug 534935 (RHQ-1683)

Summary:	performance test tracking
Product:	[Other] RHQ Project	Reporter:	Joseph Marques <jmarques>
Component:	Configuration	Assignee:	Ian Springer <ian.springer>
Status:	CLOSED NEXTRELEASE	QA Contact:
Severity:	medium	Docs Contact:
Priority:	medium
Version:	1.2	CC:	ccrouch
Target Milestone:	---	Keywords:	SubTask
Target Release:	---
Hardware:	All
OS:	All
URL:	http://jira.rhq-project.org/browse/RHQ-1683
Whiteboard:
Fixed In Version:	1.2	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	534605

Description Joseph Marques 2009-02-27 19:50:00 UTC

I've updated the service-a Resource type in the perftest plugin to expose a fairly large Resource config that contains all of the various types of properties (string simple, integer simple, open map, list of maps, etc.). To performance test, a perftest scenario that deploys 5-20 service-a Resources should be used. The perftest plugin should be deployed to multiple Agents (5-20) - either agentspawn Agents, or ideally, Agents running on different machines (perhaps the perf environment that Charles has setup can be used). Start up the Server and all Agents, import the perftest Resources, and create a compatible group "BigCompatGroup" that contains all of the service-a Resources (the group should contain 100 members, since this is the maximum we intend to support for group config). Once this environment is up and running, the following scenarios should be tested:

A) Loading
************
go to BigCompatGroup's Configure tab and see how long it takes for the group config to be loaded and displayed (record how long it takes). It should take no more than 20 seconds (if it does take more than 20 seconds, you should get a timeout error in the GUI).

B) Updating
*************
make edits to several properties (set some to homogeneous member values and others to heterogeneous), then click the SAVE button to update the config. Keep refreshing the Configure>History subtab until the group config update completes. Verify it completes with a SUCCESS status and record how long it takes to complete. Ideally, it shouldn't take more than 20 seconds.

Retest the above scenarios in an environment where the DB is on a different machine than the Server.

Comment 1 Ian Springer 2009-03-03 16:10:44 UTC

I've also written up the perf test scenarios on the DEV wiki at http://jon01.qa.atl2.redhat.com:8080/confluence/display/DEV/Performance+Testing. Please update this page with any tweaks made as the procedure is actually carried out.

Comment 2 Ian Springer 2009-03-04 04:43:19 UTC

r3287 enhances the perftest plugin so it can be used for group resource config testing. Specifically, the service-a Resource type now exposes a decent-sized Resource config.

Comment 3 Ian Springer 2009-03-16 22:33:56 UTC

Ok, after numerous diversions, I've run the above scenarios on jon03, whose DB is on a different server. Specifically, I used agentspawn to spawn 50 Agents each with 2 server-a servers with 2 service-a services each, and then added all 100 service-a Resources to a compat group.

A) It took 10 seconds to load the view/edit Resource config pages for the group. 
B) It took 21 seconds for the update of the group's Resource config to complete - it completed successfully. This is actually good considering I added a Thread.sleep(1000) to the updateConfiguration() method of the service-a ResourceComponent. The fact that the updates finished in < 100 seconds shows that end-to-end concurrency was working nicely.

I think these numbers are acceptable for x.2. I'm sure the performance can be further tweaked if we want to improve it for a later release.

Comment 4 Red Hat Bugzilla 2009-11-10 20:38:11 UTC

This bug was previously known as http://jira.rhq-project.org/browse/RHQ-1683