Bug 1306626

Summary:	[scale] Test DWH scalability with lower interval collection rate (to improve monitoring for System Dashboard)
Product:	[oVirt] ovirt-engine-dwh	Reporter:	Shirly Radco <sradco>
Component:	ETL	Assignee:	Shirly Radco <sradco>
Status:	CLOSED NOTABUG	QA Contact:	mlehrer
Severity:	high	Docs Contact:
Priority:	high
Version:	3.6.5	CC:	bugs, gklein, mlehrer, ylavi
Target Milestone:	ovirt-3.6.7	Keywords:	TestOnly
Target Release:	3.6.6	Flags:	rule-engine: ovirt-3.6.z+ ylavi: planning_ack+ rule-engine: devel_ack+ gklein: testing_ack+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-06-15 10:52:31 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Metrics	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Shirly Radco 2016-02-11 13:23:45 UTC

Please test this bug using the private build from bug #1285788 in DEBUG mode.
Please set samples collection for 15 seconds.

# Samples Collection Interleave in Seconds
DWH_SAMPLING=15

In an environment with 200 hosts, 10,000 vms and 2 disks for each vm.

1.How long the samples job takes and if it finishes under 15 seconds.
  If job takes more than 15 second please increase by 5 second and try again.
  Do this until job finishes before next run and report what id the refresh rate    
possible.

2.Does environment stays stable.

Comment 1 Gil Klein 2016-04-18 10:13:40 UTC

Still in progress, we will continue this effort as part of 3.6.6

Comment 2 Yaniv Lavi 2016-05-24 11:19:30 UTC

Please test with this amount of entities and provide the samples interval performance:

vms_disks: 6000
vms: 3000
hosts: 300
vms_interfaces: 6000
hosts_interfaces: 3000

Comment 3 mlehrer 2016-06-07 12:16:43 UTC

Results showed increased Statistics Sync time, increased errors of "unable to connect to engine", and ovirt_engine_history growth increase of 9GB to 32GB 

Details available: https://mojo.redhat.com/docs/DOC-1082713