Bug 1811869

Summary:	[Scale] Webadmin\REST for host interface list response time is too long because of excessive amount of qos related sql queries
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	eraviv
Component:	ovirt-engine	Assignee:	eraviv
Status:	CLOSED ERRATA	QA Contact:	mlehrer
Severity:	medium	Docs Contact:
Priority:	medium
Version:	4.2.0	CC:	dagur, dholler, mlehrer, mperina, mtessun
Target Milestone:	ovirt-4.4.0	Keywords:	Performance
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	rhv-4.4.0-28	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-08-04 13:21:58 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Network	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description eraviv 2020-03-10 02:12:57 UTC

Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. create 95 host environment with 300 vlans on one host interface (non mgmt)
2. Setup: 
- No REST calls
- VdsRefreshRate=default
- Host in UP status
3. let engine run idle for 15 minutes after startup
4. measure load time of:
- webadmin host interface list view page
- 'curl <engine>/ovirt-engine/api/hosts/<host>/nics'

Actual results:
webadmin freezes, REST call takes long minutes to return

Expected results:
no more than a few seconds to complete both flows

Additional info:
https://docs.google.com/document/d/1-Mv2JjDkV2gKiXsomVKCtMf5r4TZbxjiFRtbygBd-fU/edit?usp=sharing

Comment 1 eraviv 2020-03-10 02:20:43 UTC

typo in description. Should be:

Expected results: approximately 2 minutes to complete load of web page and get REST response

Comment 3 mlehrer 2020-05-11 14:29:04 UTC

4.4.0-0.31.master.el8
HE environment with 200 nested hosts of which 150 hosts have 100 networks per host.
DWH separated and JVM set to 4G with engine set with with 200 pool connections / 250 db connections.



# /usr/share/ovirt-engine/dbscripts/engine-psql.sh -c ""SELECT count(*) from vds_interface_view""
 count 
-------
 15825
(1 row)"



+-------------------------------------------------------------------------------------+-------------------+------------------+
|         150 Hosts / 100 networks per host                                           |      Firefox      |      Chrome      |
+-------------------------------------------------------------------------------------+-------------------+------------------+
| UI click Hosts                                                                      | 5s                | 6s               |
| UI select specific host                                                             | 1s                | 1s               |
| UI View list of Interface                                                           | 1s                | 2s               |
| UI Expand interfaces of host                                                        | 1s                | 1s               |
| UI Click Events tab on hosts                                                        | 1s                | 2s               |
| UI return to Hosts view from host detail view                                       | 5s                | 6s               |
| UI Load Dashboard Page from Hosts                                                   | 1s                | 4s               |
| UI Setup Host networks                                                              | 12s               | 14s              |
|                                                                                     |                   |                  |
|                                                                                     | client across WAN | client on engine |
| API /ovirt-engine/api/hosts                                                         | 10s               | 0.5s             |
| API /ovirt-engine/api/hosts/2ece4045-d94d-4f25-afd7-f6966e09ec12/networkattachments | 6s                | 0.3s             |
| API /ovirt-engine/api/hosts/2ece4045-d94d-4f25-afd7-f6966e09ec12/nics               | 4s                | 1.1s             |
+-------------------------------------------------------------------------------------+-------------------+------------------+

Measurements used hosts which had 100, and another host with had 200 networks per interface.

Comment 6 errata-xmlrpc 2020-08-04 13:21:58 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: RHV Manager (ovirt-engine) 4.4 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:3247