Bug 1811869 - [Scale] Webadmin\REST for host interface list response time is too long because of excessive amount of qos related sql queries
Summary: [Scale] Webadmin\REST for host interface list response time is too long becau...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ovirt-4.4.0
: ---
Assignee: eraviv
QA Contact: mlehrer
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-10 02:12 UTC by eraviv
Modified: 2020-08-04 13:22 UTC (History)
5 users (show)

Fixed In Version: rhv-4.4.0-28
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-08-04 13:21:58 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2020:3247 0 None None None 2020-08-04 13:22:36 UTC
oVirt gerrit 105925 0 master MERGED core: AttachmentDao - host: test less QoS dao calls 2020-05-26 00:51:45 UTC
oVirt gerrit 106072 0 master MERGED core: network_sp.sql - get by nic+net with QoS 2020-05-26 00:51:45 UTC
oVirt gerrit 106073 0 master MERGED core: AttachmentDao - get by nic+net: less QoS dao calls 2020-05-26 00:51:45 UTC
oVirt gerrit 106074 0 master MERGED core: network_sp.sql: remove unused stored procedure 2020-05-26 00:51:45 UTC

Description eraviv 2020-03-10 02:12:57 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. create 95 host environment with 300 vlans on one host interface (non mgmt)
2. Setup: 
- No REST calls
- VdsRefreshRate=default
- Host in UP status
3. let engine run idle for 15 minutes after startup
4. measure load time of:
- webadmin host interface list view page
- 'curl <engine>/ovirt-engine/api/hosts/<host>/nics'

Actual results:
webadmin freezes, REST call takes long minutes to return

Expected results:
no more than a few seconds to complete both flows

Additional info:
https://docs.google.com/document/d/1-Mv2JjDkV2gKiXsomVKCtMf5r4TZbxjiFRtbygBd-fU/edit?usp=sharing

Comment 1 eraviv 2020-03-10 02:20:43 UTC
typo in description. Should be:

Expected results: approximately 2 minutes to complete load of web page and get REST response

Comment 3 mlehrer 2020-05-11 14:29:04 UTC
4.4.0-0.31.master.el8
HE environment with 200 nested hosts of which 150 hosts have 100 networks per host.
DWH separated and JVM set to 4G with engine set with with 200 pool connections / 250 db connections.



# /usr/share/ovirt-engine/dbscripts/engine-psql.sh -c ""SELECT count(*) from vds_interface_view""
 count 
-------
 15825
(1 row)"



+-------------------------------------------------------------------------------------+-------------------+------------------+
|         150 Hosts / 100 networks per host                                           |      Firefox      |      Chrome      |
+-------------------------------------------------------------------------------------+-------------------+------------------+
| UI click Hosts                                                                      | 5s                | 6s               |
| UI select specific host                                                             | 1s                | 1s               |
| UI View list of Interface                                                           | 1s                | 2s               |
| UI Expand interfaces of host                                                        | 1s                | 1s               |
| UI Click Events tab on hosts                                                        | 1s                | 2s               |
| UI return to Hosts view from host detail view                                       | 5s                | 6s               |
| UI Load Dashboard Page from Hosts                                                   | 1s                | 4s               |
| UI Setup Host networks                                                              | 12s               | 14s              |
|                                                                                     |                   |                  |
|                                                                                     | client across WAN | client on engine |
| API /ovirt-engine/api/hosts                                                         | 10s               | 0.5s             |
| API /ovirt-engine/api/hosts/2ece4045-d94d-4f25-afd7-f6966e09ec12/networkattachments | 6s                | 0.3s             |
| API /ovirt-engine/api/hosts/2ece4045-d94d-4f25-afd7-f6966e09ec12/nics               | 4s                | 1.1s             |
+-------------------------------------------------------------------------------------+-------------------+------------------+

Measurements used hosts which had 100, and another host with had 200 networks per interface.

Comment 6 errata-xmlrpc 2020-08-04 13:21:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: RHV Manager (ovirt-engine) 4.4 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:3247


Note You need to log in before you can comment on or make changes to this bug.