Bug 1175504 - Navigating to virtual machines explorer is slow on Large Environment (15k Total VMs, 11k Archived VMs)
Summary: Navigating to virtual machines explorer is slow on Large Environment (15k Tot...
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Performance
Version: 5.3.0
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: GA
: 5.6.0
Assignee: Keenan Brock
QA Contact: Dave Johnson
URL:
Whiteboard: ui:perf
Keywords:
Depends On:
Blocks: 1290209
TreeView+ depends on / blocked
 
Reported: 2014-12-17 21:33 UTC by Alex Krzos
Modified: 2017-12-05 15:21 UTC (History)
7 users (show)

(edit)
Clone Of:
: 1290209 (view as bug list)
(edit)
Last Closed: 2016-02-18 16:01:17 UTC


Attachments (Terms of Use)
Production logs for navigating to vm_infra explorer and expanding archived and orphaned vms (245.42 KB, application/zip)
2014-12-17 21:33 UTC, Alex Krzos
no flags Details

Description Alex Krzos 2014-12-17 21:33:20 UTC
Created attachment 970320 [details]
Production logs for navigating to vm_infra explorer and expanding archived and orphaned vms

Description of problem:
On a Global VMDB appliance which is receiving replicated records from 4 other environments now has over 15,000 VM Records mostly due to archived VMs the rendering time for vm_infra/explorer is too slow.

A large amount of time is spent rendering the page 

Version-Release number of selected component (if applicable):
5.3.1.10

How reproducible:
Always with this environment/database

Steps to Reproduce:
1. Observe page rendering time while navigating to Infrastructure->Virtual Machines
2. Can duplicate similar issue by expanding archived instances on the VM & Templates Accordion. 
3.

Actual results:
INFO -- : Completed 200 OK in 87072.7ms (Views: 259.9ms | ActiveRecord: 903.4ms)

>80s page rendering time with less than 1s in active record

Expected results:
<5s page rendering time


Additional info:
$ grep "select" -i production.* -c
production.expand_archived.log:30
production.expand_orphaned.log:1618
production.vm_explorer.log:1881

$ grep "cache" -i production.* -c
production.expand_archived.log:18
production.expand_orphaned.log:1594
production.vm_explorer.log:1712

When expanding the archived VMs, a large portion of time is spent immediately after this query:
CACHE (0.0ms)  SELECT COUNT(*) FROM "vms" WHERE ((vms.id IN (1000000002911,1000000000580, .....

N+1 queries when expanding orphaned vms portion of tree:
$ grep 'SELECT "storages".* FROM "storages" WHERE "storages"."id" =' production.expand_orphaned.log  -c
1588

Comment 2 Alex Krzos 2015-05-21 12:54:15 UTC
In my CFME scale environment with two 10k VM VMware providers attached, 20k vms total in the regional vmdb + some archived VMs (~200), I am seeing apache timeouts due to the rendering time of vm_infra page exceeding 2 minutes.

[----] I, [2015-05-21T12:30:19.560097 #10611:552080]  INFO -- : Completed 200 OK in 148212.3ms (Views: 625.1ms | ActiveRecord: 2951.2ms)

Comment 3 Dave Johnson 2016-02-18 16:01:17 UTC
This BZ is now over a year old, because of this, we are closing it.  If you feel like this is a mistake and it is still a issue, please reproduce it in the latest release and create a new BZ.


Note You need to log in before you can comment on or make changes to this bug.