Bug 1378998

Summary: Slow VMWare EMS Refresh
Product: Red Hat CloudForms Management Engine Reporter: Colin Arnott <carnott>
Component: ProvidersAssignee: Adam Grare <agrare>
Status: CLOSED CURRENTRELEASE QA Contact: Alex Newman <anewman>
Severity: high Docs Contact:
Priority: high    
Version: 5.6.0CC: akrzos, benglish, cpelland, dajohnso, gblomqui, jfrey, jhardy, obarenbo, sacpatil, simaishi
Target Milestone: GAKeywords: TestOnly
Target Release: 5.8.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: provider:vsphere:ems_refresh
Fixed In Version: 5.8.0.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1399803 1399805 (view as bug list) Environment:
Last Closed: 2017-06-12 16:11:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: VMware Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1382163, 1399803, 1399805    
Attachments:
Description Flags
5.6.1.2 hotfix to retrieve all host.config.storageDevice in one API call
none
5.6.1.2_hotfix_host_storage_devices.tar.gz none

Description Colin Arnott 2016-09-23 20:44:43 UTC
Description of problem:
Related to 1378991, EMS refresh is slow causing automate methods to timeout when creating folders.

Additional info:
Providing a database dump

Comment 2 Adam Grare 2016-09-29 16:51:08 UTC
Looking at a typical full EmsRefresh using the broker cache, here are some notable durations for different steps:
Total duration:               21min 11sec
Getting cached inventory:           30sec
Getting Host Storage Devices: 11min 31sec

Storage devices for hosts are not stored in the broker cache so every refresh has to go out to the vCenter to get this information.  Currently this is done once for each host which incurs a lot of overhead.

Two possible solutions:
1. Keep host storage devices in broker cache
2. Retrieve storage devices for all hosts in one API call

Comment 3 Adam Grare 2016-09-29 19:14:05 UTC
Created attachment 1206036 [details]
5.6.1.2 hotfix to retrieve all host.config.storageDevice in one API call

Comment 4 Adam Grare 2016-10-04 12:35:37 UTC
PR https://github.com/ManageIQ/manageiq/pull/11629

Comment 9 Adam Grare 2016-10-13 13:25:33 UTC
Created attachment 1210091 [details]
5.6.1.2_hotfix_host_storage_devices.tar.gz

cherry picked from commit 59f755fe421983761bc48d3162f79b109126c4a6

tar tfz 5.6.1.2_hotfix_host_storage_devices.tar.gz 
app/models/ems_refresh/vc_updates.rb
app/models/manageiq/providers/vmware/infra_manager/refresher.rb
gems/pending/VMwareWebService/MiqVimInventory.rb

Comment 13 Adam Grare 2016-11-29 18:32:34 UTC
*** Bug 1341402 has been marked as a duplicate of this bug. ***

Comment 14 Adam Grare 2016-11-29 18:32:47 UTC
*** Bug 1353707 has been marked as a duplicate of this bug. ***

Comment 17 Adam Grare 2017-01-30 14:46:09 UTC
*** Bug 1382163 has been marked as a duplicate of this bug. ***