Bug 1378998 - Slow VMWare EMS Refresh
Summary: Slow VMWare EMS Refresh
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: 5.6.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: GA
: 5.8.0
Assignee: Adam Grare
QA Contact: Alex Newman
URL:
Whiteboard: provider:vsphere:ems_refresh
: 1341402 1353707 1382163 (view as bug list)
Depends On:
Blocks: 1382163 1399803 1399805
TreeView+ depends on / blocked
 
Reported: 2016-09-23 20:44 UTC by Colin Arnott
Modified: 2020-02-14 17:57 UTC (History)
10 users (show)

Fixed In Version: 5.8.0.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1399803 1399805 (view as bug list)
Environment:
Last Closed: 2017-06-12 16:11:41 UTC
Category: ---
Cloudforms Team: VMware
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
5.6.1.2 hotfix to retrieve all host.config.storageDevice in one API call (1.93 KB, patch)
2016-09-29 19:14 UTC, Adam Grare
no flags Details | Diff
5.6.1.2_hotfix_host_storage_devices.tar.gz (18.28 KB, application/x-gzip)
2016-10-13 13:25 UTC, Adam Grare
no flags Details

Description Colin Arnott 2016-09-23 20:44:43 UTC
Description of problem:
Related to 1378991, EMS refresh is slow causing automate methods to timeout when creating folders.

Additional info:
Providing a database dump

Comment 2 Adam Grare 2016-09-29 16:51:08 UTC
Looking at a typical full EmsRefresh using the broker cache, here are some notable durations for different steps:
Total duration:               21min 11sec
Getting cached inventory:           30sec
Getting Host Storage Devices: 11min 31sec

Storage devices for hosts are not stored in the broker cache so every refresh has to go out to the vCenter to get this information.  Currently this is done once for each host which incurs a lot of overhead.

Two possible solutions:
1. Keep host storage devices in broker cache
2. Retrieve storage devices for all hosts in one API call

Comment 3 Adam Grare 2016-09-29 19:14:05 UTC
Created attachment 1206036 [details]
5.6.1.2 hotfix to retrieve all host.config.storageDevice in one API call

Comment 4 Adam Grare 2016-10-04 12:35:37 UTC
PR https://github.com/ManageIQ/manageiq/pull/11629

Comment 9 Adam Grare 2016-10-13 13:25:33 UTC
Created attachment 1210091 [details]
5.6.1.2_hotfix_host_storage_devices.tar.gz

cherry picked from commit 59f755fe421983761bc48d3162f79b109126c4a6

tar tfz 5.6.1.2_hotfix_host_storage_devices.tar.gz 
app/models/ems_refresh/vc_updates.rb
app/models/manageiq/providers/vmware/infra_manager/refresher.rb
gems/pending/VMwareWebService/MiqVimInventory.rb

Comment 13 Adam Grare 2016-11-29 18:32:34 UTC
*** Bug 1341402 has been marked as a duplicate of this bug. ***

Comment 14 Adam Grare 2016-11-29 18:32:47 UTC
*** Bug 1353707 has been marked as a duplicate of this bug. ***

Comment 17 Adam Grare 2017-01-30 14:46:09 UTC
*** Bug 1382163 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.