Bug 1585655

Summary: [Scale] RHV provider removal take a lot of time
Product: Red Hat CloudForms Management Engine Reporter: Ilanit Stein <istein>
Component: ProvidersAssignee: Boriso <bodnopoz>
Status: CLOSED CURRENTRELEASE QA Contact: Angelina Vasileva <anikifor>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.9.0CC: gblomqui, jfrey, jhardy, mperina, obarenbo
Target Milestone: GA   
Target Release: 5.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-11 15:25:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: Bug
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: RHEVM Target Upstream Version:
Embargoed:
Attachments:
Description Flags
gr_off_automation_log
none
gr_off_evm_log
none
gr_on_automation_log
none
gr_on_evm_log none

Description Ilanit Stein 2018-06-04 10:31:29 UTC 
Description of problem:
1st removal time of the RHV provider is quite large: 
GR (Graph Refresh) OFF: 17 min,
and for GR ON it's even larger GR ON: 41 min.

Testing environment details:
2 CFME-5.9.3 machines (Memory 8G, 4 cores) (in US)::
   1. With Graph refresh OFF (default)
   2. With Graph refresh ON.

Scale RHV-4.2.3.5-0.1.el7 (in RDU):
   b01-h21-r620.rhev.openstack.engineering.redhat.com
   Clusters	 4
   Hosts	 403
   Datastores	 6
   Virtual Machines 4059
   Templates 13

Test results:
   Graph refresh OFF:
   1st removal 17 min
   2nd removal 1 min

   Graph refresh ON:
   1st removal 41 min
   2nd removal 1 min

Notes:
* Both RHV & CFME located in the US, so these tests are with no latency.
* Removal time is taken from the CFME UI tasks page, task: "Destroying  
  ManageIQ::Providers::Redhat::InfraManager with id: "

Version-Release number of selected component (if applicable):
CFME-5.9.3
RHV-4.2.3.5-0.1.el7
Comment 2 Ilanit Stein 2018-06-04 10:32:52 UTC 
Created attachment 1447385 [details]
gr_off_automation_log
Comment 3 Ilanit Stein 2018-06-04 11:47:27 UTC 
Created attachment 1447404 [details]
gr_off_evm_log
Comment 4 Ilanit Stein 2018-06-04 11:48:16 UTC 
Created attachment 1447405 [details]
gr_on_automation_log
Comment 5 Ilanit Stein 2018-06-04 12:13:49 UTC 
Created attachment 1447409 [details]
gr_on_evm_log