Bug 1569626

Summary:	Lifecycle VM Provision and Publish VM to Template Unusable/Slow
Product:	Red Hat CloudForms Management Engine	Reporter:	David Luong <dluong>
Component:	Performance	Assignee:	Nick LaMuro <nlamuro>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Himanshu Roy <hroy>
Severity:	high	Docs Contact:
Priority:	high
Version:	5.8.0	CC:	cpelland, dluong, dstanley, fwissing, jrafanie, obarenbo, rovalent, simaishi, smallamp
Target Milestone:	GA	Keywords:	TestOnly, ZStream
Target Release:	5.10.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	5.10.0.0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1593797 1593798 (view as bug list)		Environment:
Last Closed:	2019-02-11 18:51:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	Bug
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	CFME Core	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1593797, 1593798

Description David Luong 2018-04-19 15:32:51 UTC

Description of problem:
We are seeing extremely slow UI response times when using the VM lifecycle provision and publish VM to template workflow.  To the point where the app will just hang with the spinner/load overlay and never progress.

When does the behavior occur? Frequently?  Repeatedly?   At certain times?

All the time.

What information can you provide around timeframes and the business impact?

We're continuing to get more users on the environment daily and we need to ensure a good user experience.

Version-Release number of selected component (if applicable):
5.8.3.4

How reproducible:
All The time

Steps to Reproduce:
1. Use Add a new vm on default lifecycle provisioning OR
2. Publish VM to template
3.

Actual results:
Switching between tabs takes a very long time

Expected results:
Load times to be lower on switching tabs

Comment 11 Nick LaMuro 2018-04-27 16:33:06 UTC

Update:

Created a bunch of patch proposals to the ManageIQ code base to address this issue:

  - https://github.com/ManageIQ/manageiq/pull/17354
  - https://github.com/ManageIQ/manageiq/pull/17355
  - https://github.com/ManageIQ/manageiq/pull/17356
  - https://github.com/ManageIQ/manageiq/pull/17357
  - https://github.com/ManageIQ/manageiq/pull/17358
  - https://github.com/ManageIQ/manageiq/pull/17359
  - https://github.com/ManageIQ/manageiq/pull/17360


More details can be found in the in the pull requests themselves, but the overall impact with them combined is about a 4x speed improvement.

Continuing to refine those changes and determine if there are other fixes that can be done.



-Nick

Comment 12 Nick LaMuro 2018-05-03 17:08:37 UTC

Update:

Created a document that is linked in the previous pull requests above that documents the overall savings from every one of the patches I am proposing:

https://gist.github.com/NickLaMuro/36f9f394a5b3cecc4aaf33d3e84dfb27

I will be updating that shortly with additions from three other pull requests I opened this morning:

  - https://github.com/ManageIQ/manageiq/pull/17381
  - https://github.com/ManageIQ/manageiq-providers-vmware/pull/249
  - https://github.com/ManageIQ/manageiq-providers-scvmm/pull/73

Which together helps improve the efficiency of these screens further.

* * *


I will continue to look into getting these PRs merged and finding other ways to make these screens even faster.

-Nick

Comment 14 Nick LaMuro 2018-05-16 20:52:36 UTC

Update:


Opened up a few more patches since the last update:

  - https://github.com/ManageIQ/manageiq/pull/17402
  - https://github.com/ManageIQ/manageiq/pull/17403
  - https://github.com/ManageIQ/manageiq/pull/17422

#422 in particular has a low risk, quick win that saves a lot in "large host count per datacenter" environments.  That one will most likely be merged with relatively quickly, but waiting to hear about most of the others.


-Nick

Comment 17 Nick LaMuro 2018-06-20 16:04:14 UTC

Update:

There hasn't been much of an update here since I haven't made any new fixes as of late, but I figured a status update on what has been merged is in order.


The following have been merged into master:

- https://github.com/ManageIQ/manageiq/pull/17325
- https://github.com/ManageIQ/manageiq/pull/17354
- https://github.com/ManageIQ/manageiq/pull/17355
- https://github.com/ManageIQ/manageiq/pull/17357
- https://github.com/ManageIQ/manageiq/pull/17358
- https://github.com/ManageIQ/manageiq/pull/17360
- https://github.com/ManageIQ/manageiq/pull/17402
- https://github.com/ManageIQ/manageiq/pull/17422
- https://github.com/ManageIQ/manageiq/pull/17381
  - https://github.com/ManageIQ/manageiq-providers-vmware/pull/249
  - https://github.com/ManageIQ/manageiq-providers-vmware/pull/269
  - https://github.com/ManageIQ/manageiq-providers-scvmm/pull/73



The following are waiting on review, or need to address review feedback:

- https://github.com/ManageIQ/manageiq/pull/17457
- https://github.com/ManageIQ/manageiq/pull/17460
- https://github.com/ManageIQ/manageiq/pull/17461
- https://github.com/ManageIQ/manageiq/pull/17359
- https://github.com/ManageIQ/manageiq/pull/17403


Am going to look at getting the above reviewed, and possibly see if there is anything else I can do to make this faster.  Priority is focusing on getting merged what is fixed versus what else can be fixed however.


Again, changes observed via benchmarking before and after (with all of the above merged in the "After" case), can be viewed here:


https://gist.github.com/NickLaMuro/36f9f394a5b3cecc4aaf33d3e84dfb27

Comment 22 Freddy Wissing 2018-08-28 18:48:57 UTC

Hi Satoe,

I have a customer that is on these versions:

# rpm -qa | grep cfme
cfme-gemset-5.8.4.5-1.el7cf.x86_64
rh-ruby23-rubygem-redhat_access_cfme-1.1.0-1.el7cf.noarch
cfme-5.8.4.5-1.el7cf.x86_64
cfme-appliance-5.8.4.5-1.el7cf.x86_64

What is the correct supported hotfix for them to use?

/Freddy

Comment 24 Sudhir Mallamprabhakara 2019-02-11 18:51:18 UTC

5.10 is now GA.. closing this.