Bug 1962276

Summary: VM inventory tree is slow to display if the source provider has more than 4000 VMs
Product: Migration Toolkit for Virtualization Reporter: David Vaanunu <dvaanunu>
Component: GeneralAssignee: Fabien Dupont <fdupont>
Status: CLOSED ERRATA QA Contact: Ilanit Stein <istein>
Severity: high Docs Contact: Avital Pinnick <apinnick>
Priority: unspecified    
Version: 2.0.0CC: apinnick, dagur, fdupont, ibragins, istein, jortel, mguetta, mturley
Target Milestone: ---   
Target Release: 2.0.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-06-10 17:12:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description David Vaanunu 2021-05-19 16:27:07 UTC
Description of problem:
When having a source with many VMs(Tested with ~4000 VMs), creating a provider, display inventory VM tree took a long time

Version-Release number of selected component (if applicable):

CNV_2.6.3-16 (iib 75326)
MTV_2.0.0-18 (iib 76039)

How reproducible:


Steps to Reproduce:
1. Create a new provider with ~4000VMs. 
2. Create a new migration plan (using the provider in step 1)
3. VM selection, filter step - cluster&hosts / ByFolder   
4. VM selection, Select VMs step - slowness in pick VM and paging

Actual results:
Create provider took ~1-2min
Display VM tree filter by cluster&hosts / ByFolder took 2-8min


Expected results:

Get faster response


Additional info:

Comment 2 Maayan Hadasi 2021-05-20 10:35:22 UTC
Additional info:

There is a slowness also in loading the "Migration details by VM" page for the first time, when migrating from source provider with many VMs (VMware with 4K VMs)
In my case, I started a cold migration via API -> opened the "Migration details by VM" page => page loading took ~1.5 minutes

* Moving to "Migration plans" page and reopening the "Migration details by VM" => the page is shown immediately
* Reloading the "Migration details by VM" page (F5) => takes again too much time

Comment 4 Mike Turley 2021-05-20 13:37:48 UTC
Maayan, that's also concerning and I don't believe my PR would fix it. Looking into it now.

Comment 6 Maayan Hadasi 2021-05-20 13:53:02 UTC
HI @mturley 

It was on mgn06 but the cluster is not in a good shape now. I will try to reproduce it on another cluster and will update you with its connection details

Comment 7 Jeff Ortel 2021-05-21 14:37:19 UTC
I found a/the culprit for the high cpu usage when the controller should be idle.
Basically it's a situation that results in a hot (infinite) loop in the goroutine that searches for VMs to validate.  The search runs every 10 min just in case the watch notification get's overwhelmed.  It's a catch-all.  I think we pretty much always get into this situation when 1+ large providers are loaded.
More detail in the PR description: https://github.com/konveyor/forklift-controller/pull/269

Comment 9 Ilanit Stein 2021-05-26 14:47:16 UTC
This issue was tested by QE using MTV upstream May 25 2021 latest and we do not see any slowness in VMs discovery or building the inventory tree.
We consider this bug as solved.

@Fabien,
Can this bug move to MODIFIED?

Comment 10 Mike Turley 2021-05-26 14:51:03 UTC
@jortel, I assume any backend pieces of this are merged/backported already? I know the UI parts are, so I think we can move it to MODIFIED.

Comment 11 Jeff Ortel 2021-05-26 15:01:39 UTC
(In reply to Mike Turley from comment #10)
> @jortel, I assume any backend pieces of this are
> merged/backported already? I know the UI parts are, so I think we can move
> it to MODIFIED.

Yes.

Comment 12 Fabien Dupont 2021-05-27 10:39:54 UTC
Please verify with build 2.0.0-20 / iib:77891

Comment 13 Igor Braginsky 2021-05-31 12:52:18 UTC
Provider creation passed with no delay at all, picking VMs up when creating a new plan took around 10-12 sec.
Validated on build 2.0.2-23.

Comment 16 errata-xmlrpc 2021-06-10 17:12:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (MTV 2.0.0 images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2021:2381