Bug 1580982 - Viewing 200 VMware hosts or more causes UI error and exceeded UI worker
Summary: Viewing 200 VMware hosts or more causes UI error and exceeded UI worker
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Performance
Version: 5.8.0
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: GA
: 5.8.4
Assignee: Nick LaMuro
QA Contact: Matouš Mojžíš
URL:
Whiteboard: ui:infra:provider:host
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-21 22:14 UTC by Tuan
Modified: 2021-09-09 14:10 UTC (History)
9 users (show)

Fixed In Version: 5.8.4.4
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-29 15:14:15 UTC
Category: Bug
Cloudforms Team: CFME Core
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Comment 3 Nick LaMuro 2018-05-24 16:57:28 UTC
The following should be merged shortly:

https://github.com/ManageIQ/manageiq-ui-classic/pull/3989

And will fix the issue that is being reported with the least amount of changes necessary.

Comment 4 Nick LaMuro 2018-05-24 19:02:49 UTC
Fix merged.

Comment 8 Matouš Mojžíš 2018-06-12 10:36:07 UTC
Nick,

I tried to reproduce this BZ with this rails c command:
200.times { |i| ManageIQ::Providers::Vmware::InfraManager::Host.create :name => "host_%05d" % (1+i), :vmm_vendor => "vmware"}
Loading 200 hosts with pagination set to 200 items was fast so I think I have to add some more data to hosts so query is more complex and hosts are rendered slowly?

Thanks

Comment 9 Nick LaMuro 2018-06-12 17:34:18 UTC
Matouš,

So, it isn't as simple as there just being a bunch of Vm records in the database, but also a bunch of corresponding tags and compliance reports as well.  As mentioned in the patch description:

https://github.com/ManageIQ/manageiq-ui-classic/pull/3989

> This means that even just querying against 20 hosts that has 14000 compliance records associated with them, along with 17000 taggings can end up causing a query with over 2 million rows returned, with 60+ columns per row. That data returned from postgres ends up being multiple Gigs in size.


So you would need to create a script that not only creates Vm records, but applies tags to them and compliance reports (OpenScap) to them as well to properly reproduce the issue.

* * *


I don't have a console oneliner or script on hand that you could use at the moment to replicate (since I was using a customer DB to reproduce), and not sure if I have the cycles available at the moment to invest some time into determining how to put one together.  That said, if I do I will report back here with something to share.


-Nick

Comment 12 Nick LaMuro 2018-06-18 18:39:06 UTC
To replicate this bug, you can run the following script to populate a fresh database with data to replicate the issue:

    https://gist.github.com/NickLaMuro/225833358423723ed17ff294415fa6b4

To run script, you can do the following in the project dir on an appliance:

    $ bin/rails r bz_1580569_db_replication_script.rb


* * *

To replicate the issue, the easiest way is through the console:


    $ bin/rails c
    irb> MiqReport.load_from_view_options(Host, User.where(:userid => 'admin').first).paged_view_search


Without the patch, this will cause the process to consume roughly 10+ Gigs of RAM do to a poorly optimized database query.  The patch (mentioned above) prevents this.

Comment 13 Matouš Mojžíš 2018-06-19 16:23:38 UTC
Verified in 5.8.4.5.
With provided script I was able to view 250 VmWare hosts without errors and view was loaded fast.
Also this script was executed was with no issues:
MiqReport.load_from_view_options(Host, User.where(:userid => 'admin').first).paged_view_search


Note You need to log in before you can comment on or make changes to this bug.