Bug 1022642 - Discovery queue can't display large inventory
Discovery queue can't display large inventory
Status: CLOSED CURRENTRELEASE
Product: JBoss Operations Network
Classification: JBoss
Component: UI (Show other bugs)
JON 3.2
All All
high Severity high
: ER07
: JON 3.2.0
Assigned To: Jay Shaughnessy
Mike Foley
:
Depends On:
Blocks: 1012435
  Show dependency treegraph
 
Reported: 2013-10-23 13:10 EDT by Viet Nguyen
Modified: 2014-01-02 15:43 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-01-02 15:43:25 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
screenshot (880.58 KB, image/png)
2013-10-23 13:10 EDT, Viet Nguyen
no flags Details
server.log (831.67 KB, text/plain)
2013-10-23 13:15 EDT, Viet Nguyen
no flags Details

  None (edit)
Description Viet Nguyen 2013-10-23 13:10:56 EDT
Created attachment 815490 [details]
screenshot

Description of problem:

Discover queue can't display large inventory (75 agents x 25 EAP6 instances) 

Version-Release number of selected component (if applicable):
ER3

How reproducible:
100%

Steps to Reproduce:
1. Install 25 EAP6 instances
2. Start 75 Agents via agentcopy script


Actual results:

Firefox shows spinning circle/loading data... timeout exception in server.log
Comment 1 Viet Nguyen 2013-10-23 13:15:17 EDT
Created attachment 815491 [details]
server.log
Comment 4 Jay Shaughnessy 2013-10-29 16:51:30 EDT
I'll take a swing at this an try to pull data incrementally.  Not with paging but rather fetching data as needed to handle expansion.  Basically, increase the number of DB round trips.
Comment 6 Jay Shaughnessy 2013-11-06 09:58:37 EST
ETA = now.

release/jon3.2.x commit 85a41a0bc2cfc3512f13cc852def22ae5ea2cdef
Author: Jay Shaughnessy <jshaughn@redhat.com>
Date:   Wed Nov 6 09:56:11 2013 -0500

The issue is that we attempt to pull all the ADQ resources at one time.  Which
overwhelms the view if there are many discovered resources.  For example,
when many new agents monitoring many servers all start up and report
discovery results at basically the same time.

This commit changes the approach such that we pull only the relevant platforms
first.  And then the server resources as needed, such as on tree expand.  Things
are complicated somewhat by the fact that we allow child server selection
without platform expansion and also but supporting "Select All".  Select All
will pull the child servers for each server in a separate DB request.  Whether
this is the best (or an acceptable) approach remains to be seen.

Finally, assuming we now can display a large inventory of uncommitted resources
it remains to be seen whether we can select all and import them all
successfully, as a quick look at the code reveals that we do it all
in one transaction.  It may make sense to break up the import work and
process one platform at a time.

Cherry-Pick Master : 2bc5aab6ec08055c1c7c981aabaded8f05fb44f4
Comment 7 Jay Shaughnessy 2013-11-06 10:23:57 EST
I'm looking at some follow-up work now, to perhaps prevent a large import failure now that we should be able to actually perform a large import request...
Comment 8 Jay Shaughnessy 2013-11-06 15:17:50 EST
master commit fd18ea22f83297bb806deba47365e86ebe23d573
Author: Jay Shaughnessy <jshaughn@redhat.com>
Date:   Wed Nov 6 15:02:57 2013 -0500

    Related to the work on this BZ.  Given that we can now support a large
    inventory in this view (hopefully) we may have a downstream issue trying to
    import the inventory in one large import request.

    So, this commit breaks it up into one import request per platform (well,
    per relevant platform that has selected uncommitted resources)



*** note
This is in master only as it doesn't immediately affect the BZ in question.  But if desired, or certainly if there is a discovered issue importing a large inventory, this commit could move to the release branch.
***
Comment 9 Simeon Pinder 2013-11-19 10:48:53 EST
Moving to ON_QA as available for testing with new brew build.
Comment 10 Simeon Pinder 2013-11-22 00:14:19 EST
Mass moving all of these from ER6 to target milestone ER07 since the ER6 build was bad and QE was halted for the same reason.

Note You need to log in before you can comment on or make changes to this bug.