Bug 906866 - disable types on agent so avoid large number of services causing inventory report to not get processed
Summary: disable types on agent so avoid large number of services causing inventory re...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Operations Network
Classification: JBoss
Component: Inventory, Plugin Container
Version: JON 3.1.2
Hardware: All
OS: All
urgent
urgent
Target Milestone: ER01
: JON 3.2.0
Assignee: John Mazzitelli
QA Contact: Mike Foley
URL:
Whiteboard:
Depends On: 902406
Blocks: 905651 1012435
TreeView+ depends on / blocked
 
Reported: 2013-02-01 18:08 UTC by Larry O'Leary
Modified: 2018-12-01 17:42 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 902406
Environment:
Last Closed: 2014-01-02 20:36:17 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
resources (504.46 KB, image/png)
2013-09-24 14:16 UTC, Armine Hovsepyan
no flags Details
ignoredServer (286.98 KB, image/png)
2013-10-11 13:42 UTC, Armine Hovsepyan
no flags Details
serverActive (302.64 KB, image/png)
2013-10-11 13:45 UTC, Armine Hovsepyan
no flags Details
serverService (351.80 KB, image/png)
2013-10-11 13:45 UTC, Armine Hovsepyan
no flags Details
serverServiceDisabled (287.41 KB, image/png)
2013-10-11 13:45 UTC, Armine Hovsepyan
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 302893 0 None None None Never
Red Hat Knowledge Base (Solution) 308793 0 None None None Never

Description Larry O'Leary 2013-02-01 18:08:26 UTC
+++ This bug was initially created as a clone of upstream Bug #902406 +++

The scenario is:

1) agent detects a server, sends it up to server via inventory report
2) user commits that server to inventory
3) agent is told about this, commits the server and auto-detects children services of that newly committed server. The number of children services is huge - say, on the order of tens of thousands (e.g. 40,000 or 50,000)
4) The agents sends up this huge inventory report

At this point, the server will have a hard time processing this report. The server needs to handle this scenario better.

--- Additional comment from John Mazzitelli on 2013-01-21 12:55:28 EST ---

with the perfplugin deployed, this helps reproduce the issue:

-Drhq.perftest.scenario=configurable-1 -Drhq.perftest.server-a-count=1 -Drhq.perftest.service-a-count=25000

I saw the issue with 50000, jay saw it with 25000. Note that with it only 5000 (that is, server has 5,000 child services) I did not get any errors. So we can support a large number, just not a really, really large number.

--- Additional comment from John Mazzitelli on 2013-01-21 20:03:59 EST ---

(In reply to comment #1)
> Note that with it only
> 5000 (that is, server has 5,000 child services) I did not get any errors. So
> we can support a large number, just not a really, really large number.

I'll walk back from this statement - even though I had 5,000 peer services successfully in inventory, the UI was unable to render the left-hand tree. I also saw some OOMs that were probably related. In any event, 5000 direct children under a parent is not working well either, even though it avoided the issue documented in this BZ.

--- Additional comment from John Mazzitelli on 2013-01-22 10:27:53 EST ---

assigning to jay - he is currently refactoring the merge inventory report code.

--- Additional comment from John Mazzitelli on 2013-01-25 23:25:11 EST ---

A new feature is added to allow someone to disable one or more resource types at the agent level. There is an agent config setting that you can set to disable any resource types - these disable types won't be discovered and any resources that previously made it into inventory before this will now appear RED (down). You'd have to uninventory the parent and re-inventory it in order for the resources to go away. But any new resources of the given disabled types will not be discovered.

git commit to bug/902406 branch:

13cb988bb4835289393c8e3f736918651d6e2bae (the main commit)
085acde34623280a7720ee866f0efd4fc005d435 (one line fix for NPE)

--- Additional comment from Larry O'Leary on 2013-02-01 12:33:02 EST ---

*** Bug 905654 has been marked as a duplicate of this bug. ***

--- Additional comment from Larry O'Leary on 2013-02-01 13:05:16 EST ---

Bug 905654 is not a duplicate of this bug. I fixed that.

Comment 1 Armine Hovsepyan 2013-09-24 14:16:25 UTC
Created attachment 802262 [details]
resources

Comment 2 Armine Hovsepyan 2013-09-24 14:39:37 UTC
reassigning 
* agent goes down when started with performance variables like :     -Drhq.perftest.scenario=configurable-1 -Drhq.perftest.server-a-count=1 -Drhq.perftest.service-a-count=25000 or -Drhq.perftest.scenario=configurable-average -Drhq.perftest.server-a-count=1 -Drhq.perftest.service-a-count=25000
* no resources under server-a-0

Comment 3 Armine Hovsepyan 2013-09-24 14:42:01 UTC
checked for VMs having 2GB and 8GB  RAM

Comment 4 John Mazzitelli 2013-10-08 14:09:44 UTC
(In reply to Armine from comment #2)
> reassigning 
> * agent goes down when started with performance variables like :    
> -Drhq.perftest.scenario=configurable-1 -Drhq.perftest.server-a-count=1
> -Drhq.perftest.service-a-count=25000 or
> -Drhq.perftest.scenario=configurable-average -Drhq.perftest.server-a-count=1
> -Drhq.perftest.service-a-count=25000
> * no resources under server-a-0

We know very large inventories will cause the agent to run out of memory - you can't have an infinitely large inventory (or very very large inventories).  What this BZ was fixing was allowing the user to disable resource types so, even though your managed machine has that many resources, you can disable them so the agent doesn't detect them or import them.

So the idea is that you disable the service resource types that are defined by the perftest plugin, then you run the agent and notice that it doesn't load on those resources of that type.

See bug #535289 for more on the feature to disable types.

I'm moving this ON_QA because I believe all of this is already implemented and released in RHQ 4.9, unless I am misunderstanding something.

Comment 5 Armine Hovsepyan 2013-10-11 13:40:16 UTC
verified

Disabled resources are not available in discovery queue as well as hidden from inventory list (for inventoried resources)
please get screen-shots attached.

Comment 6 Armine Hovsepyan 2013-10-11 13:42:01 UTC
Created attachment 811115 [details]
ignoredServer

Comment 7 Armine Hovsepyan 2013-10-11 13:45:02 UTC
Created attachment 811116 [details]
serverActive

Comment 8 Armine Hovsepyan 2013-10-11 13:45:40 UTC
Created attachment 811117 [details]
serverService

Comment 9 Armine Hovsepyan 2013-10-11 13:45:49 UTC
Created attachment 811118 [details]
serverServiceDisabled


Note You need to log in before you can comment on or make changes to this bug.