Bug 905651

Summary:	disable types on agent so avoid large number of services causing inventory report to not get processed
Product:	[JBoss] JBoss Operations Network	Reporter:	John Mazzitelli <mazz>
Component:	Core Server	Assignee:	John Mazzitelli <mazz>
Status:	CLOSED NEXTRELEASE	QA Contact:	Mike Foley <mfoley>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	JON 3.1.2	CC:	hrupp, loleary, vnguyen
Target Milestone:	---
Target Release:	JON 3.1.3
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	902406
Clones:	906867 (view as bug list)		Environment:
Last Closed:	2013-09-06 02:25:06 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	906866
Bug Blocks:	906867

Description John Mazzitelli 2013-01-29 21:13:01 UTC

+++ This bug was initially created as a clone of Bug #902406 +++

The scenario is:

1) agent detects a server, sends it up to server via inventory report
2) user commits that server to inventory
3) agent is told about this, commits the server and auto-detects children services of that newly committed server. The number of children services is huge - say, on the order of tens of thousands (e.g. 40,000 or 50,000)
4) The agents sends up this huge inventory report

At this point, the server will have a hard time processing this report. The server needs to handle this scenario better.

--- Additional comment from John Mazzitelli on 2013-01-21 12:55:28 EST ---

with the perfplugin deployed, this helps reproduce the issue:

-Drhq.perftest.scenario=configurable-1 -Drhq.perftest.server-a-count=1 -Drhq.perftest.service-a-count=25000

I saw the issue with 50000, jay saw it with 25000. Note that with it only 5000 (that is, server has 5,000 child services) I did not get any errors. So we can support a large number, just not a really, really large number.

--- Additional comment from John Mazzitelli on 2013-01-21 20:03:59 EST ---

(In reply to comment #1)
> Note that with it only
> 5000 (that is, server has 5,000 child services) I did not get any errors. So
> we can support a large number, just not a really, really large number.

I'll walk back from this statement - even though I had 5,000 peer services successfully in inventory, the UI was unable to render the left-hand tree. I also saw some OOMs that were probably related. In any event, 5000 direct children under a parent is not working well either, even though it avoided the issue documented in this BZ.

--- Additional comment from John Mazzitelli on 2013-01-22 10:27:53 EST ---

assigning to jay - he is currently refactoring the merge inventory report code.

--- Additional comment from John Mazzitelli on 2013-01-25 23:25:11 EST ---

A new feature is added to allow someone to disable one or more resource types at the agent level. There is an agent config setting that you can set to disable any resource types - these disable types won't be discovered and any resources that previously made it into inventory before this will now appear RED (down). You'd have to uninventory the parent and re-inventory it in order for the resources to go away. But any new resources of the given disabled types will not be discovered.

git commit to bug/902406 branch:

13cb988bb4835289393c8e3f736918651d6e2bae (the main commit)
085acde34623280a7720ee866f0efd4fc005d435 (one line fix for NPE)

Comment 1 John Mazzitelli 2013-01-30 15:16:16 UTC

git commit as seen in the "master" branch as well as the "bug/902406" branch:

13cb988bb4835289393c8e3f736918651d6e2bae (the main commit)
085acde34623280a7720ee866f0efd4fc005d435 (one line fix for NPE)

These git commits, as they are seen in "release/jon3.1.x" branch, are:

811fbb8970b3013fc0e6555c0db908f9112ff13a
40caae29a4dadc018a96ae4cbf687ff3f8b594b1

You can set this new config option either as agent java opt:

-Drhq.agent.plugins.disabled-resource-types=...

or within agent config .xml

The value looks like this:

[pluginName]>[parentTypeName1]>...>[typeNameToIgnore]

You can have more than one by separating them with the "|" character.

So, for my testing, I was using this:

PerfTest>server-deep1>service-deep2>service-deep3>service-deep4>service-deep5|RHQAgent>RHQ Agent>RHQ Agent JVM>Operating System

In other words, I was ignoring the type "service-deep5" as defined in the "PerfTest" plugin and I was ignoring the type "Operating System" as defined in the "RHQAgent" plugin.

Plugin name is the name as defined in the plugin descriptor rhq-plugin.xml - in the root element <plugin> there is a name attribute. Obviously, the type names are those also defined in the plugin descriptor (<server> or <service> types).

Comment 2 Larry O'Leary 2013-02-09 00:48:34 UTC

See https://access.redhat.com/knowledge/solutions/308793 as it pertains to the use of this new configuration option.

Comment 3 Mike Foley 2013-07-18 21:51:20 UTC

QE Qaulification
----------------

Viet will need to qualify this.

Comment 4 Larry O'Leary 2013-09-06 02:25:06 UTC

Closing as there will not be a 3.1.3 release. This is being tracked for 3.2 in the 'depends on' field.