Bug 1309747

Summary: [GSS](6.4.z) JMX inefficiently handles resource adapters
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Aaron Ogburn <aogburn>
Component: JCAAssignee: Ivo Studensky <istudens>
Status: CLOSED CURRENTRELEASE QA Contact: Peter Mackay <pmackay>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.4.6CC: bbaranow, bmaxwell, hrupp, istudens, iweiss, jpederse, jtruhlar, pmackay, sdouglas
Target Milestone: CR1   
Target Release: EAP 6.4.8   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-01-17 12:38:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1279553, 1312370    

Description Aaron Ogburn 2016-02-18 15:31:01 UTC
Description of problem:

If you connect to JBoss over JMX when it has many rars deployed (over 500), long running high CPU is seen in the following calls:

        at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:188)
        at org.jboss.as.jmx.model.ResourceAccessControlUtil.getResourceAccess(ResourceAccessControlUtil.java:85)
        at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:51)
        at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
        at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
        at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
        at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
        at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
        at org.jboss.as.jmx.model.RootResourceIterator.doIterate(RootResourceIterator.java:61)
        at org.jboss.as.jmx.model.RootResourceIterator.iterate(RootResourceIterator.java:43)

Version-Release number of selected component (if applicable):


How reproducible:

Very

Steps to Reproduce:
1. Deploy many individual rars
2. Connect via JMX (for instance jconsole.sh and try to view mbeans)

Actual results:

Continuous high CPU and mbeans are never displayed

Expected results:

mbeans are displayed in reasonable amount of time with reasonable CPU usage

Additional info:

It's evident that it is iterating over each deployment (each resource adapter counting as a deployment), but for each deployment, it is also iterating over all resource adapter's and their config properties.  So it would iterate over all 500+ resource adapters once for each deployment, so it iterates over all 500 adapters at least 500 times.  Is there anyway we could avoid that n^2 iteration for deployed rars?

Comment 10 Mike McCune 2016-03-28 22:44:25 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

Comment 11 Peter Mackay 2016-05-05 12:03:29 UTC
Verified with EAP 6.4.8.CP.CR2

Comment 12 Petr Penicka 2017-01-17 12:38:20 UTC
Retroactively bulk-closing issues from released EAP 6.4 cumulative patches.