Bug 718091

Summary: Plugin operations are being queued and executed with just one thread
Product: [Other] RHQ Project Reporter: Rui Alves <rui.fc.alves>
Component: Plugin ContainerAssignee: RHQ Project Maintainer <rhq-maint>
Status: CLOSED DUPLICATE QA Contact: Mike Foley <mfoley>
Severity: medium Docs Contact:
Priority: medium    
Version: unspecifiedCC: hrupp
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-07-19 09:33:17 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Rui Alves 2011-06-30 23:02:35 UTC
Description of problem: I'm managing many jboss instances with just one rhq agent, and started to have performance issues with concurrent operations. After monitoring agent threads I've concluded that all queued operations were being executed by the same nonDaemon thread.
After some debug in RHQ source code, I found, what it seems, a miss configuration of ThreadPoolExecutor using a LinkedBlockingQueue:

"ThreadPoolExecutor threadPool = new ThreadPoolExecutor(1, maxPoolSize, 1000, TimeUnit.MILLISECONDS, queue, threadFactory);"

The ThreadPoolExecutor javadoc says:
"Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue without a predefined capacity) will cause new tasks to wait in the queue when all corePoolSize threads are busy. Thus, no more than corePoolSize threads will ever be created. (And the value of the maximumPoolSize therefore doesn't have any effect.) This may be appropriate when each task is completely independent of others, so tasks cannot affect each others execution; for example, in a web page server. While this style of queuing can be useful in smoothing out transient bursts of requests, it admits the possibility of unbounded work queue growth when commands continue to arrive on average faster than they can be processed."

For testing purposes I've changed corePoolSize from 1 to 20 and I started to have multiple threads executing parallel operations as expected.

"ThreadPoolExecutor threadPool = new ThreadPoolExecutor(20, maxPoolSize, 1000, TimeUnit.MILLISECONDS, queue, threadFactory);"


How reproducible: Executing multiple Operations in different resources at the same time with the same RHQ Agent (I've tested with stop and start Operations available in Jboss 5 Plugin).

Comment 1 Rui Alves 2012-07-19 09:33:17 UTC

*** This bug has been marked as a duplicate of bug 830996 ***