Red Hat Bugzilla – Bug 718091
Plugin operations are being queued and executed with just one thread
Last modified: 2012-07-19 05:33:17 EDT
Description of problem: I'm managing many jboss instances with just one rhq agent, and started to have performance issues with concurrent operations. After monitoring agent threads I've concluded that all queued operations were being executed by the same nonDaemon thread.
After some debug in RHQ source code, I found, what it seems, a miss configuration of ThreadPoolExecutor using a LinkedBlockingQueue:
"ThreadPoolExecutor threadPool = new ThreadPoolExecutor(1, maxPoolSize, 1000, TimeUnit.MILLISECONDS, queue, threadFactory);"
The ThreadPoolExecutor javadoc says:
"Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue without a predefined capacity) will cause new tasks to wait in the queue when all corePoolSize threads are busy. Thus, no more than corePoolSize threads will ever be created. (And the value of the maximumPoolSize therefore doesn't have any effect.) This may be appropriate when each task is completely independent of others, so tasks cannot affect each others execution; for example, in a web page server. While this style of queuing can be useful in smoothing out transient bursts of requests, it admits the possibility of unbounded work queue growth when commands continue to arrive on average faster than they can be processed."
For testing purposes I've changed corePoolSize from 1 to 20 and I started to have multiple threads executing parallel operations as expected.
"ThreadPoolExecutor threadPool = new ThreadPoolExecutor(20, maxPoolSize, 1000, TimeUnit.MILLISECONDS, queue, threadFactory);"
How reproducible: Executing multiple Operations in different resources at the same time with the same RHQ Agent (I've tested with stop and start Operations available in Jboss 5 Plugin).
*** This bug has been marked as a duplicate of bug 830996 ***