Description of problem: Feb 17 07:08:04 com..Utility : DEBUG: groups command reports: qwert users shadow qwert-users qwert Feb 17 07:08:04 com.runclicommand : DEBUG: Calling run_cli_command for: connect,/subsystem=logging:read-children-names(child-type=logger),quit Feb 17 07:08:04 com.application_server_jbcp6_impl : DEBUG: Cli java options set: -Xms64m -Xmx512m -XX:MaxPermSize=256m Feb 17 07:08:04 com.application_server_jbcp6_impl : DEBUG: Attempting jboss CLI command: /usr/bin/sh /opt/qwert/appserver/bin/jboss-cli.sh -c --controller=192.168.0.3:30001 "connect,/subsystem=logging:read-children-names(child-type=logger),quit" Feb 17 07:08:07 com.application_server_jbcp6_impl : DEBUG: Successfully ran jboss CLI command: connect,/subsystem=logging:read-children-names(child-type=logger),quit Feb 17 07:08:07 com.application_server_jbcp6_impl : DEBUG: CLI returned: { "outcome" => "success", "result" => [ "com.arjuna", "com.qwert", "com.qwert.logging.LevelManager", "com.qwert.logging.LoggerLevels", "com.qwert.logging.LoggingConfig", "com.qwert.logging.RollingOutputStream", "com.qwert.logging.SafLoggingManager", "com.qwert.logging.SafStreamConfig", "jacorb", "jacorb.config", "org.apache.tomcat.util.modeler", "org.jboss", "org.jboss.as.config", "sun.rmi" ] } Exception in thread "Remoting "cli-client" task-14" java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteConnectionChannel$3@47478104 rejected from org.xnio.XnioWorker$TaskPool@2767522b[Shutting down, pool size = 14, active threads = 2, queued tasks = 0, completed tasks = 13] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372) at org.xnio.XnioWorker.execute(XnioWorker.java:572) at org.jboss.remoting3.remote.RemoteConnectionChannel.receiveMessage(RemoteConnectionChannel.java:363) at org.jboss.as.protocol.mgmt.ManagementChannelReceiver.handleMessage(ManagementChannelReceiver.java:99) at org.jboss.remoting3.remote.RemoteConnectionChannel$5.run(RemoteConnectionChannel.java:451) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Exception running CLI commands Expected results: No exception running CLI commands Additional info:
David, Do you think this is the same issue as WFCORE-227, and if so do you think the xnio version used in EAP 6.x branch already has the fix?
Please provide test (as discussed with Carlo) to include this in CP01 payload. I will qa_ack afterwards.
qa_acking, thank you for the test
Verified on EAP 6.4.2.CP.CR1. RejectedExecutionException still can be seen while looping through these operations (approximately once in 100000 tries): client = ModelControllerClient.Factory.create(host,port); client.execute(operation); client.close(); However this seems to be the limitation of the implementation as above scenario creates/destroys 30000 clients every minute. Not seen RejectedExecutionException using jboss-cli.
Retroactively bulk-closing issues from released EAP 6.4 cumulative patches.