Bug 882048
| Summary: | Shutdown process hangs using thread pool executor | ||
|---|---|---|---|
| Product: | [JBoss] JBoss Enterprise Application Platform 6 | Reporter: | Eiichi Nagai <enagai> |
| Component: | Web | Assignee: | Rémy Maucherat <rmaucher> |
| Status: | CLOSED WONTFIX | QA Contact: | Radim Hatlapatka <rhatlapa> |
| Severity: | medium | Docs Contact: | Russell Dickenson <rdickens> |
| Priority: | unspecified | ||
| Version: | 6.0.0 | CC: | aogburn, dandread, fnasser, istudens, iweiss, kkhan, mvelas |
| Target Milestone: | --- | ||
| Target Release: | EAP 6.4.0 | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-06-23 15:10:55 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
I've checked a few things in debugging this: 1) QueueExecutor is indeed being shutdown after confirming its stop variable value in a heap dump. Since shutdown is called, the worker threads are interrupted. 2) The interrupts do nothing to the keepalive worker threads on the socket read. They do not break out of the read in any way and so shutdown just waits on the long, uninterruptible reads. Remy Maucherat <rmaucher> updated the status of jira AS7-6105 to Resolved Remy Maucherat <rmaucher> made a comment on jira AS7-6105 It is normal behavior if the threads are not daemon threads. There's no benefit to using an executor with the java.io AJP connector (it will almost never give any thread back). David Lloyd <david.lloyd> made a comment on jira AS7-6105 It would be nice if we could forbid such a configuration... Remy Maucherat <rmaucher> made a comment on jira AS7-6105 Do you think it should go as far as check if the executor configuration has a daemon flag using a simple runnable ? I don't see any other way to do it. Mathieu Lachance <mathieu> made a comment on jira AS7-6105 jboss thread subsystem do not expose the daemon flag Mathieu Lachance <mathieu> made a comment on jira AS7-6105 jboss thread subsystem do not expose the daemon flag, I think the issue should be re-opened. David Lloyd <david.lloyd> made a comment on jira AS7-6105 To elaborate.... - The daemon flag is not sufficient as the connector can be shut down without shutting down the container; to do it "right" the thread pool has to be shut down after the connector is shut down. - If this is not possible to do with an external executor-based thread pool, then we simply shouldn't allow users to configure an executor for AJP within the subsystem configuration. - If this *generally* not possible to shut down the thread pool after shutting down the connector, then it's a real bug and we have to address that. Mathieu Lachance <mathieu> made a comment on jira AS7-6105 I guess this is the same concern as expressed here : https://issues.jboss.org/browse/AS7-5771 Eiichi Nagai <enagai> updated the status of jira AS7-6105 to Reopened Eiichi Nagai <enagai> made a comment on jira AS7-6105 I have reopened this JIRA. I am interested in David's comment. I believe that you tackle again regarding this behavior. Can someone chose the right component please? Issue is still valid for EAP 6.3.0.ER10. This executor configuration looks obviously broken (same as 1042754). I don't see a real way to handle this. It should probably change to WONT_FIX. based on comment 15 and comment 16 |
Description of problem: When AJP connecotr uses thread pool executor configuration[1], AJP's worker thread is waited by AjpProcessor.read()[2]. QueueExecuter recognizes it as active thread. Therefor, unless httpd server shut down, EAP server shutdown process does not finish forever. [1] standalone.xml <subsystem xmlns="urn:jboss:domain:threads:1.1"> <bounded-queue-thread-pool name="http-executor"> <queue-length count="1"/> <max-threads count="1"/> </bounded-queue-thread-pool> </subsystem> --- snip --- <subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" native="false"> <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http"/> <connector name="ajp" protocol="AJP/1.3" scheme="http" socket-binding="ajp" executor="http-executor"/> <virtual-server name="default-host" enable-welcome-root="true"> <alias name="localhost"/> <alias name="example.com"/> </virtual-server> </subsystem> [2] "http-executor-threads - 1" prio=6 tid=0x55515000 nid=0x20c0 runnable [0x5590f000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:1131) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:1213) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:451) at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:452) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:519) at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:33) at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:801) at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:45) at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:821) at java.lang.Thread.run(Thread.java:662) at org.jboss.threads.JBossThread.run(JBossThread.java:122) Version-Release number of selected component (if applicable): 6.0 GA 6.0.1 (AS 7.1.3.Final-redhat-4) How reproducible: Always (when using thread pool executor) Steps to Reproduce: 1. Configure thread pool executor to AJP connector like [1]. 2. Connect to AJP connector in EAP6 via httpd. 3. You can confirm AJP worker thread likea like [2]. 4. Shut down EAP server 5. You can confirm that shutdown process does not finish. Actual results: EAP shutdown process hangs unless httpd server shut down. Expected results: A graceful shutdown of EAP server even if httpd server does not shut down. Additional info: N/A