Bug 1089075

Description of problem:

Queueless executor has a race condition that can lead to severe delays in between the time that a waitingWorker is assigned a task and the time that it is actually signaled so that it can start executing that task. This has been seen to lead to hangs of 30 minutes or more in a simple request that processes for less than 1 second.

Version-Release number of selected component (if applicable):

Jboss Threads 2.1.1.Final


How reproducible:

Very


Steps to Reproduce:
1. Configure a queueless executor for the http connector:

                <blocking-queueless-thread-pool name="HTTP">
                    <max-threads count="400"/>
                    <keepalive-time time="1" unit="hours"/>
                    <thread-factory name="http-connector-factory"/>
                </blocking-queueless-thread-pool>
    ...
                <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http" executor="HTTP"/>

2. Run JBoss with no keepalive. This helps lighter request traffic cause workers to loop through waiting states more quickly.

    -Dorg.apache.coyote.http11.Http11Protocol.MAX_KEEP_ALIVE_REQUESTS=1

3. Load JBoss up with requests (even just holding ctrl+R for about 10 seconds or so on my browser appears to be enough to hit this with keepalive off)
4. Check thread dumps for multiple Worker threads waiting on taskEnqueued (typically line 651)

Actual results:

Multiple threads end up waiting on taskEnqueued at once, which can cause delays as an execute() call signals the wrong worker.

Expected results:

There is onlly ever one worker waiting on taskEnqueued