Bug 1014454

Created attachment 806267 [details]
Customer proposed fix

Description of problem:

Reopenning connections in ClientGIOPConnection.closeAllowReopen() causes thread pool exhausts and NO_RESOURCES exception.
The problem is due to a non-customizable optimization in jacorb, by way of which it recreates a ClientConnection to the remote host:port once it receives a SocketException (connect reset) from the remote peer.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.	Our client CORBA application (that talks to our server-side CORBA application) shuts down

2.	jacorb ClientMessageReceptor thread receives a SocketException: connect reset

3.	Optimization in jacorb closes connection and reopens another in anticipation that a client will try to connect to the remote host again on the same port. Note that an actual connect is not attempted immediately, rather the thread goes into a wait() state

4.	A new instance of the client application is started that most probably binds to some other port.

5.	Subsequent connections to new client application instance happen via a new ClientMessageReceptor thread

6.	1-5 are repeated each time the client application is recycled, thus leaving behind a waiting thread every time

7.	Eventually after around a 1000 recycles cumulatively, jacorb’s thread pool exhausts and we get the NO_RESOURCES exception


Actual results:

jacorb’s thread pool exhausts and we get the NO_RESOURCES exception

Expected results:

Add customized configuration support on connection reopen.

Additional info:

The optimization lies in org.jacorb.orb.giop.ClientConnection.closeConnectionReceived()

Here you see that

((ClientGIOPConnection) connection).closeAllowReopen();

is called instead of close(), in anticipation that a new connection to the same host:port will come up. In our case, it doesn't, because the client application is respawned and binds to a different port. Therefore, the corresponding ClientMessageReceptor thread on the server waits forever in GIOPConnection.waitUntilConnected()

It is possible to customize this behavior by modifying ClientGIOPConnection.closeAllowReopen(). Please see the modified implementation I am attaching alongside. I read in jacorb's ChangeLog that it used to be this way earlier [i.e. always call close()] but then this optimization was added to allow reopening the connection on the same port.