Bug 1259767

Description of problem:

In a domain mode setup, when a server gets a "java.lang.OutOfMemoryError: PermGen space or Unable to create new native thread", it breaks the Management Interface. Sometimes the servers aren't listed or the servers just goes away.

Seems that the domain controller is waiting for an update of the host controller, but it will never finish since the JVM became irresponsible. Notice: The OOME should happens on deploy or if a server throws any OOM.

The environment went back to normal after we restart the host controller that contains the server which failed.

Version-Release number of selected component (if applicable):

6.4.2


How reproducible:

Easy to reproduce, but requires patience...

Steps to Reproduce:

JVM:

vcorrea$ java -version
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

EAP 6.4.2 and 6.4.3;

* Configure an EAP environment with 1 Domain Controller, 2 Host Controllers with 1 server each;
* Set a small size for the PermGen space (in our test was used 100mb);
* Pick a host controller and deploys as many application as you can until you have the "java.lang.OutOfMemoryError: PermGen space" during the deploy action;

Actual results:

* Once you have the OOME, go to the management console. You should notice some misbehavior with it, such as servers not being listed, or the metrics aren't being updated anymore;
* After restarting the host controller, everything should be back working again.

Expected results:

It should not impact the management console. The deployment action should recover from the error or kill the server automatically.