Red Hat Bugzilla – Bug 983494
Tomcat 6/7 (JBossEWS) applications not scaling correctly
Last modified: 2013-07-11 14:17:09 EDT
Created attachment 772140 [details]
HAProxy output @ 3 app gears
Description of problem:
Applications created using the Tomcat 6/7 cartridge are not scaling correctly and exhibiting strange behavior. When manually or auto-scaling an application the new gears do not come online. Ssh'ing into those new gears shows all app runtime has been copied along with the jbossews dir. However some folders inside this, in particular the logs folder are unpopulated. I cannot find any information on the new gears as to why they have not started.
Secondly once the application scales to its third gear, the third one also does not come online and the the local-gear gets knocked offline. It no longer serves requests to the app's <app-name>-<domain>.rhcloud.com url. I have included a screenshot of the HAProxy-status for reference. Once the application is scaled down a step the local-gear comes back online but the extra gears never become functional.
I have tested this with my own application and also with a 'blank' (i.e. welcome index) application that has just been created.
Steps to Reproduce:
1. Try to create a new Tomcat6/7 application.
2. Scale the application either manually or automatically.
3. Use HAProxy-status to observe new gear stauts
New gears do not come online and at 3 gears the local-gear becomes unavailable to serve requests. No log files inside jbossews dir on new gears.
Application should scale correctly and new tomcat gears should come online and server requests as directed by HAProxy.
This has been tested creating the application using RHC and using the web management page. Also this has been tested using a fresh online account and the same results have been reproduced.
I have ensured that when manually scaling the 'disable_auto_scaling' marker is in the correct place. Further to this, this issue is arising on a basic, unpaid 3-gear sub.
The local-gear is rotated out of service as a feature. The haproxy gear begins with serving to the local gear, but it needs more elbow room when we scale to higher number of gears. Its debatable on when the weight of local-gear should be set to zero, but right now it happens when the third gear comes alive.
The other issue of why the third gear does not come up properly to serve needs investigation though, assuming that the new gear has been given enough time to come up and serve. Will get back.
Closing as a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=983216, resolved by https://github.com/openshift/origin-server/pull/3045.
*** This bug has been marked as a duplicate of bug 983216 ***