Bug 983494 - Tomcat 6/7 (JBossEWS) applications not scaling correctly
Tomcat 6/7 (JBossEWS) applications not scaling correctly
Status: CLOSED DUPLICATE of bug 983216
Product: OpenShift Online
Classification: Red Hat
Component: Containers (Show other bugs)
1.x
Unspecified Windows
medium Severity high
: ---
: ---
Assigned To: Dan Mace
libra bugs
: SupportQuestion
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-11 06:24 EDT by Dan Kitchen
Modified: 2013-07-11 14:17 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-07-11 14:17:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
HAProxy output @ 3 app gears (224.15 KB, image/jpeg)
2013-07-11 06:24 EDT, Dan Kitchen
no flags Details

  None (edit)
Description Dan Kitchen 2013-07-11 06:24:21 EDT
Created attachment 772140 [details]
HAProxy output @ 3 app gears

Description of problem:

Applications created using the Tomcat 6/7 cartridge are not scaling correctly and exhibiting strange behavior. When manually or auto-scaling an application the new gears do not come online. Ssh'ing into those new gears shows all app runtime has been copied along with the jbossews dir. However some folders inside this, in particular the logs folder are unpopulated. I cannot find any information on the new gears as to why they have not started.

Secondly once the application scales to its third gear, the third one also does not come online and the the local-gear gets knocked offline. It no longer serves requests to the app's <app-name>-<domain>.rhcloud.com url. I have included a screenshot of the HAProxy-status for reference. Once the application is scaled down a step the local-gear comes back online but the extra gears never become functional.

I have tested this with my own application and also with a 'blank' (i.e. welcome index) application that has just been created.

How reproducible:

Easily

Steps to Reproduce:

1. Try to create a new Tomcat6/7 application.
2. Scale the application either manually or automatically.
3. Use HAProxy-status to observe new gear stauts

Actual results:

New gears do not come online and at 3 gears the local-gear becomes unavailable to serve requests. No log files inside jbossews dir on new gears.

Expected results:

Application should scale correctly and new tomcat gears should come online and server requests as directed by HAProxy.

Additional info:

This has been tested creating the application using RHC and using the web management page. Also this has been tested using a fresh online account and the same results have been reproduced. 

I have ensured that when manually scaling the 'disable_auto_scaling' marker is in the correct place. Further to this, this issue is arising on a basic, unpaid 3-gear sub.
Comment 1 Rajat Chopra 2013-07-11 11:41:07 EDT
The local-gear is rotated out of service as a feature. The haproxy gear begins with serving to the local gear, but it needs more elbow room when we scale to higher number of gears. Its debatable on when the weight of local-gear should be set to zero, but right now it happens when the third gear comes alive.

The other issue of why the third gear does not come up properly to serve needs investigation though, assuming that the new gear has been given enough time to come up and serve. Will get back.
Comment 2 Dan Mace 2013-07-11 14:17:09 EDT
Closing as a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=983216, resolved by https://github.com/openshift/origin-server/pull/3045.

*** This bug has been marked as a duplicate of bug 983216 ***

Note You need to log in before you can comment on or make changes to this bug.