Bug 826376 - app gear stops itself sometimes
app gear stops itself sometimes
Status: CLOSED WORKSFORME
Product: OpenShift Origin
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Unspecified
medium Severity low
: ---
: ---
Assigned To: Ram Ranganathan
libra bugs
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-30 01:38 EDT by Linqing Lu
Modified: 2015-05-14 18:54 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-07-16 20:25:08 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Linqing Lu 2012-05-30 01:38:58 EDT
Description of problem:
An user reported his app showed haproxy statistics page when being visited.
A app restart would cure this. But it came again today (second time this week).

runtime/.state file said "started" but `rhc app status` reported:
 Application '119458b26d' is either stopped or inaccessible

APP URL:  http://social-map.rhcloud.com/


Version-Release number of selected component (if applicable):
production env, May.30


Logs:

shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
==> social/logs/error_log-20120525-000000-EST <==
[Thu May 24 23:23:44 2012] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:libra_t:s0:c1,c977
[Thu May 24 23:23:44 2012] [notice] mod_bw : Memory Allocated 32 bytes (each conf takes 32 bytes)
[Thu May 24 23:23:44 2012] [notice] mod_bw : Version 0.8 - Initialized [1 Confs]
[Thu May 24 23:23:44 2012] [notice] Apache/2.2.15 (Unix) mod_wsgi/3.2 Python/2.6.6 configured -- resuming normal operations

==> social/logs/validate_config.log <==
# Logfile created on Thu May 24 23:23:55 -0400 2012 by logger.rb/1.2.6

==> social/logs/scale_events.log <==
I, [2012-05-25T06:05:05.605665 #27001]  INFO -- : Starting haproxy_ctld
D, [2012-05-25T06:05:05.608019 #27001] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2012-05-25T06:43:09.081328 #677]  INFO -- : Starting haproxy_ctld
D, [2012-05-25T06:43:09.083765 #677] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2012-05-28T12:06:54.600083 #32366]  INFO -- : Starting haproxy_ctld
D, [2012-05-28T12:06:54.602273 #32366] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2012-05-29T02:32:26.083486 #30255]  INFO -- : Starting haproxy_ctld
D, [2012-05-29T02:32:26.085209 #30255] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2012-05-29T02:33:12.897398 #31491]  INFO -- : Starting haproxy_ctld
D, [2012-05-29T02:33:12.898723 #31491] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
Comment 1 Rob Millner 2012-05-30 13:02:46 EDT
Dropping severity to low, affects only one user occasionally.
Comment 2 Ram Ranganathan 2012-07-13 23:10:35 EDT
@Linqinq, is this still an issue?? There was a fix made to the haproxy cart to check the serving gears on startup.
Comment 3 Ram Ranganathan 2012-07-13 23:12:13 EDT
Wanted to add and the app (all it says is coming soon ...) seems to be up and serving.
Comment 4 Linqing Lu 2012-07-14 00:00:37 EDT
Yep, I'm seeing the same page. Guess it's working this moment. Will confirm with the owner of this app.

@Pengfei, have you met this same issue lately?
Comment 5 Pengfei Xue 2012-07-15 21:42:47 EDT
(In reply to comment #4)
> Yep, I'm seeing the same page. Guess it's working this moment. Will confirm
> with the owner of this app.
> 
> @Pengfei, have you met this same issue lately?

Actually, i have destroyed the scaling app after i have come across this error, now it's a normal app, but i do find some other guys got this error and posted a thread in openshift forum.


ps:
https://openshift.redhat.com/community/forums/openshift/haproxy-page-instead-of-the-web-application

https://openshift.redhat.com/community/forums/openshift/getting-haproxy-page-instead-of-my-application-page
Comment 6 Ram Ranganathan 2012-07-16 00:04:18 EDT
@Pengfei, you will get the haproxy page if the app server is not serving -- and that could be for various reasons including code errors, db errors etc. The default haproxy config is to show the haproxy-status page if all the serving gears are down (no app server is serving). 

In the examples you cited, in one of those cases the db was down and in the other the app code itself had an error and the app server was returning 503.
Comment 7 Pengfei Xue 2012-07-16 01:01:11 EDT
(In reply to comment #6)
> @Pengfei, you will get the haproxy page if the app server is not serving --
> and that could be for various reasons including code errors, db errors etc.
> The default haproxy config is to show the haproxy-status page if all the
> serving gears are down (no app server is serving). 
> 
> In the examples you cited, in one of those cases the db was down and in the
> other the app code itself had an error and the app server was returning 503.


Ok, thanks for replying, I don't have any further questions for this, but we should review this bug if someone else get this issue in the future(hope it wont :)
Comment 8 Ram Ranganathan 2012-07-16 20:25:08 EDT
@Pengfei, thanks. Agreed, that sounds like a good plan. 

Going to close this bug out for now as did put in an earlier fix into haproxy to "ping" the serving gears on startup.

Note You need to log in before you can comment on or make changes to this bug.