Bug 826376 - app gear stops itself sometimes
Summary: app gear stops itself sometimes
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: OKD
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ---
: ---
Assignee: Ram Ranganathan
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-05-30 05:38 UTC by Linqing Lu
Modified: 2015-05-14 22:54 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-07-17 00:25:08 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Linqing Lu 2012-05-30 05:38:58 UTC
Description of problem:
An user reported his app showed haproxy statistics page when being visited.
A app restart would cure this. But it came again today (second time this week).

runtime/.state file said "started" but `rhc app status` reported:
 Application '119458b26d' is either stopped or inaccessible

APP URL:  http://social-map.rhcloud.com/


Version-Release number of selected component (if applicable):
production env, May.30


Logs:

shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
==> social/logs/error_log-20120525-000000-EST <==
[Thu May 24 23:23:44 2012] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:libra_t:s0:c1,c977
[Thu May 24 23:23:44 2012] [notice] mod_bw : Memory Allocated 32 bytes (each conf takes 32 bytes)
[Thu May 24 23:23:44 2012] [notice] mod_bw : Version 0.8 - Initialized [1 Confs]
[Thu May 24 23:23:44 2012] [notice] Apache/2.2.15 (Unix) mod_wsgi/3.2 Python/2.6.6 configured -- resuming normal operations

==> social/logs/validate_config.log <==
# Logfile created on Thu May 24 23:23:55 -0400 2012 by logger.rb/1.2.6

==> social/logs/scale_events.log <==
I, [2012-05-25T06:05:05.605665 #27001]  INFO -- : Starting haproxy_ctld
D, [2012-05-25T06:05:05.608019 #27001] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2012-05-25T06:43:09.081328 #677]  INFO -- : Starting haproxy_ctld
D, [2012-05-25T06:43:09.083765 #677] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2012-05-28T12:06:54.600083 #32366]  INFO -- : Starting haproxy_ctld
D, [2012-05-28T12:06:54.602273 #32366] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2012-05-29T02:32:26.083486 #30255]  INFO -- : Starting haproxy_ctld
D, [2012-05-29T02:32:26.085209 #30255] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2012-05-29T02:33:12.897398 #31491]  INFO -- : Starting haproxy_ctld
D, [2012-05-29T02:33:12.898723 #31491] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20

Comment 1 Rob Millner 2012-05-30 17:02:46 UTC
Dropping severity to low, affects only one user occasionally.

Comment 2 Ram Ranganathan 2012-07-14 03:10:35 UTC
@Linqinq, is this still an issue?? There was a fix made to the haproxy cart to check the serving gears on startup.

Comment 3 Ram Ranganathan 2012-07-14 03:12:13 UTC
Wanted to add and the app (all it says is coming soon ...) seems to be up and serving.

Comment 4 Linqing Lu 2012-07-14 04:00:37 UTC
Yep, I'm seeing the same page. Guess it's working this moment. Will confirm with the owner of this app.

@Pengfei, have you met this same issue lately?

Comment 5 Pengfei Xue 2012-07-16 01:42:47 UTC
(In reply to comment #4)
> Yep, I'm seeing the same page. Guess it's working this moment. Will confirm
> with the owner of this app.
> 
> @Pengfei, have you met this same issue lately?

Actually, i have destroyed the scaling app after i have come across this error, now it's a normal app, but i do find some other guys got this error and posted a thread in openshift forum.


ps:
https://openshift.redhat.com/community/forums/openshift/haproxy-page-instead-of-the-web-application

https://openshift.redhat.com/community/forums/openshift/getting-haproxy-page-instead-of-my-application-page

Comment 6 Ram Ranganathan 2012-07-16 04:04:18 UTC
@Pengfei, you will get the haproxy page if the app server is not serving -- and that could be for various reasons including code errors, db errors etc. The default haproxy config is to show the haproxy-status page if all the serving gears are down (no app server is serving). 

In the examples you cited, in one of those cases the db was down and in the other the app code itself had an error and the app server was returning 503.

Comment 7 Pengfei Xue 2012-07-16 05:01:11 UTC
(In reply to comment #6)
> @Pengfei, you will get the haproxy page if the app server is not serving --
> and that could be for various reasons including code errors, db errors etc.
> The default haproxy config is to show the haproxy-status page if all the
> serving gears are down (no app server is serving). 
> 
> In the examples you cited, in one of those cases the db was down and in the
> other the app code itself had an error and the app server was returning 503.


Ok, thanks for replying, I don't have any further questions for this, but we should review this bug if someone else get this issue in the future(hope it wont :)

Comment 8 Ram Ranganathan 2012-07-17 00:25:08 UTC
@Pengfei, thanks. Agreed, that sounds like a good plan. 

Going to close this bug out for now as did put in an earlier fix into haproxy to "ping" the serving gears on startup.


Note You need to log in before you can comment on or make changes to this bug.