Bug 1040304 - [performance] php application becomes unavailable after continuous visit for a long time
Summary: [performance] php application becomes unavailable after continuous visit for ...
Keywords:
Status: CLOSED EOL
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: ImageStreams
Version: 2.2.0
Hardware: Unspecified
OS: Unspecified
low
high
Target Milestone: ---
: ---
Assignee: Timothy Williams
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-12-11 07:13 UTC by Gaoyun Pei
Modified: 2017-01-13 22:36 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-13 22:36:06 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Gaoyun Pei 2013-12-11 07:13:51 UTC
Description of problem:
The php application becomes unavailable after continuous visit, and the httpd processes generated by the app didn't dissapear after stopping the visit. 

Version-Release number of selected component (if applicable):
2.0/2013-11-26.1
openshift-origin-cartridge-php-1.17.3-2.el6op.noarch
httpd-tools-2.2.15-29.el6_4.x86_64
httpd-2.2.15-29.el6_4.x86_64

How reproducible:
Always

Steps to Reproduce:
1.Create a php-5.3 application, make sure it's available. Check the processes it owns:
[php1-55555.longevity-2.com 5295992b58c8341e3100005e]\> ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
1001     11763     1  0 04:19 ?        00:00:08 /usr/sbin/httpd -C Include /var/lib/openshift/5295992b58c8341e3100005e/php//configuration/etc/con
1001     23129 23118  0 05:47 ?        00:00:00 sshd: 5295992b58c8341e3100005e@pts/1
1001     23130 23129  0 05:47 pts/1    00:00:00 /bin/bash --init-file /usr/bin/rhcsh -i
1001     23337 11763  2 05:47 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/5295992b58c8341e3100005e/php//configuration/etc/con
1001     23342 11763  2 05:47 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/5295992b58c8341e3100005e/php//configuration/etc/con
1001     23771 11763  0 05:48 ?        00:00:00 [rotatelogs] <defunct>
1001     23772 23130  1 05:48 pts/1    00:00:00 ps -ef


2.Visit the app with a large number
Here I use curl-loader tool, 30 users visit the app concurrently with a 500ms interval. 
After running uninterruptedly for nearly 20 hours, the php app becomes unavailable. Stop the curl-loader, wait for a while, the app is still not available.
[root@broker longevity]# rhc app show php1 --gears
ID                       State   Cartridges Size  SSH URL
------------------------ ------- ---------- ----- ---------------------------------------------------
5295992b58c8341e3100005e started php-5.3    small 5295992b58c8341e3100005e.com

[root@broker longevity]# curl -I php1-55555.longevity-2.com
HTTP/1.1 502 Proxy Error
Date: Wed, 11 Dec 2013 06:14:15 GMT
Connection: close
Content-Type: text/html; charset=iso-8859-1

3.Log into the app, found the app had 61 httpd processes running, most of them should be terminated when there's no more http request.
[php1-55555.longevity-2.com 5295992b58c8341e3100005e]\> ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
1001     11763     1  0 Dec10 ?        00:00:28 /usr/sbin/httpd -C Include /var/lib/openshift/5295992b58c8341e3100005e/php//configuration/etc/con
1001     15440 11763  0 Dec10 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/5295992b58c8341e3100005e/php//configuration/etc/con
1001     15881 11763  0 Dec10 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/5295992b58c8341e3100005e/php//configuration/etc/con
1001     15896 11763  0 Dec10 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/5295992b58c8341e3100005e/php//configuration/etc/con
1001     15898 11763  0 Dec10 ?        00:00:00 /usr/sbin/httpd -C Include 
...

[php1-55555.longevity-2.com 5295992b58c8341e3100005e]\> ps -ef|grep httpd|grep -v grep |wc
     61     915   15677

4.After restart the app, it becomes availale again, and the redundant httpd processes are gone.

Actual results:

Expected results:
The php app could handle well with the continuous visit, and when there's no more http request, the new httpd processes should be terminated.

Additional info:
The result from curl-loader:
Test total duration was 74898 seconds and CAPS average 11:
H/F   Req:659725,1xx:0,2xx:658988,3xx:0,4xx:0,5xx:0,Err:708,T-Err:233895,D:2ms,D-2xx:2ms,Ti:47555B/s,To:1537B/s
H/F/S Req:0,1xx:0,2xx:0,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:0ms,D-2xx:0ms,Ti:0B/s,To:0B/s
 Operations:		 Success		 Failed			Timed out
URL0:php1        	0      658988  		17     234603  		17     233895  

This issue only happens on php, other type of cartridges could handle this well.

Comment 2 Rory Thrasher 2017-01-13 22:36:06 UTC
OpenShift Enterprise v2 has officially reached EoL.  This product is no longer supported and bugs will be closed.

Please look into the replacement enterprise-grade container option, OpenShift Container Platform v3.  https://www.openshift.com/container-platform/

More information can be found here: https://access.redhat.com/support/policy/updates/openshift/


Note You need to log in before you can comment on or make changes to this bug.