Bug 1572527

Summary:	Application pod keeps restarting if build with cakephp.json
Product:	Red Hat Software Collections	Reporter:	Wenjing Zheng <wzheng>
Component:	rh-php71-container	Assignee:	Stephen Cuppett <scuppett>
Status:	CLOSED UPSTREAM	QA Contact:	BaseOS QE - Apps <qe-baseos-apps>
Severity:	low	Docs Contact:
Priority:	medium
Version:	rh-php71	CC:	aos-bugs, bparees, jokerman, jorton, mmccomas, wzheng
Target Milestone:	---	Keywords:	Regression
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-06-05 10:51:53 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Wenjing Zheng 2018-04-27 08:47:07 UTC

Description of problem:
Pods keeps restarting when build with cakephp.json template and deploy pod will go error with below log:
$ oc logs pods/cakephp-example-2-deploy
--> Scaling cakephp-example-2 to 1
error: update acceptor rejected cakephp-example-2: pods for rc 'wzheng2/cakephp-example-2' took longer than 600 seconds to become available

Version-Release number of selected component (if applicable):
openshift v3.10.0-0.29.0
kubernetes v1.10.0+b81c8f8
etcd 3.2.16
cri-o://1.10.0-beta.1

How reproducible:
always

Steps to Reproduce:
1. oc new-app -f https://raw.githubusercontent.com/openshift/cakephp-ex/master/openshift/templates/cakephp.json 
2. oc get builds
3. oc get pods

Actual results:
$ oc get pods
NAME                       READY     STATUS      RESTARTS   AGE
cakephp-example-1-build    0/1       Completed   0          12m
cakephp-example-1-deploy   1/1       Running     0          9m
cakephp-example-1-kc99d    0/1       Running     4          9m

Expected results:
Pods should be running

Additional info:
$ oc logs pods/cakephp-example-1-kc99d
-> Cgroups memory limit is set, using HTTPD_MAX_REQUEST_WORKERS=34
=> sourcing 20-copy-config.sh ...
---> 08:33:08     Processing additional arbitrary httpd configuration provided by s2i ...
=> sourcing 00-documentroot.conf ...
=> sourcing 50-mpm-tuning.conf ...
=> sourcing 40-ssl-certs.sh ...
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.129.0.138. Set the 'ServerName' directive globally to suppress this message
[Fri Apr 27 08:33:08.896200 2018] [ssl:warn] [pid 1] AH01909: 10.129.0.138:8443:0 server certificate does NOT include an ID which matches the server name
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.129.0.138. Set the 'ServerName' directive globally to suppress this message
[Fri Apr 27 08:33:08.978301 2018] [ssl:warn] [pid 1] AH01909: 10.129.0.138:8443:0 server certificate does NOT include an ID which matches the server name
[Fri Apr 27 08:33:08.978492 2018] [http2:warn] [pid 1] AH10034: The mpm module (prefork.c) is not supported by mod_http2. The mpm determines how things are processed in your server. HTTP/2 has more demands in this regard and the currently selected mpm will just not do. This is an advisory warning. Your server will continue to work, but the HTTP/2 protocol will be inactive.
[Fri Apr 27 08:33:08.979407 2018] [lbmethod_heartbeat:notice] [pid 1] AH02282: No slotmem from mod_heartmonitor
[Fri Apr 27 08:33:09.057190 2018] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.27 (Red Hat) OpenSSL/1.0.1e-fips configured -- resuming normal operations
[Fri Apr 27 08:33:09.057249 2018] [core:notice] [pid 1] AH00094: Command line: 'httpd -D FOREGROUND'
10.129.0.1 - - [27/Apr/2018:08:33:17 +0000] "GET / HTTP/1.1" 500 86175 "-" "kube-probe/1.10+"

Comment 1 Ben Parees 2018-04-27 13:58:59 UTC

it works fine for me.

are you able to run other SCL images on your cluster?

can you try setting HTTPD_MAX_REQUEST_WORKERS to a lower value?

Comment 4 Ben Parees 2018-04-28 16:48:24 UTC

It looks like this is an issue in the sample app[1].  When the app is brought up w/o a DB(oc new-app -f https://raw.githubusercontent.com/openshift/cakephp-ex/master/openshift/templates/cakephp.json
) and you rsh into the pod and do "curl http://localhost:8080" you get 500 errors.

Accessing the app from outside the pod (via the service) or from inside the pod via the pod ip, works fine.  So there is an issue w/ the sample app's routing/listener configuration.

[1] https://github.com/sclorg/cakephp-ex

Since Steve delivered the updates to the cakephp app most recently, assigning to him for now to investigate his changes.

Note that deploying the app *with* a database also does not have this problem:
oc new-app -f https://raw.githubusercontent.com/openshift/cakephp-ex/master/openshift/templates/cakephp-mysql-persistent.json

Comment 5 Ben Parees 2018-04-28 19:17:23 UTC

(If this becomes critical for anyone, the workaround is probably to remove the liveness probe in the template.  I'm not sure why, but the readiness probe, which is identical, seems to be succeeding from what I can tell because the pod is reporting as ready and accepting traffic from the service).

Comment 6 Stephen Cuppett 2018-04-29 18:30:38 UTC

PR #97 open for this.

https://github.com/sclorg/cakephp-ex/pull/97

Comment 7 Ben Parees 2018-04-29 20:29:12 UTC

PR is merged, moving to ON_QA since the PR doesn't require a new OCP build.

Thanks Steve!

Comment 8 Joe Orton 2019-06-05 10:51:53 UTC

Looks like this is fixed in the example and doesn't need to be tracked against RHSCL.