Bug 826374

Summary:	fail to create sacalable nodejs app on integration env
Product:	OKD	Reporter:	Meng Bo <bmeng>
Component:	Containers	Assignee:	Rob Millner <rmillner>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	high	Docs Contact:
Priority:	high
Version:	2.x	CC:	admiller, mfisher, mmcgrath
Target Milestone:	---	Keywords:	Regression, Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-06-08 17:58:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Meng Bo 2012-05-30 05:35:53 UTC

Description of problem:
[root@localhost ~]# rhc app create -a no1s -t nodejs-0.6 -s -d
Password: ******

Submitting form:
rhlogin: bmeng
debug: true
Contacting https://int.openshift.redhat.com
Creating application: no1s in bmengint
/usr/lib/ruby/gems/1.8/gems/rhc-0.93.15/lib/rhc-rest.rb:144:in `process_error_response': Failed to create application no1s due to:Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support. (Rhc::Rest::ServerErrorException)
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.93.15/lib/rhc-rest.rb:142:in `each'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.93.15/lib/rhc-rest.rb:142:in `process_error_response'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.93.15/lib/rhc-rest.rb:86:in `send'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.93.15/lib/rhc-rest/domain.rb:30:in `add_application'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.93.15/lib/rhc-common.rb:499:in `create_app'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.93.15/bin/rhc-app:227:in `create_app'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.93.15/bin/rhc-app:499
	from /usr/bin/rhc-app:19:in `load'
	from /usr/bin/rhc-app:19

Version-Release number of selected component (if applicable):
int.openshift.redhat.com

How reproducible:
always

Steps to Reproduce:
1.register user on int env
2.create domain for user
3.create scalable nodejs app
rhc app create -a no1s -t nodejs-0.6 -s
  
Actual results:
get node execution failure , and app cannot be created.

Expected results:
app soould be created without problem.

Additional info:
this issue do not occur for other cartridges, 
and cannot reproduce this on devenv_1806

Comment 1 Meng Bo 2012-05-30 05:42:36 UTC

cannot reproduce this on latest devenv_1808

Comment 2 Rob Millner 2012-05-30 20:30:40 UTC

It looks like the app ctl script is failing to start after haproxy config.

D, [2012-05-30T15:41:06.179758 #29997] DEBUG -- : libra.rb:307:in `cartridge_do_action' cartridge_do_action validation = embedded/haproxy-1.4 configure 'no1s' 'rlmtest' '44ea328b76854d90b1b989ea5238f2b3'
D, [2012-05-30T15:41:09.044273 #29997] DEBUG -- : libra.rb:364:in `cartridge_do_action' cartridge_do_action ERROR (121)
------
Haproxy already stopped
SSH_KEY_ADD: AAAAB3NzaC1yc2EAAAABIwAAAQEA3unosSAfgg/4r3FzMV7J/QmYqyAeXm9BjSsewfHyUWn+b6dGudW62kHDTjsqtgk7MKK3fwpVLp5fcKzbkSx0How0SIZ+ZvOTy7clAADqKvH9FDIB07sEy57IuYm03uutyO+0l7OyZop7knKPCTPBnenFAzhVuDfmXza42HIy95yvHw+bW4XY1aNuyv+7Vhr0M2bsibcXyJKZgTVRVnmbtZeMdDl1jTd8ZXCplYO3JtQCu7mevU/vcZt+9gfFe2fUpTyZaD+xEIqpsfW5nFWsQm885jp7B6vOnXUURINxKLZGrVJprVWmM2lPL2Ty0vSulxW+wH6W2hAajYL0uMtRhw==
BROKER_AUTH_KEY_ADD: 
Failed to start /var/lib/stickshift/44ea328b76854d90b1b989ea5238f2b3/nodejs-0.6/no1s_ctl.sh

------)

Comment 3 Rob Millner 2012-05-30 20:49:19 UTC

Need to be able to instrument the ctl script on INT to see what's happening.

It doesn't reproduce on devenv so I'm declaring it not a build blocker unless it reproduces on the stage build.

Comment 4 Meng Bo 2012-06-06 06:03:17 UTC

this issue also can be reproduced on latest stage.

Comment 5 Adam Miller 2012-06-06 13:37:12 UTC

I can confirm on current stage environment.

$ rhc app create -a no1s -t nodejs-0.6 -s -d --config ~/.openshift/express-stg.conf

Submitting form:
debug: true
rhlogin: admiller
Contacting https://stg.openshift.redhat.com
Creating application: no1s in admillerstg
/usr/lib/ruby/gems/1.8/gems/rhc-0.94.1/lib/rhc-rest.rb:144:in `process_error_response': Failed to create application no1s due to:Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support. (Rhc::Rest::ServerErrorException)
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.94.1/lib/rhc-rest.rb:142:in `each'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.94.1/lib/rhc-rest.rb:142:in `process_error_response'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.94.1/lib/rhc-rest.rb:86:in `send'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.94.1/lib/rhc-rest/domain.rb:30:in `add_application'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.94.1/lib/rhc-common.rb:511:in `create_app'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.94.1/bin/rhc-app:227:in `create_app'
	from /usr/lib/ruby/gems/1.8/gems/rhc-0.94.1/bin/rhc-app:499
	from /usr/bin/rhc-app:19:in `load'
	from /usr/bin/rhc-app:19

Comment 6 Mike McGrath 2012-06-06 14:17:34 UTC

Also lets make sure there's a cucumber test for this scenario

Comment 7 Mike McGrath 2012-06-06 15:05:22 UTC

Looks like this is an issue with the haproxy embedding:

http://pastebin.test.redhat.com/91862

Line 95

Comment 8 Rob Millner 2012-06-07 19:33:15 UTC

Crankcase commit 3f4897c0dd05fcfb7417e9c36fbbec7c07593216 appears to fix the problem.

Scalable nodejs creation shouldn't have worked on devenv - this may have masked another issue.

Comment 9 Rob Millner 2012-06-07 20:43:52 UTC

Pull request accepted.

I've added an @extended test where the scalable app cucumber tests are run for all app frameworks instead of just php.

Comment 10 Meng Bo 2012-06-08 10:56:40 UTC

verified on latest stage , issue has been fixed.