Bug 1040780

Summary: Failed to create nodejs app due to failing to start app
Product: OpenShift Online Reporter: Lei Zhang <lzhang>
Component: ImageAssignee: Paul Morie <pmorie>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 2.xCC: andris.reinman, cdaley, jhadvig, mfojtik, nirob.std, nmone, pmorie, xtian
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-30 00:53:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
node platform-trace
none
broker development.log
none
node platform.log none

Description Lei Zhang 2013-12-12 05:56:36 UTC
Description of problem:
When creating one node-0.6 app(e.g., mynodejs06), failed to create it due to failing to start app.

[rayzhang@ray Work]$ rhc app create mynodejs06 nodejs-0.6 
Application Options
-------------------
  Domain:     chunchen
  Cartridges: nodejs-0.6
  Gear Size:  default
  Scaling:    no
Creating application 'mynodejs06' ... Application 'mynodejs06' failed to start

/var/log/openshift/broker/development.log

DEBUG: [#<MCollective::RPC::Result:0x0000000828c438 @agent="openshift", @action="cartridge_do", @results={:sender=>"ip-10-71-129-16", :statuscode=>1, :statusmsg=>"cartridge_do_action failed 157. Output Error: Failed to execute: 'control start' for /var/lib/openshift/52a922b69f2c33473f000053/nodejs\nCLIENT_MESSAGE: Starting NodeJS cartridge\nCLIENT_ERROR: Application 'mynodejs06' failed to start", :data=>{:time=>nil, :output=>"Error: Failed to execute: 'control start' for /var/lib/openshift/52a922b69f2c33473f000053/nodejs\nCLIENT_MESSAGE: Starting NodeJS cartridge\nCLIENT_ERROR: Application 'mynodejs06' failed to start", :exitcode=>157, :addtl_params=>nil}}>] (Request ID: f3f3b68eb2437b8946d8ecd60239b2c3) (pid:2688) 
DEBUG: MCollective Response Time (execute_direct: post-configure): 10.874491553s  (Request ID: f3f3b68eb2437b8946d8ecd60239b2c3) (pid:2688)
DEBUG: server results: Error: Failed to execute: 'control start' for /var/lib/openshift/52a922b69f2c33473f000053/nodejs (pid:2688)
Application 'mynodejs06' failed to start (pid:2688)

Version-Release number of selected component (if applicable):
devenv-4125

How reproducible:
always

Steps to Reproduce:
1.create nodejs-0.6 app
#rhc app create mynodejs06 nodejs-0.6 
2.
3.

Actual results:
Failed to create nodejs-0.6 due to "Application 'mynodejs06' failed to start"

Expected results:
Nodejs-0.6 app should be created successfully.

Additional info:

Comment 1 Lei Zhang 2013-12-12 05:57:40 UTC
Created attachment 835620 [details]
node platform-trace

Comment 2 Lei Zhang 2013-12-12 05:58:14 UTC
Created attachment 835621 [details]
broker development.log

Comment 3 Lei Zhang 2013-12-12 05:58:49 UTC
Created attachment 835622 [details]
node platform.log

Comment 4 Lei Zhang 2013-12-12 09:59:41 UTC
This issue doesn't exist on devenv-4126, but exist on devenv-4125 and devenv-4127

[rayzhang@ray Work]$ rhc app create mynodejs06 nodejs-0.6 
Application Options
-------------------
  Domain:     chunchen
  Cartridges: nodejs-0.6
  Gear Size:  default
  Scaling:    no

Creating application 'mynodejs06' ... Application 'mynodejs06' failed to start

Comment 5 Michal Fojtik 2013-12-12 12:47:05 UTC
We identified the problem to be this PR: https://github.com/openshift/origin-server/commit/83c292ffbb4d225523b0181b4ccec2cf03f27af2#diff-2e4b78043382112d944aa9d01dafc856L155

If I revert the changes back to the previous version, the nodejs app git push works.

jhadvig is currently working on fixing this.

Comment 6 Michal Fojtik 2013-12-12 12:48:40 UTC
Setting to high as this is affecting the current PROD.

Comment 7 Michal Fojtik 2013-12-12 14:08:16 UTC
In PROD this could be reproduced like this:

rhc app create nodeapp nodejs-0.10 --from-code=https://github.com/google/google-api-nodejs-client.git

The application will fail with the same error as above.

Comment 8 Paul Morie 2013-12-12 22:57:49 UTC
The problem that's affecting devenvs and production is that we switch to using npm start to start nodejs apps that are not using hot deploy.  Our implementation of supervisor uses the 'main' element of the package.json file, while npm start looks  for the 'scripts/start' element of the same file (ie, "scripts": { "start": xxx }}).  The npm behavior if that element isn't defined is to run 'node server.js' if there is a server.js file in the app.  So, there are many apps/quickstarts which will run correctly with how the nodejs handles supervisor, but will not work when started with 'npm start'.

As a temporary fix, I've changed the nodejs to cart to only use supervisor.

Comment 9 openshift-github-bot 2013-12-13 00:16:39 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/16e81743df8954b0ba99fc99ac1cdd8ccc597276
Fix bug 1040780: Use supervisor by default (temporary)

Comment 10 Lei Zhang 2013-12-13 05:36:53 UTC
Verified on devenv_4132

[rayzhang@ray Work]$ rhc app create mynodejs06 nodejs-0.6
Application Options
-------------------
  Domain:     chunchen
  Cartridges: nodejs-0.6
  Gear Size:  default
  Scaling:    no

Creating application 'mynodejs06' ... done


Waiting for your DNS name to be available ... done

Cloning into 'mynodejs06'...
The authenticity of host 'mynodejs06-chunchen.dev.rhcloud.com (107.22.156.25)' can't be established.
RSA key fingerprint is b4:43:ab:49:92:59:5d:e4:b2:79:fa:64:e6:cc:b3:71.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'mynodejs06-chunchen.dev.rhcloud.com' (RSA) to the list of known hosts.

Your application 'mynodejs06' is now available.

  URL:        http://mynodejs06-chunchen.dev.rhcloud.com/
  SSH to:     52aa9c004a59b423de0000b0.rhcloud.com
  Git remote: ssh://52aa9c004a59b423de0000b0.rhcloud.com/~/git/mynodejs06.git/
  Cloned to:  /home/rayzhang/Work/mynodejs06

Run 'rhc show-app mynodejs06' for more details about your app.

Comment 11 Lei Zhang 2013-12-13 05:48:26 UTC
Verified on devenv-stage-613, This issue has been fixed

rhc app create zzjmzpa9ro nodejs-0.6 -l openshifth3bb96+3 -p redhat --no-git --insecure --timeout 360

Application Options
-------------------
Domain: m10yyo8ta0
Cartridges: nodejs-0.6
Gear Size: default
Scaling: no

Creating application 'zzjmzpa9ro' ... done


Waiting for your DNS name to be available ... done

Your application 'zzjmzpa9ro' is now available.

URL: http://zzjmzpa9ro-m10yyo8ta0.dev.rhcloud.com/
SSH to: 52aa9d86be1aa9d4c00000c0.rhcloud.com
Git remote: ssh://52aa9d86be1aa9d4c00000c0.rhcloud.com/~/git/zzjmzpa9ro.git/

Run 'rhc show-app zzjmzpa9ro' for more details about your app.
Command Return: 0

Comment 12 Lei Zhang 2013-12-13 08:11:59 UTC
Check Comment #7 on devenv-stage_613, nodejs application also can be created.

[lijun@ray Work]$ rhc app create nodeapp nodejs-0.10 --from-code=https://github.com/google/google-api-nodejs-client.git
Application Options
-------------------
  Domain:      chunchen
  Cartridges:  nodejs-0.10
  Source Code: https://github.com/google/google-api-nodejs-client.git
  Gear Size:   default
  Scaling:     no

Creating application 'nodeapp' ... done


Waiting for your DNS name to be available ... done

Cloning into 'nodeapp'...

Your application 'nodeapp' is now available.

  URL:        http://nodeapp-chunchen.dev.rhcloud.com/
  SSH to:     52aac016be1aa94bdb000128.rhcloud.com
  Git remote: ssh://52aac016be1aa94bdb000128.rhcloud.com/~/git/nodeapp.git/
  Cloned to:  /home/lijun/Work/nodeapp

Run 'rhc show-app nodeapp' for more details about your app.

Comment 13 Md Nurullah 2013-12-13 11:27:22 UTC
http://s14.postimg.org/okrzbto41/screenshot_221.png

I am getting this error

Comment 14 Lei Zhang 2013-12-16 07:30:14 UTC
Verified on STG(devenv-stage_616)

[rayzhang@ray Work]$ rhc app create nodeapp nodejs-0.10 --from-code=https://github.com/google/google-api-nodejs-client.git
Application Options
-------------------
  Domain:      ray
  Cartridges:  nodejs-0.10
  Source Code: https://github.com/google/google-api-nodejs-client.git
  Gear Size:   default
  Scaling:     no

Creating application 'nodeapp' ... done


Waiting for your DNS name to be available ... done

Cloning into 'nodeapp'...
The authenticity of host 'nodeapp-ray.stg.rhcloud.com (50.19.129.28)' can't be established.
RSA key fingerprint is cf:ee:77:cb:0e:fc:02:d7:72:7e:ae:80:c0:90:88:a7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'nodeapp-ray.stg.rhcloud.com,50.19.129.28' (RSA) to the list of known hosts.

Your application 'nodeapp' is now available.

  URL:        http://nodeapp-ray.stg.rhcloud.com/
  SSH to:     52ae7e00dbd93c9f0100001b.rhcloud.com
  Git remote: ssh://52ae7e00dbd93c9f0100001b.rhcloud.com/~/git/nodeapp.git/
  Cloned to:  /home/rayzhang/Work/nodeapp

Run 'rhc show-app nodeapp' for more details about your app.
[rayzhang@ray Work]$ rhc app create mynodejs06 nodejs-0.6 
Application Options
-------------------
  Domain:     ray
  Cartridges: nodejs-0.6
  Gear Size:  default
  Scaling:    no

Creating application 'mynodejs06' ... done


Waiting for your DNS name to be available ... done

Cloning into 'mynodejs06'...
The authenticity of host 'mynodejs06-ray.stg.rhcloud.com (23.22.197.142)' can't be established.
RSA key fingerprint is cf:ee:77:cb:0e:fc:02:d7:72:7e:ae:80:c0:90:88:a7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'mynodejs06-ray.stg.rhcloud.com,23.22.197.142' (RSA) to the list of known hosts.

Your application 'mynodejs06' is now available.

  URL:        http://mynodejs06-ray.stg.rhcloud.com/
  SSH to:     52ae7ebd2587c826d80000ea.rhcloud.com
  Git remote: ssh://52ae7ebd2587c826d80000ea.rhcloud.com/~/git/mynodejs06.git/
  Cloned to:  /home/rayzhang/Work/mynodejs06

Run 'rhc show-app mynodejs06' for more details about your app.