Bug 1070570 - Gear creating failed due to watchman tries to restart the gear before the gear is created completely
Summary: Gear creating failed due to watchman tries to restart the gear before the gea...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Online
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Jhon Honce
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-27 06:13 UTC by Lei Zhang
Modified: 2015-05-14 23:35 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-04-09 15:19:09 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
development.log (54.64 KB, application/octet-stream)
2014-02-27 06:14 UTC, Lei Zhang
no flags Details

Description Lei Zhang 2014-02-27 06:13:15 UTC
Description of problem:
Try to create one application(e.g, non-scalable jbossas-7/jbosseap-6, jenkins server), found failed to create it due to "failed to start, Failed to execute: 'control  start' for..." error.
BTW, when remove the gear_state_plugin.rb from /etc/openshift/watchman/plugins.d and restart openshift-watchman, this issue won't happen again.


[rayzhang@ray Work]$ rhc app create myjbossas7 jbossas-7 
Application Options
-------------------
Domain:     ray
Cartridges: jbossas-7
Gear Size:  default
Scaling:    no
Creating application 'myjbossas7' ... 
Starting jbossas cartridge
jbossas process failed to start
Failed to execute: 'control start' for
/var/lib/openshift/530ea3a3be3c76f4e800002c/jbossas

Version-Release number of selected component (if applicable):
devenv_4427


How reproducible:
always

Steps to Reproduce:
1.try to create jbossas-7/jbosseap-6 app
#rhc app create myjbossas7 jbossas-7
2.
3.

Actual results:
Failed to create the app due to "Failed to execute: 'control start' for..." error

Expected results:
App created succeesfully

Additional info:
Following log appears during the gear being created.
Feb  26 22:15:16 ip-10-113-149-230 watchman[1922]: watchman restarted user  530ead943abce6635f000006: application jbas1 (retries: 0)
Feb  26 22:15:16 ip-10-113-149-230 watchman[1922]: watchman results for  530ead943abce6635f000006 restart: exitcode: 254#012stdout: Stopping gear  530ead943abce6635f0000
06  ... [  OK ]#012Starting gear 530ead943abce6635f000006 ... [  FAILED  ]#012 CLIENT_ERROR: Failed to execute: 'control start' for  /var/lib/openshift/530ead943abce6635f000006/jbossas]#012#012stderr:

Comment 1 Lei Zhang 2014-02-27 06:14:28 UTC
Created attachment 868334 [details]
development.log

Comment 2 XiuJuan Wang 2014-02-28 08:53:03 UTC
Met same issue on fork_ami_online_ui_443_1064
Could not create jenkins.

Comment 3 Jhon Honce 2014-03-13 22:44:44 UTC
Blocked on https://trello.com/c/5A50jd0o

Comment 4 Meng Bo 2014-03-18 06:50:27 UTC
Checked on fork_ami_origin_node_39_1110
After new environment STATE_CHANGE_DELAY introduced, this issue can be fixed.

Comment 5 Jhon Honce 2014-03-18 16:29:48 UTC
Fixed in https://github.com/openshift/origin-server/pull/4986

Comment 6 Lei Zhang 2014-03-19 04:39:31 UTC
Verified on devenv_4532

[rayzhang@ray Work]$ rhc app create myjbossas7 jbossas-7 
Application Options
-------------------
Domain:     chunchen
Cartridges: jbossas-7
Gear Size:  default
Scaling:    no

Creating application 'myjbossas7' ... Artifact: ./ROOT.war is still deploying
done


Waiting for your DNS name to be available ... done

Cloning into 'myjbossas7'...
Warning: Permanently added 'myjbossas7-chunchen.dev.rhcloud.com' (RSA) to the list of known hosts.

Your application 'myjbossas7' is now available.

  URL:        http://myjbossas7-chunchen.dev.rhcloud.com/
  SSH to:     53291debcaca6378c600034e.rhcloud.com
  Git remote: ssh://53291debcaca6378c600034e.rhcloud.com/~/git/myjbossas7.git/
  Cloned to:  /home/rayzhang/Work/myjbossas7

Run 'rhc show-app myjbossas7' for more details about your app.
[rayzhang@ray Work]$ rhc app create myjbosseap6 jbosseap-6
Application Options
-------------------
Domain:     chunchen
Cartridges: jbosseap-6 (addtl. costs may apply)
Gear Size:  default
Scaling:    no

Creating application 'myjbosseap6' ... done

r
Waiting for your DNS name to be available ... done

fatal: destination path 'myjbosseap6' already exists and is not an empty directory.
The directory you are cloning into already exists.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
WARNING:  Your application was created successfully but had problems during
          configuration. Below is a list of the issues and steps you can
          take to complete the configuration of your application.

  Application URL: http://myjbosseap6-chunchen.dev.rhcloud.com/

  Issues:
    1. We were unable to clone your application's git repo - The directory you are cloning into already exists.

  Steps to complete your configuration:
    1. Clone your git repo
      $ rhc git-clone myjbosseap6

  If you continue to experience problems after completing these steps,
  you can try destroying and recreating the application:

    $ rhc app delete myjbosseap6 --confirm

  Please contact us if you are unable to successfully create your
  application:

    Support - https://www.openshift.com/support

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!



Your application 'myjbosseap6' is now available.

  URL:        http://myjbosseap6-chunchen.dev.rhcloud.com/
  SSH to:     53291e8dcaca6378c6000366.rhcloud.com
  Git remote: ssh://53291e8dcaca6378c6000366.rhcloud.com/~/git/myjbosseap6.git/

Run 'rhc show-app myjbosseap6' for more details about your app.
[rayzhang@ray Work]$ rm -rf myjbosseap6
[rayzhang@ray Work]$ rhc git-clone myjbosseap6
Cloning into 'myjbosseap6'...
Warning: Permanently added 'myjbosseap6-chunchen.dev.rhcloud.com' (RSA) to the list of known hosts.
Your application Git repository has been cloned to '/home/rayzhang/Work/myjbosseap6'


Note You need to log in before you can comment on or make changes to this bug.