Bug 1032906

Summary: [origin_node_331]The application is not started after deploying an invalid binary file by artifact_url
Product: OpenShift Online Reporter: chunchen <chunchen>
Component: ImageAssignee: Dan Mace <dmace>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: agoldste, dmace, wsun
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1033768 (view as bug list) Environment:
Last Closed: 2014-01-24 03:33:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1033768    

Description chunchen 2013-11-21 08:45:52 UTC
Description of problem:
The application will be stopped after deploying  an invalid binary file by artifact_url

Version-Release number of selected component (if applicable):
devenv_4056

How reproducible:
always

Steps to Reproduce:
1. Create an app
rhc app create cphp00 php-5.3
2. Configure deployment type to binary
curl -k -H "Accept: application/xml" --user "chunchen:redhat7" https://ec2-23-22-121-187.compute-1.amazonaws.com/broker/rest/application/528dbe7ae55e7539ba00013e -X PUT -d deployment_type=binary
3. Deploy an invalid binary file to this app by artifact_url,the http://cphp-ccyd.rhcloud.com/cp1.tar.gz indeed is a ASCII text format file
curl -k -H "Accept: application/xml" --user "chunchen:redhat7" https://ec2-23-22-121-187.compute-1.amazonaws.com/broker/rest/application/528dbe7ae55e7539ba00013e/deployments -X POST -d artifact_url=http://cphp-ccyd.rhcloud.com/cp1.tar.gz
4. Check the status of this app
rhc app show cphp00 --state

Actual results:
at step 3:
<?xml version="1.0" encoding="UTF-8"?>
<response>
  <status>created</status>
  <type>deployment</type>
  <data>
    <deployment>
      <created-at>2013-11-21T03:58:41Z</created-at>
      <hot-deploy>false</hot-deploy>
      <force-clean-build>false</force-clean-build>
      <ref>master</ref>
      <sha1 nil="true"></sha1>
      <id>b525ae05</id>
      <activations>
        <activation>2013-11-21T03:58:45Z</activation>
      </activations>
    </deployment>
  </data>
  <messages>
    <message>
      <severity>info</severity>
      <text>Added b525ae05 to application cphp00</text>
      <exit-code>0</exit-code>
      <field nil="true"></field>
      <index nil="true"></index>
    </message>
  </messages>
  <version>1.6</version>
  <api-version>1.6</api-version>
  <supported-api-versions>
    <supported-api-version>1.0</supported-api-version>
    <supported-api-version>1.1</supported-api-version>
    <supported-api-version>1.2</supported-api-version>
    <supported-api-version>1.3</supported-api-version>
    <supported-api-version>1.4</supported-api-version>
    <supported-api-version>1.5</supported-api-version>
    <supported-api-version>1.6</supported-api-version>
  </supported-api-versions>
</response>

at step 4: Cartridge php-5.3 is stopped

Expected results:
1. deploy should be failed and response some messages like "Invalid Binary Artifact URL(<url>)".
2. the app should be started.

Additional info:

Comment 1 Dan Mace 2013-11-22 17:49:48 UTC
https://github.com/openshift/origin-server/pull/4220

Fixed the error reporting. The state of the application is not something we can  control in this case due to the deployments/builds being non-transactional (as a design decision). Moving this ON_QA for the improved error handling; we'll need to discuss whether to treat the transaction issue as a bug or an RFE.

Comment 2 openshift-github-bot 2013-11-22 17:59:09 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/70fc1810e4cb781543466d79905206392513e6aa
Bug 1032906: Fix generic error handling in Openshift agent

Comment 3 chunchen 2013-11-25 10:44:50 UTC
Checked on devenv_4067, we can get friendly messages for this issue now, but also meet new problem, so assign it back.

1. The new problem is that: the app can not start/restart any more after deploying an invalid binary file by artifact_url, please refer to the following results:
$ rhc app start cphp00
Unable to complete the requested operation due to: Failed to correctly execute all parallel operations - ["StartCompOp"].
Reference ID: 3421b8b95d71423a63fc45c12b876fc6
$ rhc app restart cphp00
Unable to complete the requested operation due to: Failed to correctly execute all parallel operations - ["RestartCompOp"].
Reference ID: 22cf36d21a4c05bff6bdf03692c392e8


2. The friendly messages for the issue are like below:
<?xml version="1.0" encoding="UTF-8"?>
<response>
  <status>internal_server_error</status>
  <type nil="true"></type>
  <data>
    <datum nil="true"></datum>
  </data>
  <messages>
    <message>
      <severity>debug</severity>
      <text>Error: Unable to extract deployment archive using command: /usr/bin/curl http://cphp-ccyd.rhcloud.com/cp1.tar.gz | /bin/tar -xz</text>
      <exit-code>2</exit-code>
      <field nil="true"></field>
      <index nil="true"></index>
    </message>
    <message>
      <severity>error</severity>
      <text>Unable to complete the requested operation due to: An invalid exit code (2) was returned from the server ip-10-60-87-20.  This indicates an unexpected problem during the execution of your request..
Reference ID: 5bbd9bfcc2e183cedae64ab90138bd36</text>
      <exit-code>2</exit-code>
      <field nil="true"></field>
      <index nil="true"></index>
    </message>
  </messages>
  <version>1.6</version>
  <api-version>1.6</api-version>
  <supported-api-versions>
    <supported-api-version>1.0</supported-api-version>
    <supported-api-version>1.1</supported-api-version>
    <supported-api-version>1.2</supported-api-version>
    <supported-api-version>1.3</supported-api-version>
    <supported-api-version>1.4</supported-api-version>
    <supported-api-version>1.5</supported-api-version>
    <supported-api-version>1.6</supported-api-version>
  </supported-api-versions>
</response>

Comment 4 Andy Goldstein 2013-11-25 14:07:08 UTC
If you deploy a bad artifact, you have to deploy something valid afterwards to get the app working again. There's nothing we can do to fix a bad deployment.

Comment 5 chunchen 2013-11-26 01:43:00 UTC
Check again according to Comment 4, the issue is fixed, so mark it as VERIFIED.