Bug 964123

Summary: Error messages lost when "--from-code" fails
Product: OpenShift Container Platform Reporter: Gaoyun Pei <gpei>
Component: ContainersAssignee: Brenton Leanhardt <bleanhar>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 1.2.0CC: libra-onpremise-devel
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-06-28 15:47:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 912672, 965757, 968345    
Bug Blocks:    
Attachments:
Description Flags
broker_production.log
none
node_mcollective.log none

Description Gaoyun Pei 2013-05-17 10:05:25 UTC
Created attachment 749284 [details]
broker_production.log

Description of problem:
If try to create an app with from-code option, it would return Node execution failure.

Version-Release number of selected component (if applicable):
http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterprise/1.2/2013-05-16.1

How reproducible:
Always

Steps to Reproduce:
1.Create an app with --from-code option
[root@dhcp-8-133 workspace]# rhc app create app15 python --from-code git://github.com/openshift/django-example.git
Using python-2.6 (Python 2.6) for 'python'

Application Options
-------------------
  Namespace:   4444
  Cartridges:  python-2.6
  Source Code: git://github.com/openshift/django-example.git
  Gear Size:   default
  Scaling:     no

Creating application 'app15' ... 
Unable to complete the requested operation due to: Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat
support..
Reference ID: 03248488950f956e2158de6963e7b4ef
  
Actual results:

Expected results:
App should be created successfully

Additional info:
production.log and mcollective.log added as attchment

Comment 1 Gaoyun Pei 2013-05-17 10:06:02 UTC
Created attachment 749286 [details]
node_mcollective.log

Comment 3 Brenton Leanhardt 2013-05-23 18:33:19 UTC
This is mostly blocked by Bug #965757.  Error messages are being swallowed so there is currently no way for the user to know if the problem is with their quickstart or with the Node runtime.

Comment 4 Brenton Leanhardt 2013-05-23 20:52:47 UTC
The first error I hit while debugging this was due to Bug #912672.  The second issue is due to the fact that the python quickstart does not yet support the v2 cartridge format.

Comment 5 Gaoyun Pei 2013-05-31 10:06:37 UTC
For the two issues said in Comment 4, QE removed the abrt-addon-python.x86_64 package to work around python virtualenv issue, then create django quick-start, it could be done. So the python quickstart should support v2 cartridge format now.

[root@dhcp-8-133 workspace]# rhc app create app15 python --from-code git://github.com/openshift/django-example.git
Using python-2.6 (Python 2.6) for 'python'

Application Options
-------------------
  Namespace:   00
  Cartridges:  python-2.6
  Source Code: git://github.com/openshift/django-example.git
  Gear Size:   default
  Scaling:     no

Creating application 'app15' ... done

Waiting for your DNS name to be available ... done

Downloading the application Git repository ...
Initialized empty Git repository in /root/workspace/app15/.git/
The authenticity of host 'app15-00.osetestv2manual.com (10.66.78.214)' can't be established.
RSA key fingerprint is a3:ba:c0:09:5f:0f:13:50:8e:1e:2e:95:7f:66:ae:7c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'app15-00.osetestv2manual.com' (RSA) to the list of known hosts.

Your application code is now in 'app15'

app15 @ http://app15-00.osetestv2manual.com/ (uuid: 51a872401a181b85a5000001)
-----------------------------------------------------------------------------
  Created:         5:49 PM
  Gears:           1 (defaults to newsize)
  Git URL:         ssh://51a872401a181b85a5000001.com/~/git/app15.git/
  Initial Git URL: git://github.com/openshift/django-example.git
  SSH:             51a872401a181b85a5000001.com

  python-2.6 (Python 2.6)
  -----------------------
    Gears: 1 newsize

RESULT:
Application app15 was created.

Comment 6 Brenton Leanhardt 2013-06-03 14:38:41 UTC
If the --from-code option is generally working we should work to have this VERIFIED.

Django quickstarts will be blocked by the virtenv / abrt bug but the real fix is that we have now added debugging output that will be displayed to the user.  Here's how to test:

1.Create an app with --from-code option
[root@dhcp-8-133 workspace]# rhc app create app15 python --from-code git://github.com/openshift/django-example.git
Using python-2.6 (Python 2.6) for 'python'

Make sure abrt-addon-python is installed.  rhc should return the django error.  Googling that error message will likely find the related bug.  That's the best we can do for now.  If that works then please move this to VERIFIED.

Comment 7 Gaoyun Pei 2013-06-04 06:46:08 UTC
Test it on puddle:
http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterprise/1.2/2013-06-03.1/
It still throw out the exception that setting up virtualenv failed.
Waiting for next puddle to re-test this.


The nodes all have abrt-addon-python installed
[root@node1 ~]# rpm -qa|grep abrt-addon-python
abrt-addon-python-2.0.8-15.el6.x86_64

Using the latest rhc client:
[root@dhcp-8-133 workspace]# rpm -qa|grep rhc
rhc-1.10.1-1.git.1.ceb777a.el6op.noarch

[root@dhcp-8-133 workspace]# rhc app create app15 python --from-code git://github.com/openshift/django-example.git
Using python-2.6 (Python 2.6) for 'python'

Application Options
-------------------
  Namespace:   00
  Cartridges:  python-2.6
  Source Code: git://github.com/openshift/django-example.git
  Gear Size:   default
  Scaling:     no

Creating application 'app15' ... 
The initial build for the application failed. Last 10 kB of build output:
Stopping PYTHON cart
httpd (no pid file) not running
setup.py found.  Setting up virtualenv
'import site' failed; use -v for traceback
New python executable in /var/lib/openshift/51ad82c86e0f5d96f2000005/python//virtenv/bin/python
Installing setuptools....................................
  Complete output from command /var/lib/openshift/5...n/virtenv/bin/python -c "#!python
\"\"\"Bootstra...sys.argv[1:])
...

OSError: Command /var/lib/openshift/5...n/virtenv/bin/python -c "#!python
\"\"\"Bootstra...sys.argv[1:])

" /usr/lib/python2.6/s...ols-0.6c11-py2.6.egg failed with error code 1
An error occurred executing 'gear postreceive'

For more details about the problem, try running the command again with the '--trace' option.

Comment 8 Brenton Leanhardt 2013-06-04 12:07:37 UTC
Apologies for not being clear.  The development team had a meeting yesterday to discuss the underlying problem of this bug.  The virtenv/abrt issue is known and being tracked separately and the django quickstart v2 problem was fixed before a bug was filed.

We feel like what needs to be tested here is that when "--from-code" fails from now on users should be presented with an error message.  Before errors were being lost.  eg, download errors, system errors (the virtenv/abrt issue), errors in the user's build script were all lost.

The "The initial build for the application failed. Last 10 kB of build output:" text and the error that follows are the fix for this bug.  If you agree please mark this VERIFIED.

In the meantime QE will need to test the "--from-code" functionality using a different quickstart (or add logic to ensure the required abrt packages are removed).

Comment 9 Gaoyun Pei 2013-06-05 09:07:30 UTC
yeah, that prompt for user would be great, move this bug to VERIFIED.

Comment 10 Luke Meyer 2013-06-28 15:47:56 UTC
Closing all bugs introduced, fixed, and verified during 1.2 release work (thus never shipped).