Bug 965757

Summary: Failure to download code should result in an error message to the user
Product: OpenShift Online Reporter: Paul Morie <pmorie>
Component: ContainersAssignee: Dan Mace <dmace>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: bleanhar, bmeng, chunchen, redhatbug
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-06-11 04:07:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 964123    

Description Paul Morie 2013-05-21 16:37:44 UTC
Currently when the client specifies --from-code and the code download fails, they will see 'Node execution failure' and get a ref id.  

$ rhc app create -a ptest -t python-2.6 --from-code https://github.com/openshift/reviewboard-example --no-git -p none
Application Options
-------------------
  Namespace:   pmorie165
  Cartridges:  python-2.6
  Source Code: https://github.com/openshift/reviewboard-example
  Gear Size:   default
  Scaling:     no

Creating application 'ptest' ... 
Unable to complete the requested operation due to: Node execution failure (invalid exit
code from node).  If the problem persists please contact Red Hat support..
Reference ID: 160ea443672b2888941abf93a3a89172


This should be changed to be a descriptive error message.

Comment 1 Jhon Honce 2013-05-22 03:09:43 UTC
The code to download the git template has the messages you are requesting. This example is timing out in post-configure. The code was downloaded in configure.

I, [2013-05-21T22:52:13.092331 #20267]  INFO -- : openshift.rb:83:in `execute_action' Executing action [post-configure] using method oo_post_configure with args [{"--with-app-uuid"=>"506865982781279204016128", "--with-app-name"=>"ptest", "--with-container-uuid"=>"506865982781279204016128", "--with-container-name"=>"ptest", "--with-namespace"=>"honcenet", "--with-request-id"=>"259cbbe53c6221f4670b8974438e54ed", "--cart-name"=>"python-2.6", "--with-template-git-url"=>"https://github.com/openshift/reviewboard-example"}]
E, [2013-05-21T22:56:13.093845 #20267] ERROR -- : openshift.rb:162:in `rescue in with_container_from_args' execution expired
E, [2013-05-21T22:56:13.099740 #20267] ERROR -- : openshift.rb:163:in `rescue in with_container_from_args' /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.9.3/lib/openshift-origin-node/utils/shell_exec.rb:160:in `select'

Comment 2 Brenton Leanhardt 2013-05-23 18:39:22 UTC
It's also important that we address failures whenever /usr/bin/gear is called during post_configure as well.  eg, Some quickstarts explicitly check for the existence of another cartridge in order to start.  The rails-example does this.  It will fail to run because it can't find a mysql cartridge.  There are plenty of other ways for quickstarts to fail that are the user's fault.  We need better error handling.

Here's the patch I'm using to debug Bug #964123 which is related:
https://github.com/brenton/origin-server/commit/475f500eaf7ac1a7b255508d45efee7471bc24e0

After talking on #openshift-dev-node there's some feeling that this patch could break post_configure.

Comment 3 Dan Mace 2013-05-23 21:26:37 UTC
Resolved by https://github.com/openshift/origin-server/pull/2620

The last 10 kB of build output will now be returned to the client if there are any errors during post-configure builds.

Comment 4 openshift-github-bot 2013-05-24 16:55:52 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/5b82d9a8e714b2dc674955e01aaf91d6f591e639
Bug 965757: Provide output to client on post-configure failure

Comment 5 Meng Bo 2013-05-28 07:08:58 UTC
Checked on devenv_3278, issue has been fixed.

The build log will be shown to the console output now.

[bmeng@localhost devtest]$ rhc app create -a ptest -t python-2.6 --from-code https://github.com/openshift/reviewboard-example --no-git -p none
Application Options
-------------------
  Namespace:   bmengdev
  Cartridges:  python-2.6
  Source Code: https://github.com/openshift/reviewboard-example
  Gear Size:   default
  Scaling:     no

Creating application 'ptest' ... 
The initial build for the application failed. Last 10 kB of build output:
Stopping PYTHON cart
httpd (no pid file) not running
setup.py found.  Setting up virtualenv
New python executable in /var/lib/openshift/731948133661229604405248/python//virtenv/bin/python
Installing setuptools............done.
Installing pip...............done.
running develop
running egg_info
creating YourAppName.egg-info
writing requirements to YourAppName.egg-info/requires.txt
writing YourAppName.egg-info/PKG-INFO
writing top-level names to YourAppName.egg-info/top_level.txt
writing dependency_links to YourAppName.egg-info/dependency_links.txt
writing requirements to YourAppName.egg-info/requires.txt
writing YourAppName.egg-info/PKG-INFO
writing top-level names to YourAppName.egg-info/top_level.txt
writing dependency_links to YourAppName.egg-info/dependency_links.txt
writing manifest file 'YourAppName.egg-info/SOURCES.txt'
warning: manifest_maker: standard file 'setup.py' not found
reading manifest file 'YourAppName.egg-info/SOURCES.txt'
writing manifest file 'YourAppName.egg-info/SOURCES.txt'
running build_ext
Creating /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/YourAppName.egg-link (link to .)
Adding YourAppName 1.0 to easy-install.pth file

Installed /var/lib/openshift/731948133661229604405248/python/virtenv
Processing dependencies for YourAppName==1.0
Searching for ReviewBoard
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/ReviewBoard/
Reading http://www.reviewboard.org/
Reading http://downloads.reviewboard.org/releases/ReviewBoard/1.7/
Reading http://downloads.reviewboard.org/releases/ReviewBoard/1.6/
Reading http://downloads.reviewboard.org/releases/ReviewBoard/1.5/
Reading http://downloads.reviewboard.org/releases/ReviewBoard/1.0/
Reading http://www.review-board.org/
Reading http://downloads.review-board.org/releases/
Reading http://downloads.review-board.org/releases/ReviewBoard/1.0/
Best match: ReviewBoard 1.7.7.1
Downloading http://downloads.reviewboard.org/releases/ReviewBoard/1.7/ReviewBoard-1.7.7.1-py2.6.egg
Processing ReviewBoard-1.7.7.1-py2.6.egg
creating /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/ReviewBoard-1.7.7.1-py2.6.egg
Extracting ReviewBoard-1.7.7.1-py2.6.egg to /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages
Adding ReviewBoard 1.7.7.1 to easy-install.pth file
Installing rb-site script to /var/lib/openshift/731948133661229604405248/python/virtenv/bin
Installing rbssh script to /var/lib/openshift/731948133661229604405248/python/virtenv/bin

Installed /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/ReviewBoard-1.7.7.1-py2.6.egg
Reading http://downloads.reviewboard.org/mirror/
Searching for recaptcha-client
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/recaptcha-client/
Reading http://recaptcha.net
Reading http://recaptcha.net/
Best match: recaptcha-client 1.0.6
Downloading http://mirror1.ops.rhcloud.com/mirror/python/web/packages/source/r/recaptcha-client/recaptcha-client-1.0.6.tar.gz#md5=74228180f7e1fb76c4d7089160b0d919
Processing recaptcha-client-1.0.6.tar.gz
Running recaptcha-client-1.0.6/setup.py -q bdist_egg --dist-dir /tmp/easy_install-yUcICd/recaptcha-client-1.0.6/egg-dist-tmp-8f3LPi
zip_safe flag not set; analyzing archive contents...
Adding recaptcha-client 1.0.6 to easy-install.pth file

Installed /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/recaptcha_client-1.0.6-py2.6.egg
Searching for pytz>=2012h
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/pytz/
Reading http://pytz.sourceforge.net
Reading http://sourceforge.net/project/showfiles.php?group_id=79122
Reading http://www.stuartbishop.net/Software/pytz
Reading http://sourceforge.net/projects/pytz/
Best match: pytz 2013b
Downloading http://mirror1.ops.rhcloud.com/mirror/python/web/packages/2.6/p/pytz/pytz-2013b-py2.6.egg#md5=47c21362c5a2e9bafaa91c3f4c200a4b
Processing pytz-2013b-py2.6.egg
Moving pytz-2013b-py2.6.egg to /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages
Adding pytz 2013b to easy-install.pth file

Installed /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/pytz-2013b-py2.6.egg
Searching for python-memcached
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/python-memcached/
Reading http://www.tummy.com/Community/software/python-memcached/
Reading ftp://ftp.tummy.com/pub/python-memcached/
Best match: python-memcached 1.51
Downloading http://mirror1.ops.rhcloud.com/mirror/python/web/packages/source/p/python-memcached/python-memcached-1.51.tar.gz#md5=580603a82b7078642ed18ea000f825c5
Processing python-memcached-1.51.tar.gz
Running python-memcached-1.51/setup.py -q bdist_egg --dist-dir /tmp/easy_install-KSe1mk/python-memcached-1.51/egg-dist-tmp-XD2HVU
warning: no files found matching '*.rst'
warning: no files found matching '*.txt'
warning: no files found matching 'MakeFile'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '.gitignore' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
zip_safe flag not set; analyzing archive contents...
Adding python-memcached 1.51 to easy-install.pth file

Installed /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/python_memcached-1.51-py2.6.egg
Searching for python-dateutil==1.5
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/python-dateutil/
Reading http://labix.org/python-dateutil
Best match: python-dateutil 1.5
Downloading http://labix.org/download/python-dateutil/python-dateutil-1.5.tar.gz
Processing python-dateutil-1.5.tar.gz
Running python-dateutil-1.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Z7dMTn/python-dateutil-1.5/egg-dist-tmp-rvdSPy
Adding python-dateutil 1.5 to easy-install.pth file

Installed /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/python_dateutil-1.5-py2.6.egg
Searching for Pygments>=1.5
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/Pygments/
Best match: Pygments 1.6
Downloading http://mirror1.ops.rhcloud.com/mirror/python/web/packages/2.6/P/Pygments/Pygments-1.6-py2.6.egg#md5=2584ae5795d01cefbff0744136df3f65
Processing Pygments-1.6-py2.6.egg
creating /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/Pygments-1.6-py2.6.egg
Extracting Pygments-1.6-py2.6.egg to /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages
Adding Pygments 1.6 to easy-install.pth file
Installing pygmentize script to /var/lib/openshift/731948133661229604405248/python/virtenv/bin

Installed /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/Pygments-1.6-py2.6.egg
Searching for paramiko>=1.9.0
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/paramiko/
Best match: paramiko 1.10.1
Downloading http://mirror1.ops.rhcloud.com/mirror/python/web/packages/source/p/paramiko/paramiko-1.10.1.tar.gz#md5=4ba105e2d8535496fd633889396b20b7
Processing paramiko-1.10.1.tar.gz
Running paramiko-1.10.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Al1zvG/paramiko-1.10.1/egg-dist-tmp-TXFOte
zip_safe flag not set; analyzing archive contents...
Adding paramiko 1.10.1 to easy-install.pth file

Installed /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/paramiko-1.10.1-py2.6.egg
Searching for mimeparse>=0.1.3
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/mimeparse/
Reading http://code.google.com/p/mimeparse/
Reading http://mimeparse.googlecode.com/issues/attachment?aid=4530072823814897419&name=mimeparse-0.1.2.tar.gz
Best match: mimeparse 0.1.3
Downloading http://mimeparse.googlecode.com/files/mimeparse-0.1.3.tar.gz
Processing mimeparse-0.1.3.tar.gz
Running mimeparse-0.1.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-dIAeEa/mimeparse-0.1.3/egg-dist-tmp-JrYtI5
Adding mimeparse 0.1.3 to easy-install.pth file

Installed /var/lib/openshift/731948133661229604405248/python/virtenv/lib/python2.6/site-packages/mimeparse-0.1.3-py2.6.egg
Searching for markdown>=2.2.1
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/markdown/
Couldn't find index page for 'markdown' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/
No local packages or download links found for markdown>=2.2.1
Best match: None
Traceback (most recent call last):
  File "/var/lib/openshift/731948133661229604405248/app-root/runtime/repo/setup.py", line 10, in <module>
    dependency_links = ['https://www.djangoproject.com/download/1.4.5/tarball/#egg=Django-1.4.5',],
  File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "build/bdist.linux-i686/egg/setuptools/command/develop.py", line 27, in run
  File "build/bdist.linux-i686/egg/setuptools/command/develop.py", line 102, in install_for_development
  File "build/bdist.linux-i686/egg/setuptools/command/easy_install.py", line 519, in process_distribution
    
  File "build/bdist.linux-i686/egg/pkg_resources.py", line 563, in resolve
    distributions, errors = working_set.find_plugins(
  File "build/bdist.linux-i686/egg/pkg_resources.py", line 799, in best_match
    
  File "build/bdist.linux-i686/egg/pkg_resources.py", line 811, in obtain
    raise TypeError("Can't add %r to environment" % (other,))
  File "build/bdist.linux-i686/egg/setuptools/command/easy_install.py", line 434, in easy_install
    
  File "build/bdist.linux-i686/egg/setuptools/package_index.py", line 475, in fetch_distribution
AttributeError: 'NoneType' object has no attribute 'clone'
An error occurred executing 'gear postreceive'

For more details about the problem, try running the command again with the '--trace' option.

Comment 6 Dis 2013-06-06 15:49:56 UTC
When is it expected that this will go to production? I'm having similar issues debugging action_hooks, and the only output is:
Unable to complete the requested operation due to: Node execution failure (invalid exit code from node).  If the
problem persists please contact Red Hat support..
Reference ID: b54d455e482d5244a5ca2b67bbb3c440