Bug 1020365 - [deploy] Providing invalid Git ref to `rhc deploy` corrupts app-deployments and breaks application
[deploy] Providing invalid Git ref to `rhc deploy` corrupts app-deployments a...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Online
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Dan Mace
libra bugs
:
: 1020862 (view as bug list)
Depends On: 1019646
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-17 10:12 EDT by Dan Mace
Modified: 2015-05-14 19:30 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-02-26 14:08:02 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Check git ref before running deploy (1.29 KB, application/mbox)
2013-10-17 13:30 EDT, Hiro Asari
no flags Details

  None (edit)
Description Dan Mace 2013-10-17 10:12:40 EDT
Description of problem:

Passing an invalid Git ref to `rhc deploy` with an app configured for `--no-auto-deploy` corrupts the app-deployments directory and leaves the gear stopped and broken.

Version-Release number of selected component (if applicable):


How reproducible:

Create a nodejs 0.6 application with `--no-auto-deploy` and run `rhc deploy` with an invalid Git ref.


Steps to Reproduce:
1. rhc app-create nodeapp nodejs-0.6
2. rhc configure-app -a nodeapp --no-auto-deploy
3. push a change to the application to receive a new Git ref and verify that the app still works
4. rhc deploy bogusrefid -a nodeapp

Actual results:

A cryptic error message such as:

Deployment of git ref 'd015767a' in progress for application nodeapp ...
Stopping NodeJS cartridge
Saving away previously installed Node modules
An error occurred executing 'gear deploy' (exit code: 1)
Error message: Shell command '
set -xe;
shopt -s dotglob;
if [ "$(find objects -type f 2>/dev/null | wc -l)" -eq "0" ]; then
  exit 0;
fi
git archive --format=tar d015767a | (cd
/var/lib/openshift/525fe917a9b79abcce000030/app-deployments/2013-10-17_09-59-49.869/repo && tar --warning=no-timestamp -xf
-);
' returned an error. rc=1
stdout: 
stderr: + shopt -s dotglob
++ find objects -type f
++ wc -l
+ '[' 28 -eq 0 ']'
+ git archive --format=tar d015767a
+ cd /var/lib/openshift/525fe917a9b79abcce000030/app-deployments/2013-10-17_09-59-49.869/repo
+ tar --warning=no-timestamp -xf -
fatal: Not a valid object name
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

For more details about the problem, try running the command again with the '--trace' option.
Success


Additionally:
* The current deployment symlink is missing from ~/app-deployments/by-id which is an invalid state/gear corruption
* The gear is now in a stopped state


Expected results:

A meaningful error message and no deployment attempt.


Additional info:
Comment 1 Hiro Asari 2013-10-17 13:30:42 EDT
Created attachment 813462 [details]
Check git ref before running deploy

A straightforward approach is to check that the given git ref is a valid one before deploying. This patch does the job.

However, this is apparently insufficient. For some reason the error message is not getting on the extended SSH data channel. As a result, 'rhc' command still thinks that the deploy command succeeded:

$ bx bin/rhc deploy foobar -a rack
Deployment of git ref 'foobar' in progress for application rack ...
Git ref foobar is invalid
Success
Comment 2 Dan Mace 2013-10-17 14:08:04 EDT
https://github.com/openshift/origin-server/pull/3916

NOTE: Please test this issue using ssh commands directly against the gear, e.g. `ssh user@host "gear deploy bogusref"` since a current rhc bug will report success even when the gear command reports failure:

https://bugzilla.redhat.com/show_bug.cgi?id=1020473
Comment 3 Andy Goldstein 2013-10-18 08:35:33 EDT
*** Bug 1020862 has been marked as a duplicate of this bug. ***
Comment 4 Hiro Asari 2013-10-18 10:20:21 EDT
Adding Bug 1019646 as dependency, due to SSH handling of 'rhc'.
Comment 5 Meng Bo 2013-10-21 07:15:57 EDT
$ ssh 52650b4c2b21cea5e0000190@nodeapp-bmengdev.dev.rhcloud.com "gear deploy aaaassss"
Git ref aaaassss is invalid


Checked on devenv_3921, issue has been fixed.

Note You need to log in before you can comment on or make changes to this bug.