Bug 1006031 - rhc snapshot restore app_name -f poc_app.tar.gz may not work from Fedora 19
rhc snapshot restore app_name -f poc_app.tar.gz may not work from Fedora 19
Status: CLOSED CURRENTRELEASE
Product: OpenShift Online
Classification: Red Hat
Component: Command Line Interface (Show other bugs)
1.x
Unspecified Linux
unspecified Severity medium
: ---
: ---
Assigned To: Jordan Liggitt
libra bugs
: UpcomingRelease
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-09-09 15:58 EDT by Anderson Silva
Modified: 2013-09-25 04:25 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-19 12:50:15 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Anderson Silva 2013-09-09 15:58:40 EDT
rhc snapshot restore isn't working on a simple PHP app in Fedora 19. It works fine on CentOS/RHEL and Mac that I've seen

This issue was experencied on Openshift Enterprise, going from one environment to another. It happened everytime.

rhc gem version: 1.13.6 or 1.12.4

The problem is the remote call to restore the file should look something like this:
 
DEBUG: cat phprestore.tar.gz | ssh 521f7c4e25535ddce300003c@test-ansilva.itos.redhat.com 'restore INCLUDE_GIT'
 
But on Fedora 19 it looks like this:
 
DEBUG: cat phprestore.tar.gz | ssh 521f7c4e25535ddce300003c@test-ansilva.itos.redhat.com 'restore'

As a work around, if the person just call up the whole command and add INCLUDE_GIT to it, their app should restore. (at least the php ones).
Comment 1 Wei Sun 2013-09-10 01:40:18 EDT
Check this bug on devenv_3762 with rhc 1.13.6 and rhc 1.14.4 in Fedora 19,could not reproduce this bug.

[openshift@dhcp-10-143 test]$ rhc snapshot restore phpt -f phpt.tar.gz -d
DEBUG: Using config file /home/openshift/.openshift/express.conf
DEBUG: Authenticating with RHC::Auth::Token
DEBUG: Connecting to https://ec2-54-226-37-232.compute-1.amazonaws.com/broker/rest/api
DEBUG: Getting all domains
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5
DEBUG: Using token authentication
DEBUG: Request GET https://ec2-54-226-37-232.compute-1.amazonaws.com/broker/rest/api
DEBUG: SSL Verification failed -- Using self signed cert
DEBUG:    code 200 1329 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6
DEBUG:    Using API version 1.5
DEBUG: Client API version 1.5 is not current. Refetching API
DEBUG: Using token authentication
DEBUG: Request GET https://ec2-54-226-37-232.compute-1.amazonaws.com/broker/rest/api
DEBUG:    code 200  322 ms
DEBUG: Using token authentication
DEBUG: Request GET https://ec2-54-226-37-232.compute-1.amazonaws.com/broker/rest/domains
DEBUG: SSL Verification failed -- Using self signed cert
DEBUG:    code 200 1127 ms
DEBUG: Getting all domains
DEBUG: Using token authentication
DEBUG: Request GET https://ec2-54-226-37-232.compute-1.amazonaws.com/broker/rest/domains/wsundev3762/applications/phpt
DEBUG:    code 200  291 ms
Restoring from snapshot phpt.tar.gz...
DEBUG: cat phpt.tar.gz | ssh 522ea8ca9e48bb10cb0000f4@phpt-wsundev3762.dev.rhcloud.com 'restore INCLUDE_GIT'
Removing old git repo: ~/git/phpt.git/
Removing old data dir: ~/app-root/data/*
Restoring ~/git/phpt.git and ~/app-root/data

RESULT:
Success
Comment 2 Clayton Coleman 2013-09-10 17:03:37 EDT
The model in snapshot restore is somewhat flawed - we shouldn't be looking in the repo to determine whether to INCLUDE_GIT - instead, we should be asking users for that.  INCLUDE_GIT runs a build prior to the restore, but a lot of the logic we're assuming can be pushed into the gear.
Comment 3 Clayton Coleman 2013-09-10 17:08:22 EDT
We need to revisit the whole restore stack code and determine what a better solution is.
Comment 4 Anderson Silva 2013-09-10 17:10:44 EDT
I believe this is the "save" I made that always fails because INCLUDE_GIT doesn't show on the remote command:

http://file.rdu.redhat.com/~ansilva/phpldap.tar.gz
Comment 5 Clayton Coleman 2013-09-11 14:38:48 EDT
In rhc/tar_gz.rb - we're calling system "<> arg1 arg2 arg3" - when args have spaces, the command fails.  Use system "command", "arg1", "arg2", etc.
Comment 6 Clayton Coleman 2013-09-11 15:03:08 EDT
Doesn't block release exit, will try to fix anyway
Comment 7 Jordan Liggitt 2013-09-11 16:23:28 EDT
multi-arg version of system doesn't redirect stderr/stdout like we want. wrapping args in quotes '' will handle spaces.

I'm not sure that's the issue, though, since Anderson's backup didn't have any spaces in the filename.
Comment 8 Anderson Silva 2013-09-11 16:27:51 EDT
if this helps, this is what I get with the file: (again, if it helps, this is from one Openshift Enterprise instance to another)

rhc --debug snapshot restore test -f phpldap.tar.gz                                                            
DEBUG: Using config file /home/ansilva/.openshift/express.conf
DEBUG: Authenticating with RHC::Auth::Token
DEBUG: Connecting to https://broker02.sys.itos.redhat.com/broker/rest/api
DEBUG: Getting all domains
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5
DEBUG: Using token authentication
DEBUG: Request GET https://broker02.sys.itos.redhat.com/broker/rest/api
DEBUG: SSL Verification failed -- Preverify: false, Error: unable to get local issuer certificate (20)
DEBUG:    code 200  533 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4
DEBUG:    Using API version 1.4
DEBUG: Client API version 1.4 is not current. Refetching API
DEBUG: Using token authentication
DEBUG: Request GET https://broker02.sys.itos.redhat.com/broker/rest/api
DEBUG: SSL Verification failed -- Preverify: false, Error: unable to get local issuer certificate (20)
DEBUG:    code 200  425 ms
DEBUG: Using token authentication
DEBUG: Request GET https://broker02.sys.itos.redhat.com/broker/rest/domains
DEBUG: SSL Verification failed -- Preverify: false, Error: unable to get local issuer certificate (20)
DEBUG:    code 200 1363 ms
DEBUG: Getting all domains
DEBUG: Using token authentication
DEBUG: Request GET https://broker02.sys.itos.redhat.com/broker/rest/domains/ansilva/applications/test
DEBUG: SSL Verification failed -- Preverify: false, Error: unable to get local issuer certificate (20)
DEBUG:    code 200  446 ms
Restoring from snapshot phpldap.tar.gz...
DEBUG: cat phpldap.tar.gz | ssh 521f7c4e25535ddce300003c@test-ansilva.itos.redhat.com 'restore'
Removing old data dir: ~/app-root/data/*
Restoring ~/app-root/data

RESULT:
Success
Comment 9 Jason DeTiberus 2013-09-11 16:47:48 EDT
https://github.com/openshift/rhc/pull/464 

This addresses clayton's initial request for updating system. 

Jordan, does this PR not handle stdout and stderr the way we expect it to?

Also, after trying to troubleshoot the issue, I don't think that spaces in the filename (or the path to the filename) were actually the issue, since the cat is broken as well.  The PR above addresses that.

I'll work on getting a copy of the updated gem with my PR to anderson to see if that fixes the issue for him.
Comment 10 Jordan Liggitt 2013-09-11 16:52:28 EDT
Jason, that pull fixes spaces and the :err/:out => ... redirects successfully, so the PR looks fine.

Running from master with that phpldap.tar.gz snapshot with the exact command Anderson ran, I'm seeing this:

DEBUG: cat phpldap.tar.gz | ssh 5230cc46ad8a3999b7000007@php1-bob1.dev.rhcloud.com 'restore INCLUDE_GIT'

The INCLUDE_GIT detection seems correct, even before any changes... not sure what's going on.
Comment 11 Jason DeTiberus 2013-09-11 17:01:09 EDT
Jordan, 

I saw the same thing...  I wonder if the regex it is using is getting expanded by the shell as well?  

I tried to recreate by untarring the restore file into my cwd, but it didn't seem to do the trick.

I sent Anderson a link to the updated rhc from the PR to see if it indeed fixes his problem either way.
Comment 12 Jordan Liggitt 2013-09-12 10:03:52 EDT
Issue with spaces in filename will merge in https://github.com/openshift/rhc/pull/464 

Can't recreate issue detecting INCLUDE_GIT. Moving to QE.
Comment 13 openshift-github-bot 2013-09-12 13:00:20 EDT
Commit pushed to master at https://github.com/openshift/rhc

https://github.com/openshift/rhc/commit/4a71d9afb5e122c7e0975f9d2cc2358bbaca1970
Bug 1006031 - rhc restore with spaces in filename
Comment 15 Wei Sun 2013-09-13 02:03:17 EDT
Verified  on devenv-stage_471 in Fedora 19 with rhc 1.14.7

Could snapshot restore an existent or new application with spaces in the filename.

Result:
[wsun@dhcp-8-229 dev]$ rhc snapshot restore phpws -f 'php t.tar.gz' -d
DEBUG: Using config file /home/wsun/.openshift/express.conf
DEBUG: Authenticating with RHC::Auth::Token
DEBUG: Connecting to https://ec2-54-227-89-196.compute-1.amazonaws.com/broker/rest/api
DEBUG: Getting all domains
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5
DEBUG: Using token authentication
DEBUG: Request GET https://ec2-54-227-89-196.compute-1.amazonaws.com/broker/rest/api
DEBUG: SSL Verification failed -- Using self signed cert
DEBUG:    code 200 1262 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6
DEBUG:    Using API version 1.5
DEBUG: Client API version 1.5 is not current. Refetching API
DEBUG: Using token authentication
DEBUG: Request GET https://ec2-54-227-89-196.compute-1.amazonaws.com/broker/rest/api
DEBUG:    code 200  295 ms
DEBUG: Using token authentication
DEBUG: Request GET https://ec2-54-227-89-196.compute-1.amazonaws.com/broker/rest/domains
DEBUG: SSL Verification failed -- Using self signed cert
DEBUG:    code 200 1179 ms
DEBUG: Getting all domains
DEBUG: Using token authentication
DEBUG: Request GET https://ec2-54-227-89-196.compute-1.amazonaws.com/broker/rest/domains/wsundevstg471/applications/phpws
DEBUG:    code 200  308 ms
Restoring from snapshot php t.tar.gz...
DEBUG: cat 'php t.tar.gz' | ssh 5232a8ba151b7529fe00004f@phpws-wsundevstg471.dev.rhcloud.com 'restore INCLUDE_GIT'
The authenticity of host 'phpws-wsundevstg471.dev.rhcloud.com (54.227.89.196)' can't be established.
RSA key fingerprint is 0f:50:08:ff:fd:f3:28:08:db:79:01:30:20:1a:22:e4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'phpws-wsundevstg471.dev.rhcloud.com' (RSA) to the list of known hosts.
[Fri Sep 13 01:56:49 2013] [warn] PassEnv variable SHELL was undefined
[Fri Sep 13 01:56:49 2013] [warn] PassEnv variable USER was undefined
[Fri Sep 13 01:56:49 2013] [warn] PassEnv variable LOGNAME was undefined
Removing old git repo: ~/git/phpws.git/
Removing old data dir: ~/app-root/data/*
Restoring ~/git/phpws.git and ~/app-root/data

RESULT:
Success

Note You need to log in before you can comment on or make changes to this bug.