Bug 1025261 - Failed to snapshot restore after migration for cartridge(php/python-3.3/python-2.6/nodejs/ruby-1.8 at least)
Summary: Failed to snapshot restore after migration for cartridge(php/python-3.3/pytho...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Online
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Dan Mace
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-10-31 10:41 UTC by zhaozhanqi
Modified: 2015-06-02 02:37 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-01-24 03:28:14 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description zhaozhanqi 2013-10-31 10:41:54 UTC
Description of problem:
Given some app prepared. after migration. snapshot restore will be failed.


Version-Release number of selected component (if applicable):
devenv_3973

How reproducible:
always


Steps to Reproduce:
1.Prepare applications in devenv-stage_528 instance
2. Upgrade the devent-stage instance to latest as devenv_3973
3. After upgrade, clear broker cache, restart broker and mcollective services
4. Do datastore migration
  rhc-admin-migrate-datastore --compatible --version 2.0.35
5. Do gear migration
# oo-admin-upgrade upgrade-node --version 2.0.35 --ignore-cartridge-version
6. snapshot save this app
7. change some thing in the repo and git push
8. snapshot restore this app 
Actual results:

rhc snapshot restore zqnodejs -n zqd1
Restoring from snapshot zqnodejs.tar.gz...
Removing old git repo: ~/git/zqnodejs.git/
Removing old data dir: ~/app-root/data/*
Restoring ~/git/zqnodejs.git and ~/app-root/data
Activation status: failure
Activation failed for the following gears:
error: undefined method `values' for nil:NilClass. Use --trace to view backtrace
Error in trying to restore snapshot. You can try to restore manually by running:
cat 'zqnodejs.tar.gz' | ssh 5271c65d5c116d831c000004.rhcloud.com 'restore INCLUDE_GIT'


Expected results:
snapshot could be succeed.

Additional info:

Comment 1 Andy Goldstein 2013-10-31 12:35:42 UTC
Could you please provide full details of the app(s) that failed:

- cartridge(s)
- scalable or not?
- # of gears (if scalable)

Comment 2 Paul Morie 2013-10-31 19:35:48 UTC
The cause of this was that the activate method does not return the same schema if the 'all' or 'gears' option isn't specified.

Comment 3 openshift-github-bot 2013-10-31 20:30:24 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/d3ffef632823662978f5ebdbcd4ae528587d9d87
Fix bug 1025261: call activate with 'all' during restore

Comment 4 Meng Bo 2013-11-01 10:36:08 UTC
Checked after migration to latest stage_branch from devenv-stage_528, both snapshot and restore can work well.

Pulling down a snapshot to redmine.tar.gz...
Creating and sending tar.gz
RESULT:
Success

Pulling down a snapshot to drupal.tar.gz...
Creating and sending tar.gz
RESULT:
Success

Pulling down a snapshot to jbeap1s.tar.gz...
Saving snapshot for secondary mongodb-2.2 gear
Creating and sending tar.gz
Saving snapshot for secondary postgresql-8.4 gear
Creating and sending tar.gz
Saving snapshot for secondary mysql-5.1 gear
Creating and sending tar.gz
Creating and sending tar.gz
RESULT:
Success

Pulling down a snapshot to php1s.tar.gz...
Saving snapshot for secondary mysql-5.1 gear
Creating and sending tar.gz
Saving snapshot for secondary mongodb-2.2 gear
Creating and sending tar.gz
Creating and sending tar.gz
RESULT:
Success



Restoring from snapshot redmine.tar.gz...
Removing old git repo: ~/git/redmine.git/
Removing old data dir: ~/app-root/data/*
Restoring ~/git/redmine.git and ~/app-root/data
Old and new names are the same, no need to rename
MySQL already running
/opt/rh/ruby193/root/usr/bin/ruby /var/lib/openshift/5273407c83aa6edfa9000027/app-root/runtime/repo/vendor/bundle/ruby/1.9.1/bin/rake assets:precompile:all RAILS_ENV=production RAILS_GROUPS=assets
Activation status: success

RESULT:
Success
Restoring from snapshot drupal.tar.gz...
Removing old git repo: ~/git/drupal.git/
Removing old data dir: ~/app-root/data/*
Restoring ~/git/drupal.git and ~/app-root/data
Old and new names are the same, no need to rename
MySQL already running
Activation status: success

RESULT:
Success
Restoring from snapshot jbeap1s.tar.gz...
Removing old git repo: ~/git/jbeap1s.git/
Removing old data dir: ~/app-root/data/*
Restoring ~/git/jbeap1s.git and ~/app-root/data
Restoring snapshot for mongodb-2.2 gear
Removing old data dir: ~/app-root/data/*
Restoring ~/app-root/data
connected to: 127.1.247.1:27017
Fri Nov  1 03:16:29.054 dump/admin/system.users.bson
Fri Nov  1 03:16:29.054 	going into namespace [admin.system.users]
1 objects found
Fri Nov  1 03:16:29.061 	Creating index: { key: { _id: 1 }, ns: "admin.system.users", name: "_id_" }
Fri Nov  1 03:16:29.062 	Creating index: { key: { user: 1, userSource: 1 }, unique: true, ns: "admin.system.users", name: "user_1_userSource_1" }
Fri Nov  1 03:16:29.062 dump/jbeap1s/openshift.bson
Fri Nov  1 03:16:29.062 	going into namespace [jbeap1s.openshift]
Fri Nov  1 03:16:29.062 	 dropping
1 objects found
Fri Nov  1 03:16:29.132 	Creating index: { key: { _id: 1 }, ns: "jbeap1s.openshift", name: "_id_" }
Fri Nov  1 03:16:29.169 dump/jbeap1s/system.users.bson
Fri Nov  1 03:16:29.169 	going into namespace [jbeap1s.system.users]
1 objects found
Fri Nov  1 03:16:29.169 	Creating index: { key: { _id: 1 }, ns: "jbeap1s.system.users", name: "_id_" }
Fri Nov  1 03:16:29.173 	Creating index: { key: { user: 1, userSource: 1 }, unique: true, ns: "jbeap1s.system.users", name: "user_1_userSource_1" }
Restoring snapshot for postgresql-8.4 gear
Removing old data dir: ~/app-root/data/*
Restoring ~/app-root/data
ERROR:  database "jbeap1s" already exists
Restoring snapshot for mysql-5.1 gear
Removing old data dir: ~/app-root/data/*
Restoring ~/app-root/data
Old and new names are the same, no need to rename
Activation status: success

RESULT:
Success
Restoring from snapshot php1s.tar.gz...
Removing old git repo: ~/git/php1s.git/
Removing old data dir: ~/app-root/data/*
Restoring ~/git/php1s.git and ~/app-root/data
Restoring snapshot for mysql-5.1 gear
Removing old data dir: ~/app-root/data/*
Restoring ~/app-root/data
Old and new names are the same, no need to rename
Restoring snapshot for mongodb-2.2 gear
Removing old data dir: ~/app-root/data/*
Restoring ~/app-root/data
connected to: 127.1.246.129:27017
Fri Nov  1 03:18:12.908 dump/php1s/openshift.bson
Fri Nov  1 03:18:12.908 	going into namespace [php1s.openshift]
Fri Nov  1 03:18:12.908 	 dropping
1 objects found
Fri Nov  1 03:18:12.916 	Creating index: { key: { _id: 1 }, ns: "php1s.openshift", name: "_id_" }
Fri Nov  1 03:18:12.922 dump/php1s/system.users.bson
Fri Nov  1 03:18:12.922 	going into namespace [php1s.system.users]
1 objects found
Fri Nov  1 03:18:12.923 	Creating index: { key: { _id: 1 }, ns: "php1s.system.users", name: "_id_" }
Fri Nov  1 03:18:12.924 	Creating index: { key: { user: 1, userSource: 1 }, unique: true, ns: "php1s.system.users", name: "user_1_userSource_1" }
Fri Nov  1 03:18:12.924 dump/admin/system.users.bson
Fri Nov  1 03:18:12.924 	going into namespace [admin.system.users]
1 objects found
Fri Nov  1 03:18:12.925 	Creating index: { key: { _id: 1 }, ns: "admin.system.users", name: "_id_" }
Fri Nov  1 03:18:12.926 	Creating index: { key: { user: 1, userSource: 1 }, unique: true, ns: "admin.system.users", name: "user_1_userSource_1" }
Activation status: success

RESULT:
Success


Note You need to log in before you can comment on or make changes to this bug.