Description of problem: Create an app, force stop it and then the app can't be restarted. The issue can be reproduced on non-scale or scale app with cartridge perl-5.10, ruby-1.9, python-2.6, python-2.7, python-2.8. Version-Release number of selected component (if applicable): devenv_4569 How reproducible: Always Steps to Reproduce: 1. create an app with cartridge(perl-5.10, ruby-1.9, python-2.6, 2.7, 2.8) # rhc app create perl510 perl-5.10 2. force- stop the app # rhc app force-stop perl510 3. restart it # rhc app restart perl510 Actual results: [root@dhcp-65-16 qwang]# rhc app force-stop perl510 RESULT: perl510 force stopped [root@dhcp-65-16 qwang]# rhc app restart perl510 Failed to execute: 'control restart' for /var/lib/openshift/533150f587b853d763000001/perl Expected results: App should be restarted Additional info: php-5.3, nodejs-0.6, nodejs-0.10 is ok.
Created attachment 878378 [details] development log
Created attachment 878379 [details] platform log
Created attachment 878380 [details] mcollective log
Test on devenv_4570, Failed to restore snapshot after force-stop app Step： 1. force-stop app 2. do restore snapshot [lijun@ray Work]$ rhc snapshot restore -a myphp53 -f myphp53.tar.gz Restoring from snapshot myphp53.tar.gz... Removing old git repo: ~/git/myphp53.git/ Removing old data dir: ~/app-root/data/* Restoring ~/git/myphp53.git and ~/app-root/data Error in trying to restore snapshot. You can try to restore manually by running: cat 'myphp53.tar.gz' | ssh 53315d8e87b85301db0004d1.rhcloud.com 'restore INCLUDE_GIT'
(In reply to Lei Zhang from comment #4) > Test on devenv_4570, > > Failed to restore snapshot after force-stop app > > Step： > 1. force-stop app > 2. do restore snapshot > > [lijun@ray Work]$ rhc snapshot restore -a myphp53 -f myphp53.tar.gz > Restoring from snapshot myphp53.tar.gz... > Removing old git repo: ~/git/myphp53.git/ > Removing old data dir: ~/app-root/data/* > Restoring ~/git/myphp53.git and ~/app-root/data > Error in trying to restore snapshot. You can try to restore manually by > running: > cat 'myphp53.tar.gz' | ssh > 53315d8e87b85301db0004d1.rhcloud.com 'restore INCLUDE_GIT' Can you please open a separe bug for this issue?
The issue also can be reproduced on INT(devenv_4574).
The fix for original problem is here: https://github.com/openshift/origin-server/pull/5073 (should be merged in a moment. i'm looking on the snapshot issue now).
Lei Zhang: The snapshot/restore is unrelated to this bug. I tried to debug it a bit and I found that when you do restore via 'rhc', it fails with return code 255: DEBUG: code 200 210 ms Restoring from snapshot myphp53.tar.gz... DEBUG: cat 'myphp53.tar.gz' | ssh 5332a91a9cd2dc6dba000001.rhcloud.com 'restore INCLUDE_GIT' Removing old git repo: ~/git/myphp53.git/ Removing old data dir: ~/app-root/data/* Restoring ~/git/myphp53.git and ~/app-root/data DEBUG: Activating deployment Starting PHP 5.3 cartridge (Apache+mod_php) Application directory "/" selected as DocumentRoot return: 0 # <- this is from the end of control#start Error in trying to restore snapshot. You can try to restore manually by running: cat 'myphp53.tar.gz' | ssh 5332a91a9cd2dc6dba000001.rhcloud.com 'restore INCLUDE_GIT' If you do: cat 'myphp53.tar.gz' | ssh 5332a91a9cd2dc6dba000001.rhcloud.com 'restore INCLUDE_GIT' then the app is restored and started, however the return code in shell is 255 for some reason... So far I don't have any good answer for this. Also, the rhc restore should preserve the state of the gear (force-stopped) and do not attempt to start it... in any case, we should move this to a new bug.
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/d379990619948b102facd9c518b5ca738953cc2e Bug 1080381 - Fixed problem with httpd based carts restart after force-stop
Another interesting bit about the restore: ~/tmp → cat 'myphp53.tar.gz' | ssh 5332a91a9cd2dc6dba000001.rhcloud.com 'restore INCLUDE_GIT' Removing old git repo: ~/git/myphp53.git/ Removing old data dir: ~/app-root/data/* Restoring ~/git/myphp53.git and ~/app-root/data Activating deployment Starting PHP 5.3 cartridge (Apache+mod_php) Application directory "/" selected as DocumentRoot ~/tmp → echo $? 255 I think this is the primary cause of the error message rhc is giving us.
The restore bug was moved here: https://bugzilla.redhat.com/show_bug.cgi?id=1081020
Test on devenv_4586: Verified on python, ruby, perl(non-scale or scale), it works, thanks. [root@qwang 20140327]# rhc app force-stop perl510 RESULT: perl510 force stopped [root@qwang 20140327]# rhc app restart perl510 RESULT: perl510 restarted [root@qwang 20140327]# rhc app force-stop perl510s RESULT: perl510s force stopped [root@qwang 20140327]# rhc app restart perl510s RESULT: perl510s restarted