+++ This bug was initially created as a clone of Bug #1058251 +++ Description of problem: When doing deployment roll back for a scalable app that have multiple web gears, not every web gear is rolled back, only head web gear does. Version-Release number of selected component (if applicable): devenv_stage-655 How reproducible: Always Steps to Reproduce: 1. Create a scalable app and set min gear to 2. # rhc app-show phps --gears ID State Cartridges Size SSH URL ------------------------ ------- ------------------- ----- ---------------------------------------------------------------------------- 52e630235babf124c1000003 started php-5.3 haproxy-1.4 small 52e630235babf124c1000003.rhcloud.com 859079266427041736556544 started php-5.3 haproxy-1.4 small 859079266427041736556544.rhcloud.com 2. Configure app to enable keep-deployments # rhc app-configure phps --no-auto-deploy --keep-deployments 5 3. Did some change to app web page, and git push # vim php/index.php # git add . # git commit -am 'commit1' [master 22b2fdb] commit1 <--snip--> 4. Manually deploy the above commit. # rhc deploy 22b2fdb -a phps Deployment of git ref '22b2fdb' in progress for application phps ... Stopping PHP 5.3 cartridge (Apache+mod_php) [Mon Jan 27 05:16:02 2014] [warn] PassEnv variable SHELL was undefined [Mon Jan 27 05:16:02 2014] [warn] PassEnv variable USER was undefined [Mon Jan 27 05:16:02 2014] [warn] PassEnv variable LOGNAME was undefined Waiting for stop to finish Syncing git content to other proxy gears Building git ref '22b2fdb', commit 22b2fdb Checking deplist.txt for PEAR dependency.. Preparing build for deployment Deployment id is 63da69cc Distributing deployment to child gears Activating deployment HAProxy already running HAProxy instance is started Starting PHP 5.3 cartridge (Apache+mod_php) Success 5. Roll back the change to old one # rhc deployments phps 5:09 AM, deployment 8a4e7e84 5:16 AM, deployment 63da69cc # rhc deployment-activate 8a4e7e84 -a phps Activating deployment '8a4e7e84' on application phps ... Activating deployment Stopping PHP 5.3 cartridge (Apache+mod_php) Waiting for stop to finish HAProxy already running HAProxy instance is started Starting PHP 5.3 cartridge (Apache+mod_php) Success 6. Check page of every gears for this app. # curl phps-wsunstg655.dev.rhcloud.com # curl 859079266427041736556544-wsunstg655.dev.rhcloud.com Actual results: Only head web gear (phps-wsunstg655.dev.rhcloud.com) is rolled back, the other one (859079266427041736556544-wsunstg655.dev.rhcloud.com) dose NOT. Expected results: When rolling back, every web gear should be rolled back. Additional info: --- Additional comment from Johnny Liu on 2014-01-27 06:07:22 EST --- Client Version: rhc-1.19.4 After enable debug options at client side, seem like rhc does not call "gear remotedeploy" in head gear. # rhc deployment-activate 8a4e7e84 -a phps -d DEBUG: Using config file /root/.openshift/express.conf DEBUG: Git config 'git config --get rhc.app-id' returned '52e630235babf124c1000003' DEBUG: Git config 'git config --get rhc.app-name' returned 'phps' DEBUG: Git config 'git config --get rhc.domain-name' returned 'wsunstg655' DEBUG: Authenticating with RHC::Auth::Token DEBUG: Connecting to https://localhost/broker/rest/api DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5, 1.6 DEBUG: Using token authentication DEBUG: Created new httpclient DEBUG: Request GET https://localhost/broker/rest/api DEBUG: SSL Verification failed -- Using self signed cert DEBUG: code 200 60 ms DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6 DEBUG: Using API version 1.6 DEBUG: Client API version 1.6 is not current. Refetching API DEBUG: Using token authentication DEBUG: Request GET https://localhost/broker/rest/api DEBUG: code 200 39 ms DEBUG: Using token authentication DEBUG: Request GET https://localhost/broker/rest/domains/wsunstg655/applications/phps DEBUG: code 200 79 ms Activating deployment '8a4e7e84' on application phps ... DEBUG: Opening Net::SSH connection to phps-wsunstg655.dev.rhcloud.com, 52e630235babf124c1000003, gear activate 8a4e7e84 Activating deployment Stopping PHP 5.3 cartridge (Apache+mod_php) Waiting for stop to finish HAProxy already running HAProxy instance is started Starting PHP 5.3 cartridge (Apache+mod_php) DEBUG: Terminating ... Success Then I log into head gear, run "gear remotedeploy" manually. [phps-wsunstg655.dev.rhcloud.com 52e630235babf124c1000003]\> gear remotedeploy Preparing build for deployment Deployment id is cf80cb52 Distributing deployment to child gears Activating deployment Stopping PHP 5.3 cartridge (Apache+mod_php) Waiting for stop to finish HAProxy already running HAProxy instance is started Starting PHP 5.3 cartridge (Apache+mod_php) Then all the gears is synced, change take effect now. So this should be RHC client bug. --- Additional comment from Andy Goldstein on 2014-01-27 09:27:36 EST --- "gear remotedeploy" is only intended to be executed by CI builders. The more appropriate invocation here is "gear activate --all" --- Additional comment from Fabiano Franz on 2014-01-27 22:40:04 EST --- The rhc command was originally calling 'gear activate'. Fixed by changing to 'gear activate --all' in https://github.com/openshift/rhc/pull/537. (In reply to Andy Goldstein from comment #2) > "gear remotedeploy" is only intended to be executed by CI builders. The more > appropriate invocation here is "gear activate --all" --- Additional comment from openshift-github-bot on 2014-01-27 23:45:13 EST --- Commit pushed to master at https://github.com/openshift/rhc https://github.com/openshift/rhc/commit/669697f29d240dd263bef2a4767008206452e18f Bug 1058251 - gear activate requires --all to be applied to all gears --- Additional comment from Meng Bo on 2014-01-28 01:04:28 EST --- Checked with rhc client which built from the latest source code on github. Issue has been fixed. [bmeng@localhost php54s]$ rhc ssh php54s --gear "gear deployments" === 52e7465e7625230aa6000003 php-5.4+haproxy-1.4 Activation time - Deployment ID - Git Ref - Git SHA1 2014-01-28 00:58:42 -0500 - 23204687 - master - 120be63 - ACTIVE 2014-01-28 00:54:23 -0500 - 32371551 - master - fa8029d [52e7465e7625230aa6000003 php-5.4+haproxy-1.4] === 52e745ef762523b56d00001f php-5.4+haproxy-1.4 Activation time - Deployment ID - Git Ref - Git SHA1 2014-01-28 00:58:35 -0500 - 23204687 - master - 120be63 - ACTIVE 2014-01-28 00:54:23 -0500 - 32371551 - master - fa8029d [52e745ef762523b56d00001f php-5.4+haproxy-1.4] [bmeng@localhost php54s]$ rhc deployment-activate 32371551 Activating deployment '32371551' on application php54s ... Activating deployment Stopping PHP 5.4 cartridge (Apache+mod_php) Waiting for stop to finish HAProxy already running HAProxy instance is started Starting PHP 5.4 cartridge (Apache+mod_php) Success [bmeng@localhost php54s]$ rhc ssh php54s --gear "gear deployments" === 52e7465e7625230aa6000003 php-5.4+haproxy-1.4 Activation time - Deployment ID - Git Ref - Git SHA1 2014-01-28 01:00:50 -0500 - 32371551 - master - fa8029d - ACTIVE 2014-01-28 00:58:42 -0500 - 23204687 - master - 120be63 [52e7465e7625230aa6000003 php-5.4+haproxy-1.4] === 52e745ef762523b56d00001f php-5.4+haproxy-1.4 Activation time - Deployment ID - Git Ref - Git SHA1 2014-01-28 01:00:38 -0500 - 32371551 - master - fa8029d - ACTIVE 2014-01-28 00:58:35 -0500 - 23204687 - master - 120be63 [52e745ef762523b56d00001f php-5.4+haproxy-1.4] Move bug to verified.
The upstream commit is a one line change. Given the severity I'm proposing it for 2.0.3: commit 669697f29d240dd263bef2a4767008206452e18f Author: fabianofranz <contact> Date: Tue Jan 28 01:37:03 2014 -0200 Bug 1058251 - gear activate requires --all to be applied to all gears
Verified this bug with rhc-1.17.5.2-1.el6op.noarch, and PASS. 1. Create app. 2. # rhc app-configure -a py27mindelopyscal --no-auto-deploy --keep-deployments 3 -l jialiu -p redhat Configuring application 'py27mindelopyscal' ... done py27mindelopyscal @ http://py27mindelopyscal-jialiu.my-ose202.com.cn/ (uuid: 52fad50db04a074c970006f2) ------------------------------------------------------------------------------------------------------ Deployment: manual (use 'rhc deploy') Keep Deployments: 3 Deployment Type: git Deployment Branch: master Your application 'py27mindelopyscal' is now configured as listed above. Use 'rhc show-app py27mindelopyscal --configuration' to check your configuration values any time. 3. Do git push. 4. New deploy is not be active now. # curl 'http://py27mindelopyscal-jialiu.my-ose202.com.cn/' | grep 'title' | grep 'Welcome to OpenShift' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5233 100 5233 0 0 568k 0 --:--:-- --:--:-- --:--:-- 851k <title>Welcome to OpenShift</title> # curl 'http://52fad53fb04a074c9700071a-jialiu.my-ose202.com.cn/' | grep 'title' | grep 'Welcome to OpenShift' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5233 100 5233 0 0 570k 0 --:--:-- --:--:-- --:--:-- 851k <title>Welcome to OpenShift</title> 5. Manual active the deploy. # rhc app-deploy 300e3f0 -a py27mindelopyscal -l jialiu -p redhat 6. Check change take effects now # curl 'http://py27mindelopyscal-jialiu.my-ose202.com.cn/' | grep 'title' | grep '1' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5214 100 5214 0 0 440k 0 --:--:-- --:--:-- --:--:-- 565k <title>1</title> # curl 'http://52fad53fb04a074c9700071a-jialiu.my-ose202.com.cn/' | grep 'title' | grep '1' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5214 100 5214 0 0 168k 0 --:--:-- --:--:-- --:--:-- 188k <title>1</title> 6. # rhc deployment list -a py27mindelopyscal -l jialiu -p redhat 6:58 PM, deployment ac8da9dc 7:01 PM, deployment 46aa023d 7. Roll back the previous deployment. # rhc deployment-activate ac8da9dc -a py27mindelopyscal -l jialiu -p redhat 8. Check all gear of this app. # curl 'http://py27mindelopyscal-jialiu.my-ose202.com.cn/' | grep 'title' | grep 'Welcome to OpenShift' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5233 100 5233 0 0 1014k 0 --:--:-- --:--:-- --:--:-- 1703k <title>Welcome to OpenShift</title> # curl 'http://52fad53fb04a074c9700071a-jialiu.my-ose202.com.cn/' | grep 'title' | grep 'Welcome to OpenShift' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5233 100 5233 0 0 142k 0 --:--:-- --:--:-- --:--:-- 150k <title>Welcome to OpenShift</title>
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-0209.html