Red Hat Bugzilla – Bug 806473
rhc-admin-move doesn't respect the -t option
Last modified: 2015-05-14 18:52:46 EDT
Description of problem: While doing moves, we had one move that took a long time to run chcon / restorecon because it had many, many files. It took so long that the move would time out and fail, even though there was no problem at all. While working with Dan, we tried specifying -t, but that didn't help. We even tried upping the timeout value in the broker's production.rb file to over 60 seconds (from 120 to 240). This also had no affect. Every time, the move would time out right at 60 seconds (we could see this in the mcollective log file of the destination node). Version-Release number of selected component (if applicable): 2.0.6, not sure what RPM versions. How reproducible: With the right app, very. The app has to have a ton of files so that the move takes longer than 60 seconds. Steps to Reproduce: 1. Move an app that takes more than 60 seconds 2. Notice that the move fails at exactly 60 seconds. 3. Specify a -t of 120 seconds. 4. Notice that it had no effect and that the move still failed after 60 seconds. Actual results: The move times out at 60 seconds no matter what. Expected results: The -t option to rhc-admin-move should be able to determine how long before the move times out.
Note that the -n param is now -a The max time is now set to 240 secs. We can increase if needed.
Verified this bug with devenv_1677, and PASS. 1. Create 30000 file in app 2. Log into instance run rhc-admin-move successfully without timeout 3. Run rhc-admin-move successfully with timeout (5) Now it failed. ######################## Body End ############################ DEBUG: Stopping existing app 'phptest' before moving DEBUG: Force stopping existing app 'phptest' before moving DEBUG: Reserved uid '' on district: 'NONE' DEBUG: Creating new account for app 'phptest' on ip-10-224-109-199 DEBUG: Moving content for app 'phptest' to ip-10-224-109-199 Identity added: /var/www/stickshift/broker/config/keys/rsync_id_rsa (/var/www/stickshift/broker/config/keys/rsync_id_rsa) Agent pid 16936 DEBUG: Performing cartridge level move for 'phptest' on ip-10-224-109-199 DEBUG: Moving failed. Rolling back 'phptest' with remove-httpd-proxy on 'ip-10-224-109-199' DEBUG: Moving failed. Rolling back 'phptest' with destroy on 'ip-10-224-109-199' URL: http://phptest-jialiu.dev.rhcloud.com /var/www/stickshift/broker/lib/express/broker/application_container_proxy.rb:745:in `parse_result': Node execution failure (error getting result from node). If the problem persists please contact Red Hat support. (StickShift::NodeException) from /var/www/stickshift/broker/lib/express/broker/application_container_proxy.rb:129:in `destroy' from /var/www/stickshift/broker/lib/express/broker/application_container_proxy.rb:540:in `move_app' from /usr/bin/rhc-admin-move:114 So timeout take effect now.