Description of problem: Version-Release number of selected component (if applicable): devenv_1786 How reproducible: Always Steps to Reproduce: 1. Create a scalable app with jenkins embeded. 2. Log into broker, move haproxy gears 3. Actual results: Fail to move haproxy gear. # rhc-admin-move --gear_uuid 875500b7aa754887b804c52d688ceb5e -i ip-10-85-3-183 /usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/cloud_user.rb:8: warning: already initialized constant DEFAULT_SSH_KEY_NAME /usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/validators/key_validator.rb:2: warning: already initialized constant KEY_NAME_MAX_LENGTH /usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/validators/key_validator.rb:3: warning: already initialized constant KEY_NAME_MIN_LENGTH /usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/validators/namespace_validator.rb:2: warning: already initialized constant NAMESPACE_MAX_LENGTH /usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/validators/namespace_validator.rb:3: warning: already initialized constant NAMESPACE_MIN_LENGTH URL: http://myapp2-jialiu.dev.rhcloud.com Login: jialiu App UUID: 875500b7aa754887b804c52d688ceb5e Gear UUID: 875500b7aa754887b804c52d688ceb5e DEBUG: Source district uuid: 4956ebcde88d4123b4a6ba566dbbde06 DEBUG: Destination district uuid: 4956ebcde88d4123b4a6ba566dbbde06 DEBUG: District unchanged keeping uid DEBUG: Getting existing app 'myapp2' status before moving DEBUG: App 'myapp2' was running DEBUG: Getting existing app 'myapp2' status before moving DEBUG: App 'myapp2' was running DEBUG: Getting existing app 'myapp2' status before moving DEBUG: App 'myapp2' was running DEBUG: Stopping existing app cartridge 'jenkins-client-1.4' before moving DEBUG: Performing cartridge level pre-move for embedded jenkins-client-1.4 for 'myapp2' on ip-10-98-86-232 DEBUG: Stopping existing app cartridge 'haproxy-1.4' before moving DEBUG: Stopping existing app cartridge 'perl-5.10' before moving DEBUG: Creating new account for gear 'myapp2' on ip-10-85-3-183 DEBUG: Moving content for app 'myapp2', gear 'myapp2' to ip-10-85-3-183 Identity added: /var/www/stickshift/broker/config/keys/rsync_id_rsa (/var/www/stickshift/broker/config/keys/rsync_id_rsa) Warning: Permanently added '10.85.3.183' (RSA) to the list of known hosts. Agent pid 2462 DEBUG: Performing cartridge level move for 'perl-5.10' on ip-10-85-3-183 DEBUG: Performing cartridge level move for 'haproxy-1.4' on ip-10-85-3-183 DEBUG: Performing cartridge level move for embedded jenkins-client-1.4 for 'myapp2' on ip-10-85-3-183 DEBUG: Moving failed. Rolling back gear 'myapp2' 'myapp2' with remove-httpd-proxy on 'ip-10-85-3-183' DEBUG: Moving failed. Rolling back gear 'myapp2' in 'myapp2' with destroy on 'ip-10-85-3-183' /usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:1627:in `run_cartridge_command': Node execution failure (invalid exit code from node). If the problem persists please contact Red Hat support. (StickShift::NodeException) from /usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:366:in `expose_port' from /usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:613:in `move_gear' from /usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:585:in `each' from /usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:585:in `move_gear' from /usr/bin/rhc-admin-move:129 Expected results: Should move haproxy gear with jenkins client embeded Additional info:
Created attachment 586309 [details] broker log
Created attachment 586314 [details] taget node mcollective log
Fixed with rev#bbb24229ff6962a89f9dbb0566d91a05b1d19afd in li repo
Retest this bug with devenv_1806, and failed. Broker log: DEBUG: Performing cartridge level move for embedded jenkins-client-1.4 for 'pytest' on ip-10-99-38-17 DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7fba312f65f0> DEBUG: rpc_client.custom_request('cartridge_do', {:action=>"move", :cartridge=>"embedded/jenkins-client-1.4", :args=>"'pytest' 'jialiu' '2967a176b67140ecbad7bd9b328010de'"}, @id, {'identity' => @id}) DEBUG: [#<MCollective::RPC::Result:0x7fba3129bdd0 @results={:data=>{:output=>"", :exitcode=>0}, :statusmsg=>"OK", :statuscode=>0, :sender=>"ip-10-99-38-17"}, @action="cartridge_do", @agent="libra">] DEBUG: Cartridge command embedded/jenkins-client-1.4::move exitcode = 0 DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7fba31299378> DEBUG: rpc_client.custom_request('cartridge_do', {:action=>"expose-port", :cartridge=>"jenkins-client-1.4", :args=>"'pytest' 'jialiu' '2967a176b67140ecbad7bd9b328010de'"}, @id, {'identity' => @id}) DEBUG: [#<MCollective::RPC::Result:0x7fba31232f38 @results={:data=>{:exitcode=>127}, :statusmsg=>"cartridge_do_action ERROR action 'expose-port' not found.", :statuscode=>1, :sender=>"ip-10-99-38-17"}, @action="cartridge_do", @agent="libra">] DEBUG: Cartridge command jenkins-client-1.4::expose-port exitcode = 127 DEBUG: Performing cartridge level move for 'haproxy-1.4' on ip-10-99-38-17 DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7fba3122ef78> DEBUG: rpc_client.custom_request('cartridge_do', {:action=>"move", :cartridge=>"haproxy-1.4", :args=>"'pytest' 'jialiu' '2967a176b67140ecbad7bd9b328010de'"}, @id, {'identity' => @id}) DEBUG: [#<MCollective::RPC::Result:0x7fba311a5818 @results={:data=>{:output=>"pytest. Cartridge type '' not set.\nchcon: missing operand\nTry `chcon --help' for more information.\n", :exitcode=>1}, :statusmsg=>"cartridge_do_action failed 1. Output pytest. Cartridge type '' not set.\nchcon: missing operand\nTry `chcon --help' for more information.\n", :statuscode=>1, :sender=>"ip-10-99-38-17"}, @action="cartridge_do", @agent="libra">] DEBUG: server results: pytest. Cartridge type '' not set. DEBUG: server results: chcon: missing operand DEBUG: server results: Try `chcon --help' for more information. DEBUG: Cartridge command haproxy-1.4::move exitcode = 1 Looks like "expose-port" action for jenkins-client-1.4 does not break move operation now, but move action for haproxy-1.4 failed, this issue is introduced by the fix for #824423, so keep this bug in "ON_QA" status, once 824423 is fixed, will re-test this bug.
verified on devenv_1808