Bug 824375 - Fail to move haproxy gear with jenkins client embedded
Fail to move haproxy gear with jenkins client embedded
Status: CLOSED CURRENTRELEASE
Product: OpenShift Origin
Classification: Red Hat
Component: Kubernetes (Show other bugs)
2.x
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Rajat Chopra
libra bugs
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-23 07:21 EDT by Johnny Liu
Modified: 2015-05-14 21:54 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-06-08 13:59:32 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
broker log (15.43 KB, application/octet-stream)
2012-05-23 07:22 EDT, Johnny Liu
no flags Details
taget node mcollective log (815 bytes, application/octet-stream)
2012-05-23 07:22 EDT, Johnny Liu
no flags Details

  None (edit)
Description Johnny Liu 2012-05-23 07:21:37 EDT
Description of problem:


Version-Release number of selected component (if applicable):
devenv_1786

How reproducible:
Always

Steps to Reproduce:
1. Create a scalable app with jenkins embeded.
2. Log into broker, move haproxy gears
3.
  
Actual results:
Fail to move haproxy gear.

# rhc-admin-move --gear_uuid 875500b7aa754887b804c52d688ceb5e -i ip-10-85-3-183
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/cloud_user.rb:8: warning: already initialized constant DEFAULT_SSH_KEY_NAME
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/validators/key_validator.rb:2: warning: already initialized constant KEY_NAME_MAX_LENGTH
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/validators/key_validator.rb:3: warning: already initialized constant KEY_NAME_MIN_LENGTH
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/validators/namespace_validator.rb:2: warning: already initialized constant NAMESPACE_MAX_LENGTH
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.11.3/lib/stickshift-controller/app/models/validators/namespace_validator.rb:3: warning: already initialized constant NAMESPACE_MIN_LENGTH
URL: http://myapp2-jialiu.dev.rhcloud.com
Login: jialiu@redhat.com
App UUID: 875500b7aa754887b804c52d688ceb5e
Gear UUID: 875500b7aa754887b804c52d688ceb5e
DEBUG: Source district uuid: 4956ebcde88d4123b4a6ba566dbbde06
DEBUG: Destination district uuid: 4956ebcde88d4123b4a6ba566dbbde06
DEBUG: District unchanged keeping uid
DEBUG: Getting existing app 'myapp2' status before moving
DEBUG: App 'myapp2' was running
DEBUG: Getting existing app 'myapp2' status before moving
DEBUG: App 'myapp2' was running
DEBUG: Getting existing app 'myapp2' status before moving
DEBUG: App 'myapp2' was running
DEBUG: Stopping existing app cartridge 'jenkins-client-1.4' before moving
DEBUG: Performing cartridge level pre-move for embedded jenkins-client-1.4 for 'myapp2' on ip-10-98-86-232
DEBUG: Stopping existing app cartridge 'haproxy-1.4' before moving
DEBUG: Stopping existing app cartridge 'perl-5.10' before moving
DEBUG: Creating new account for gear 'myapp2' on ip-10-85-3-183
DEBUG: Moving content for app 'myapp2', gear 'myapp2' to ip-10-85-3-183
Identity added: /var/www/stickshift/broker/config/keys/rsync_id_rsa (/var/www/stickshift/broker/config/keys/rsync_id_rsa)
Warning: Permanently added '10.85.3.183' (RSA) to the list of known hosts.
Agent pid 2462
DEBUG: Performing cartridge level move for 'perl-5.10' on ip-10-85-3-183
DEBUG: Performing cartridge level move for 'haproxy-1.4' on ip-10-85-3-183
DEBUG: Performing cartridge level move for embedded jenkins-client-1.4 for 'myapp2' on ip-10-85-3-183
DEBUG: Moving failed.  Rolling back gear 'myapp2' 'myapp2' with remove-httpd-proxy on 'ip-10-85-3-183'
DEBUG: Moving failed.  Rolling back gear 'myapp2' in 'myapp2' with destroy on 'ip-10-85-3-183'
/usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:1627:in `run_cartridge_command': Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support. (StickShift::NodeException)
	from /usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:366:in `expose_port'
	from /usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:613:in `move_gear'
	from /usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:585:in `each'
	from /usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.10.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:585:in `move_gear'
	from /usr/bin/rhc-admin-move:129


Expected results:
Should move haproxy gear with jenkins client embeded

Additional info:
Comment 1 Johnny Liu 2012-05-23 07:22:15 EDT
Created attachment 586309 [details]
broker log
Comment 2 Johnny Liu 2012-05-23 07:22:54 EDT
Created attachment 586314 [details]
taget node mcollective log
Comment 3 Rajat Chopra 2012-05-25 14:13:49 EDT
Fixed with rev#bbb24229ff6962a89f9dbb0566d91a05b1d19afd in li repo
Comment 4 Johnny Liu 2012-05-28 03:25:59 EDT
Retest this bug with devenv_1806, and failed.


Broker log:

DEBUG: Performing cartridge level move for embedded jenkins-client-1.4 for 'pytest' on ip-10-99-38-17
DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7fba312f65f0>
DEBUG: rpc_client.custom_request('cartridge_do', {:action=>"move", :cartridge=>"embedded/jenkins-client-1.4", :args=>"'pytest' 'jialiu' '2967a176b67140ecbad7bd9b328010de'"}, @id, {'identity' => @id})
DEBUG: [#<MCollective::RPC::Result:0x7fba3129bdd0 @results={:data=>{:output=>"", :exitcode=>0}, :statusmsg=>"OK", :statuscode=>0, :sender=>"ip-10-99-38-17"}, @action="cartridge_do", @agent="libra">]
DEBUG: Cartridge command embedded/jenkins-client-1.4::move exitcode = 0
DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7fba31299378>
DEBUG: rpc_client.custom_request('cartridge_do', {:action=>"expose-port", :cartridge=>"jenkins-client-1.4", :args=>"'pytest' 'jialiu' '2967a176b67140ecbad7bd9b328010de'"}, @id, {'identity' => @id})
DEBUG: [#<MCollective::RPC::Result:0x7fba31232f38 @results={:data=>{:exitcode=>127}, :statusmsg=>"cartridge_do_action ERROR action 'expose-port' not found.", :statuscode=>1, :sender=>"ip-10-99-38-17"}, @action="cartridge_do", @agent="libra">]
DEBUG: Cartridge command jenkins-client-1.4::expose-port exitcode = 127
DEBUG: Performing cartridge level move for 'haproxy-1.4' on ip-10-99-38-17
DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7fba3122ef78>
DEBUG: rpc_client.custom_request('cartridge_do', {:action=>"move", :cartridge=>"haproxy-1.4", :args=>"'pytest' 'jialiu' '2967a176b67140ecbad7bd9b328010de'"}, @id, {'identity' => @id})
DEBUG: [#<MCollective::RPC::Result:0x7fba311a5818 @results={:data=>{:output=>"pytest. Cartridge type '' not set.\nchcon: missing operand\nTry `chcon --help' for more information.\n", :exitcode=>1}, :statusmsg=>"cartridge_do_action failed 1.  Output pytest. Cartridge type '' not set.\nchcon: missing operand\nTry `chcon --help' for more information.\n", :statuscode=>1, :sender=>"ip-10-99-38-17"}, @action="cartridge_do", @agent="libra">]
DEBUG: server results: pytest. Cartridge type '' not set.
DEBUG: server results: chcon: missing operand
DEBUG: server results: Try `chcon --help' for more information.
DEBUG: Cartridge command haproxy-1.4::move exitcode = 1


Looks like "expose-port" action for jenkins-client-1.4 does not break move operation now, but move action for haproxy-1.4 failed, this issue is introduced by the fix for #824423, so keep this bug in "ON_QA" status, once 824423 is fixed, will re-test this bug.
Comment 5 Rony Gong 2012-05-30 02:33:56 EDT
verified on devenv_1808

Note You need to log in before you can comment on or make changes to this bug.