Bug 892124

Summary:

[fork][model refactor]Can't move the scaled-up gear for scalable application

Product:

OKD

Reporter:

Rony Gong 🔥 <qgong>

Component:

Pod

Assignee:

Rajat Chopra <rchopra>

Status:

CLOSED CURRENTRELEASE

QA Contact:

libra bugs <libra-bugs>

Severity:

medium

Docs Contact:

Priority:

unspecified

Version:

2.x

CC:

dmcphers, rchopra

Target Milestone:

---

Target Release:

---

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2013-02-13 23:16:48 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
development.log	none

Description Rony Gong 🔥 2013-01-05 10:35:24 UTC

Created attachment 672820 [details]
development.log

Description of problem:
For scalable-app, scale-up this app by REST API, then move the scaled-up gear
Failed.
And after this move, there always show error message in development.log about can't find Document of ...

Version-Release number of selected component (if applicable):
fork_ami_refctr1_381

How reproducible:
always


Steps to Reproduce:
1.Setup multi node env.
2.Create scalable app then scale up by RESTAPI, disable auto-scaling
3.Move the scaled-up gear within district.

  
Actual results:
[root@ip-10-38-21-76 ~]# oo-admin-move --gear_uuid 50e7f0c0776856a80800053c -i ip-10-38-21-76
URL: http://qs2php-qgong8.dev.rhcloud.com
Login: qgong
App UUID: 50e7ef79776856a8080004f6
Gear UUID: 50e7f0c0776856a80800053c
DEBUG: Source district uuid: 50e79b17776856de32000001
DEBUG: Destination district uuid: 50e79b17776856de32000001
DEBUG: District unchanged keeping uid
DEBUG: Getting existing app 'qs2php' status before moving
DEBUG: Gear component 'php-5.3' was running
DEBUG: Stopping existing app cartridge 'haproxy-1.4' before moving
DEBUG: Error performing stop on existing app on try 1: Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.
DEBUG: Error performing stop on existing app on try 2: Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:1131:in `parse_result': Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support. (OpenShift::NodeException)
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:1204:in `run_cartridge_command'
	from /var/www/openshift/broker/lib/express/broker/mcollective_ext.rb:12:in `run_cartridge_command'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:1063:in `stop_component'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:378:in `stop'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:648:in `block (2 levels) in move_gear_pre'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:1014:in `block in do_with_retry'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:1012:in `each'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:1012:in `do_with_retry'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:647:in `block in move_gear_pre'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:640:in `each'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:640:in `move_gear_pre'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:701:in `move_gear'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:670:in `block in move_gear_secure'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.3.2/app/models/application.rb:947:in `run_in_application_lock'
	from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-msg-broker-mcollective-1.3.1/lib/openshift/mcollective_application_container_proxy.rb:669:in `move_gear_secure'
	from /usr/sbin/oo-admin-move:110:in `<main>'

Expected results:
Could move this gear successfully.

Additional info:

Comment 1 Rajat Chopra 2013-01-08 20:43:19 UTC

Fixed with rev#658b4ccd220d02d074b5e1d087b552a11940f3d9

Comment 2 Rony Gong 🔥 2013-01-10 08:12:41 UTC

Verified on fork_ami_refctr1_395
1.Create scalable app, then scale-up
2.Move this scaled-up gear
[root@ip-10-212-70-181 openshift]# oo-admin-move --gear_uuid 50ee76058d51f0f70b000049 -i ip-10-212-70-181
URL: http://qshp-qgong8.dev.rhcloud.com
Login: qgong
App UUID: 50ee75378d51f0f70b000023
Gear UUID: 50ee76058d51f0f70b000049
DEBUG: Source district uuid: 50ee70128d51f0cb0b000001
DEBUG: Destination district uuid: 50ee70128d51f0cb0b000001
DEBUG: District unchanged keeping uid
DEBUG: Getting existing app 'qshp' status before moving
DEBUG: Gear component 'php-5.3' was running
DEBUG: Stopping existing app cartridge 'php-5.3' before moving
DEBUG: Force stopping existing app cartridge 'php-5.3' before moving
DEBUG: Creating new account for gear '50ee76058d' on ip-10-212-70-181
DEBUG: Moving content for app 'qshp', gear '50ee76058d' to ip-10-212-70-181
Identity added: /var/www/openshift/broker/config/keys/rsync_id_rsa (/var/www/openshift/broker/config/keys/rsync_id_rsa)
Agent pid 20033
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 20033 killed;
DEBUG: Moving system components for app 'qshp', gear '50ee76058d' to ip-10-212-70-181
Identity added: /var/www/openshift/broker/config/keys/rsync_id_rsa (/var/www/openshift/broker/config/keys/rsync_id_rsa)
Agent pid 20095
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 20095 killed;
DEBUG: Restarting httpd proxy for 'php-5.3' on ip-10-212-70-181
DEBUG: Starting cartridge 'php-5.3' in 'qshp' after move on ip-10-212-70-181
DEBUG: Fixing DNS and mongo for gear '50ee76058d' after move
DEBUG: Changing server identity of '50ee76058d' from 'ip-10-64-62-60' to 'ip-10-212-70-181'
DEBUG: Deconfiguring old app 'qshp' on ip-10-64-62-60 after move
Successfully moved 'qshp' with gear uuid '50ee76058d51f0f70b000049' from 'ip-10-64-62-60' to 'ip-10-212-70-181'