Bug 1155290

Summary:	oo-admin-move does not filter nodes by region
Product:	OpenShift Container Platform	Reporter:	Brenton Leanhardt <bleanhar>
Component:	Node	Assignee:	Luke Meyer <lmeyer>
Status:	CLOSED ERRATA	QA Contact:	libra bugs <libra-bugs>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	2.2.0	CC:	abhgupta, adellape, agrimm, jokerman, libra-bugs, libra-onpremise-devel, mmccomas, pruan, xiama
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	rubygem-openshift-origin-msg-broker-mcollective-1.31.1.1-1	Doc Type:	Bug Fix
Doc Text:	Due to a bug in broker gear placement during gear moves, the oo-admin-move command could fail at random by selecting a destination node in a different region than the gear started in. This bug fix updates the oo-admin-move command to filter potential destination nodes correctly by region.	Story Points:	---
Clone Of:	1150140	Environment:
Last Closed:	2014-12-10 13:24:35 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1150140
Bug Blocks:

Description Brenton Leanhardt 2014-10-21 20:08:06 UTC

+++ This bug was initially created as a clone of Bug #1150140 +++

Description of problem:

oo-admin-move forbids region changes by default, but the placement algorithm doesn't filter by region, so it's possible that a move will fail because a node in a different region was chosen at random.

Version-Release number of selected component (if applicable):
rubygem-openshift-origin-msg-broker-mcollective-1.29.3-1.el6oso.noarch
openshift-origin-broker-util-1.29.3-1.el6oso.noarch

How reproducible:
The degree of reproducibility is dependent on which regions the nodes with the lowest active usage percentage for a given gear profile belong to.

Steps to Reproduce:
1. oo-admin-move --gear_uuid <UUID> -p medium --change_district

Actual results:

Login: agrimm
App UUID: <UUID>
Gear UUID: <UUID>
DEBUG: Destination container: euw-med-node2.prod.rhcloud.com
DEBUG: Source district uuid: 532cf5185973ca2c17000001
DEBUG: Destination district uuid: 5416f5eae0b8cd9892000001
Error moving gear. Old and new servers must belong to the same region, source region: aws-us-east-1 destination region: aws-eu-west-1

Expected results:
move should succeed if there is available capacity in the source region

--- Additional comment from Abhishek Gupta on 2014-10-20 17:55:04 EDT ---

Can I get the UUID  of the gear that was being moved?

--- Additional comment from Andy Grimm on 2014-10-20 20:46:45 EDT ---

We attempted this with multiple gears; I know one of them was 53e4f1ab500446680a000440, which was just a test app of mine and has been deleted.  I think we could easily reproduce this in stg if you'd like.

--- Additional comment from Andy Grimm on 2014-10-21 14:24:59 EDT ---

Here, opts should contain region_id (in plugins/msg-broker/mcollective/lib/openshift/mcollective_application_container_proxy.rb#OpenShift::MCollectiveApplicationContainerProxy::resolve_destination):

        opts = { :node_profile => destination_gear_size, :district_uuid => destination_district_uuid,
                 :gear => gear, :gear_exists_in_district => gear_exists_in_district, :required_uid => required_uid }

        # the ApplicationContainerProxy method is used so that the node selector plugin can be invoked
        destination_container = ApplicationContainerProxy.find_available(opts)

Comment 3 Ma xiaoqiang 2014-11-25 06:33:22 UTC

Check on puddle [2.2.z/2014-11-24.3]

1. setup 3 nodes env
node1 dist1 region1
node2 dist2 region2
node3 dist3 region2

2. set ALLOW_REGION_SELECTION to true on broker
# vim /etc/openshift/broker.conf
<--snip-->
# Allow region selection when creating applications. Default is false
ALLOW_REGION_SELECTION="true"
<--snip-->

3. Create two apps in region2(make sure the app is on different nodes)
#rhc app create testapp1 php-5.4 --region region2
#sleep 300
#rhc app create testapp2 php-5.4 --region region2

4. Move testapp1 ten times
for i in {1..10};do oo-admin-move --gear_uuid 54741db782611d7869000001 --change_district ; done

Move successfully.

Comment 5 errata-xmlrpc 2014-12-10 13:24:35 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2014-1979.html