Bug 1155290 - oo-admin-move does not filter nodes by region
Summary: oo-admin-move does not filter nodes by region
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 2.2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Luke Meyer
QA Contact: libra bugs
URL:
Whiteboard:
Depends On: 1150140
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-10-21 20:08 UTC by Brenton Leanhardt
Modified: 2014-12-10 13:24 UTC (History)
9 users (show)

Fixed In Version: rubygem-openshift-origin-msg-broker-mcollective-1.31.1.1-1
Doc Type: Bug Fix
Doc Text:
Due to a bug in broker gear placement during gear moves, the oo-admin-move command could fail at random by selecting a destination node in a different region than the gear started in. This bug fix updates the oo-admin-move command to filter potential destination nodes correctly by region.
Clone Of: 1150140
Environment:
Last Closed: 2014-12-10 13:24:35 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:1979 0 normal SHIPPED_LIVE Red Hat OpenShift Enterprise 2.2.2 bug fix and enhancement update 2014-12-10 18:23:46 UTC

Description Brenton Leanhardt 2014-10-21 20:08:06 UTC
+++ This bug was initially created as a clone of Bug #1150140 +++

Description of problem:

oo-admin-move forbids region changes by default, but the placement algorithm doesn't filter by region, so it's possible that a move will fail because a node in a different region was chosen at random.

Version-Release number of selected component (if applicable):
rubygem-openshift-origin-msg-broker-mcollective-1.29.3-1.el6oso.noarch
openshift-origin-broker-util-1.29.3-1.el6oso.noarch

How reproducible:
The degree of reproducibility is dependent on which regions the nodes with the lowest active usage percentage for a given gear profile belong to.

Steps to Reproduce:
1. oo-admin-move --gear_uuid <UUID> -p medium --change_district

Actual results:

Login: agrimm
App UUID: <UUID>
Gear UUID: <UUID>
DEBUG: Destination container: euw-med-node2.prod.rhcloud.com
DEBUG: Source district uuid: 532cf5185973ca2c17000001
DEBUG: Destination district uuid: 5416f5eae0b8cd9892000001
Error moving gear. Old and new servers must belong to the same region, source region: aws-us-east-1 destination region: aws-eu-west-1

Expected results:
move should succeed if there is available capacity in the source region

--- Additional comment from Abhishek Gupta on 2014-10-20 17:55:04 EDT ---

Can I get the UUID  of the gear that was being moved?

--- Additional comment from Andy Grimm on 2014-10-20 20:46:45 EDT ---

We attempted this with multiple gears; I know one of them was 53e4f1ab500446680a000440, which was just a test app of mine and has been deleted.  I think we could easily reproduce this in stg if you'd like.

--- Additional comment from Andy Grimm on 2014-10-21 14:24:59 EDT ---

Here, opts should contain region_id (in plugins/msg-broker/mcollective/lib/openshift/mcollective_application_container_proxy.rb#OpenShift::MCollectiveApplicationContainerProxy::resolve_destination):

        opts = { :node_profile => destination_gear_size, :district_uuid => destination_district_uuid,
                 :gear => gear, :gear_exists_in_district => gear_exists_in_district, :required_uid => required_uid }

        # the ApplicationContainerProxy method is used so that the node selector plugin can be invoked
        destination_container = ApplicationContainerProxy.find_available(opts)

Comment 3 Ma xiaoqiang 2014-11-25 06:33:22 UTC
Check on puddle [2.2.z/2014-11-24.3]

1. setup 3 nodes env
node1 dist1 region1
node2 dist2 region2
node3 dist3 region2

2. set ALLOW_REGION_SELECTION to true on broker
# vim /etc/openshift/broker.conf
<--snip-->
# Allow region selection when creating applications. Default is false
ALLOW_REGION_SELECTION="true"
<--snip-->

3. Create two apps in region2(make sure the app is on different nodes)
#rhc app create testapp1 php-5.4 --region region2
#sleep 300
#rhc app create testapp2 php-5.4 --region region2

4. Move testapp1 ten times
for i in {1..10};do oo-admin-move --gear_uuid 54741db782611d7869000001 --change_district ; done

Move successfully.

Comment 5 errata-xmlrpc 2014-12-10 13:24:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2014-1979.html


Note You need to log in before you can comment on or make changes to this bug.