Bug 1150140 - oo-admin-move does not filter nodes by region
Summary: oo-admin-move does not filter nodes by region
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Online
Classification: Red Hat
Component: Pod
Version: 1.x
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Abhishek Gupta
QA Contact: libra bugs
URL:
Whiteboard:
Depends On: 1155478
Blocks: 1155290
TreeView+ depends on / blocked
 
Reported: 2014-10-07 13:45 UTC by Andy Grimm
Modified: 2016-11-08 03:47 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1155290 (view as bug list)
Environment:
Last Closed: 2015-02-18 16:51:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Andy Grimm 2014-10-07 13:45:23 UTC
Description of problem:

oo-admin-move forbids region changes by default, but the placement algorithm doesn't filter by region, so it's possible that a move will fail because a node in a different region was chosen at random.

Version-Release number of selected component (if applicable):
rubygem-openshift-origin-msg-broker-mcollective-1.29.3-1.el6oso.noarch
openshift-origin-broker-util-1.29.3-1.el6oso.noarch

How reproducible:
The degree of reproducibility is dependent on which regions the nodes with the lowest active usage percentage for a given gear profile belong to.

Steps to Reproduce:
1. oo-admin-move --gear_uuid <UUID> -p medium --change_district

Actual results:

Login: agrimm
App UUID: <UUID>
Gear UUID: <UUID>
DEBUG: Destination container: euw-med-node2.prod.rhcloud.com
DEBUG: Source district uuid: 532cf5185973ca2c17000001
DEBUG: Destination district uuid: 5416f5eae0b8cd9892000001
Error moving gear. Old and new servers must belong to the same region, source region: aws-us-east-1 destination region: aws-eu-west-1

Expected results:
move should succeed if there is available capacity in the source region

Comment 1 Abhishek Gupta 2014-10-20 21:55:04 UTC
Can I get the UUID  of the gear that was being moved?

Comment 2 Andy Grimm 2014-10-21 00:46:45 UTC
We attempted this with multiple gears; I know one of them was 53e4f1ab500446680a000440, which was just a test app of mine and has been deleted.  I think we could easily reproduce this in stg if you'd like.

Comment 3 Andy Grimm 2014-10-21 18:24:59 UTC
Here, opts should contain region_id (in plugins/msg-broker/mcollective/lib/openshift/mcollective_application_container_proxy.rb#OpenShift::MCollectiveApplicationContainerProxy::resolve_destination):

        opts = { :node_profile => destination_gear_size, :district_uuid => destination_district_uuid,
                 :gear => gear, :gear_exists_in_district => gear_exists_in_district, :required_uid => required_uid }

        # the ApplicationContainerProxy method is used so that the node selector plugin can be invoked
        destination_container = ApplicationContainerProxy.find_available(opts)

Comment 4 openshift-github-bot 2014-10-22 00:05:53 UTC
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/ed0cbe431e91bd093dae4923ee4a8c64221fba49
Bug 1150140: Region was not being correctly handled during gear moves

Comment 5 Abhishek Gupta 2014-10-22 00:07:11 UTC
Fixed with --> https://github.com/openshift/origin-server/pull/5894

Comment 6 Meng Bo 2014-10-23 07:32:45 UTC
Checked on devenv-stage_1074, with steps as below:

1. Setup multi-node env with 3 nodes placed in 3 separated districts
2. And dist1 and dist2 to region1, and dist2 to region2
3. Create app on region1
4. Move app with --change_district option

After 10 times move, all the operations succeeded with moving gear to the district in same region.

It will not try to move gear to different region.

Verify the bug.


Note You need to log in before you can comment on or make changes to this bug.