Bug 833384 - [OSS] app is not deployed with load balance manner on multi-node open-source PaaS.
[OSS] app is not deployed with load balance manner on multi-node open-source ...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Origin
Classification: Red Hat
Component: Pod (Show other bugs)
2.x
Unspecified Unspecified
medium Severity low
: ---
: ---
Assigned To: Abhishek Gupta
libra bugs
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-06-19 07:18 EDT by Johnny Liu
Modified: 2015-05-14 21:58 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-06-25 14:26:55 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Johnny Liu 2012-06-19 07:18:58 EDT
Description of problem:
In my own multi-node open-source PaaS, app is always deployed to broker node.

Broker log:
<--snip-->
Creating application phptest3
Creating gears
DEBUG: rpc_get_fact: fact=active_capacity
DEBUG: rpc_exec: rpc_client=#<MCollective::RPC::Client:0x7fa9b1a18eb8>
Current server: broker.example.com active capacity: 0
CURRENT SERVER: broker.example.com
DEBUG: find_available_impl: current_server: broker.example.com: 0
DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7fa9b1cbffb8>
<--snip-->

Call mcollective facts to get active_capacity.
[openshift@broker ~]$ sudo mco facts active_capacity -v
Determining the amount of hosts matching filter for 2 seconds .... 2
Report for fact: active_capacity

        0.0                                     found 1 times

            node0.example.com

        0.2                                     found 1 times

            broker.example.com

---- rpc stats ----
           Nodes: 2 / 2
     Pass / Fail: 0 / 0
      Start Time: Tue Jun 19 15:04:20 -0400 2012
  Discovery Time: 2042.37ms
      Agent Time: 173.53ms
      Total Time: 2215.90ms


It looks like broker convert float value to int value, that is why active_capacity is always 0 seen from broker log.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Following http://www.krishnaraman.net/multi-node-openshift-origin-from-scratch/ to setup your own multi-node open-source PaaS.
2. Create 1st app, this app is deployed onto broker node.
3. Create 2nd app, this app is still deployed onto broker node.
3. Create 3rd app, this app is still deployed onto broker node.
  
Actual results:
app is always deployed to broker node.

Expected results:
app should be deploayed to node accroding to active_capacity facts.

Additional info:
Comment 1 Johnny Liu 2012-06-19 07:49:03 EDT
In
Comment 2 Peter Ruan 2012-06-19 17:42:05 EDT
In my setup, I have 1 ruby app and 1 scalable app and they are distributed evenly among the nodes (ruby app in the broker node and the salable app in the slave node).
Comment 3 Krishna Raman 2012-06-20 11:36:58 EDT
Pushing down the severity of this issue. Balancing between nodes does happen but it currently requires about 10 apps (worst case) before it switches nodes. 
The factor used to do the balancing updates once a minute, so the 10 apps would have to be created spread out over a minute or so.

Abhishek is working on improving this already.
Comment 4 Abhishek Gupta 2012-06-20 14:07:32 EDT
I have already ported Dan's changes to over to the features/mcollective branch on crankcase.
Comment 5 Peter Ruan 2012-06-21 00:31:44 EDT
verfiied with the latest pull


[root@broker ~]# mco facts active_capacity -v
Determining the amount of hosts matching filter for 2 seconds .... 2
Report for fact: active_capacity

        0.2                                     found 1 times

            broker.example.com

        0.4                                     found 1 times

            node2.example.com


---- rpc stats ----
           Nodes: 2 / 2
     Pass / Fail: 0 / 0
      Start Time: Wed Jun 20 21:30:55 -0700 2012
  Discovery Time: 2050.12ms
      Agent Time: 216.26ms
      Total Time: 2266.38ms
Comment 6 Johnny Liu 2012-06-21 01:46:48 EDT
Verification steps in comment 5 is not correct, active_capacity facter always retrun correct value, pls refer to initial report of this bug.

The key point is gearchanger convert the value returned by mcollective to int type.


Check the broker log, take note of active_capacity, and compare with the value in initial report.
<--snip-->
Checking to see if application name is black listed
Creating application myapp
Creating gears
DEBUG: rpc_get_fact: fact=active_capacity
DEBUG: rpc_exec: rpc_client=#<MCollective::RPC::Client:0x7f2bbb221670>
Current server: broker.example.com active capacity: 0.0
CURRENT SERVER: broker.example.com
DEBUG: find_available_impl: current_server: broker.example.com: 0.0
<--snip-->

Now broker did not convert it to int type.

And apps are indeed deployed to multi nodes in load balance manner.

Note You need to log in before you can comment on or make changes to this bug.