Description of problem: min/max gear checking is currently done in application code. This creates race conditions for concurrent scaleup/scaledown requests. The check should be done at the mongo level. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. Set min or max gears allowed for a cart 2. Concurrently scale up and scale down an app with min/max gears 3. Actual results: Can have more or less gears than allowed by min/max gear settings Expected results: Should work as expected regardless of concurrency Additional info:
An important caveat is that if there's an error or exception from the save method due to Mongodb; then any steps that had been taken to create a gear must be un-done before the call completes or a half-built gear could be left behind.
fixed on friday, pull requesting coming today
Fixed in git rev 0a45d8f6f86ddfdd. Testing must be done by firing multiple scale-up/scale-down events in parallel and make sure number of application gears is consistent with consumed gears.
Tested this on devenv_1884, this bug should be partly fixed, test with the following steps: 1. Change min/max gears for php to 2 /3 Scaling: Min: 2 Max: 3 2. Create a scaling php app, then it should have already consumed 3 gears (2 web cartridge gears and 1 haproxy gear) 3. Open 3 terminals to scale up at the same time: 2 of them return success, 1 of them return: "Failed to add event scale-up to application scalphpapp1 due to: Cannot scale up beyond maximum gear limit '3' in app scalphpapp1.","field":null}], 4. Check the apps's gear_groups, the max gear is 3. data": [ { "gears": [ { "id": "eb7b4164dccc4f36887c2d3bfd5fcc55", "state": "started" } ], "cartridges": [ { "name": "haproxy-1.4" }, { "name": "php-5.3" } ], "name": "@@app/comp-proxy/cart-php-5.3", "gear_profile": "small" }, { "gears": [ { "id": "8c528a9e5036480ca986989faa505afe", "state": "started" }, { "id": "d6ec9c2ad95d4ca4a8766cc4b9633e97", "state": "started" }, { "id": "4075a090a7f04d18a957702ce28d5290", "state": "started" } ], "cartridges": [ { "name": "php-5.3" } ], "name": "@@app/comp-web/cart-php-5.3", "gear_profile": "small" } ], 5. Check consumed gears: irb(main):004:0> user1.consumed_gears => 5 irb(main):005:0> user1.applications.length => 1 irb(main):006:0> user1.applications[0].group_instances[1].gears.length => 1 irb(main):007:0> user1.applications[0].group_instances[0].gears.length => 3 Then the consumed_gears is larger than the actual gears, the consumed gears should be 4 as well.
Likely fixed in model refactor, please close when it is merged.
Fixed with model refactor.
Verified on devenv_2744 1. Edit /usr/libexec/openshift/cartridges/php-5.3/info/manifest.yml, set scaling: Scaling: Min: 2 Max: 3 2. Restart broker, create a scalable php application 3. Check consumed gears libra_rs:PRIMARY> db.cloud_users.find({"login":"jhou+1"},{consumed_gears:1}) { "_id" : ObjectId("5108d2a310c4fee306000001"), "consumed_gears" : 1 } As we see, the min/max gears checking is done at mongo level 4. Concurrently scale up 3 gears for the app, and check consumed gears libra_rs:PRIMARY> db.cloud_users.find({"login":"jhou+1"},{consumed_gears:1}) { "_id" : ObjectId("5108d2a310c4fee306000001"), "consumed_gears" : 4 } No errors.