Bug 867349 - Got error when add cartridges to scalable app after setting its min gears number bigger than its real gear size .
Got error when add cartridges to scalable app after setting its min gears num...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Origin
Classification: Red Hat
Component: Website (Show other bugs)
2.x
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Rajat Chopra
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-17 07:08 EDT by Mengjiao Gao
Modified: 2015-05-14 21:15 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-11-06 13:50:31 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
error message when add mysql. (193.28 KB, image/png)
2012-10-17 07:08 EDT, Mengjiao Gao
no flags Details
development.log when add mysql at step 6. (13.37 KB, application/octet-stream)
2012-10-17 07:10 EDT, Mengjiao Gao
no flags Details

  None (edit)
Description Mengjiao Gao 2012-10-17 07:08:34 EDT
Created attachment 628687 [details]
error message when add mysql.

Description of problem:
When I tested US2283 on devenv_2340, I found that after I set the min gear size number bigger than the real app gear size for twice, I would get error when I tried to add cartridges to the app. The error page was attached. The development.log when add cartridge was also attached.

Version-Release number of selected component (if applicable):
devenv_2340

How reproducible:
always

Steps to Reproduce:
1.Create a scalable app:
  $rhc app create -s -a spython22 -t python-2.6 -l e
2.Go to the app detail page, and clicked the "scaled up with HAProxy" link.
3.Choose 8 as the min gear size.
4.Add cartridge mongodb.
5.Choose 12 as the min gear size.
6.Add cartridge mysql.
  
Actual results:
After step 4, the mongdb can be embed successfully and the gear size of the app would be scaled up to 9.
After step 6. I got error message :"Consistency check failed. Could not update application 'spython252' for 'e'", and cannot added mysql successfully.And the gear size of the app was still 9.

Expected results:
After step 6, the mysql should be embed successfully, and the gear size of the app should be scaled up to 13.

Additional info:
I have tried to use the REST API as follows to scale up the app after step 6, and it would get error messages such as "Consistency check failed." or "Unable to create gear on node".

The REST API to scale up:
curl -k -X POST -H 'Accept: application/xml' -d event=scale-up --user e:e https://ec2-54-242-72-151.compute-1.amazonaws.com/broker/rest/domains/domainnnn/applications/spython252/events
Comment 1 Mengjiao Gao 2012-10-17 07:10:00 EDT
Created attachment 628688 [details]
development.log when add mysql at step 6.
Comment 2 Rajat Chopra 2012-10-17 20:23:28 EDT
Cannot reproduce this problem. I tried this several times over with each line on the debugger too - no issues.

Now several theoretical issues are possible which can result in that problem and one of them indeed caused the problem that you saw. I took the liberty of starting over your ec2 instance and debugging on it. From the given username/appname I see that the mongo datastore for user 'e' is inconsistent - the consumed_gears is 15 and the current total taken by the app is 6. So here is what went wrong -

1. Set the scales_from to 10
(at this point the consumed_gear count was 15 and max_gears is 16)
2. add mysql to the app
(mysql got a good gear and the consumed_gear count went up to 16)
3. but since scales_from was increased, configuring the dependencies also resulted in scaling up the python gears...
(as soon as the first gear was created, the mongo save failed because consumed_gears == max_gears)... ERROR

Now even if your consumed_gear vs actual gear count were consistent you could have hit this issue - as soon as the max_gears count is hit the said error is raised.

I do not see this bug as a functional issue - the app was still restored alright in correct state. The messaging was wrong - instead of saying 'Consistency check failed', the message should be 'you have reached the max_gears for this account'.

I will put in a fix for correcting the message after I get around some other issues I have discovered on the way.
Comment 3 Rajat Chopra 2012-10-19 13:36:53 EDT
Fixed with rev#cb8dbc6bbc2014f319239724d035470ffa30ae13 in origin-server/master
Comment 4 Yujie Zhang 2012-10-22 03:45:52 EDT
(In reply to comment #3)
Tested this issue on devev_2360, it has been fixed now, mongodb and mysql can be added successfully, thanks.

Note You need to log in before you can comment on or make changes to this bug.