Bug 855307

Summary: trigger multi scale-up at same time will cause the gear unmatched between node and mongodb
Product: OKD Reporter: Meng Bo <bmeng>
Component: PodAssignee: Dan McPherson <dmcphers>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: mgao, ramr
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-13 23:49:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 878205    
Attachments:
Description Flags
broker_log none

Description Meng Bo 2012-09-07 10:18:59 UTC 
Created attachment 610682 [details]
broker_log

Description of problem:
trigger 3 scale-up events at same time, check the gear number for the app, check the gear directory on instance under /var/lib/stickshift. The gears number does not match.

Version-Release number of selected component (if applicable):
devenv_2128

How reproducible:
always

Steps to Reproduce:
1.Create scalable app and disable auto-scaling
2.Trigger 3 scale-up events at same time
for i in `seq 1 3`; do curl -k -H 'Accept: application/xml' --user bmeng:123123 https://localhost/broker/rest/domains/bmengdev/applications/php1s/events -XPOST -d event=scale-up & done
3.Run rhc-admin-chk
  
Actual results:
Some gears exist on node but does not exist in mongodb. And the gear number is incorrect for the scalable app.

Expected results:
Should not return any issue when running rhc-admin-chk, and the gear number should be 5 for the scalable app. (2+3)

Additional info:
Attached the development log for debug.
Comment 1 Dan McPherson 2012-09-07 18:27:10 UTC 
Will not be fixed until the model refactor is in place.  Marking FutureFeature.
Comment 2 Dan McPherson 2012-10-17 18:59:53 UTC 
*** Bug 841527 has been marked as a duplicate of this bug. ***
Comment 3 Dan McPherson 2012-10-31 13:57:00 UTC 
*** Bug 871660 has been marked as a duplicate of this bug. ***
Comment 4 Dan McPherson 2013-01-26 01:12:40 UTC 
Marking ON_QA with delivery of model refactor.
Comment 5 Meng Bo 2013-01-28 08:00:44 UTC 
Checked on devenv_2735, triggered scale-up 5 times at the same time. Some of the operations failed with meaningful error message, all the succeed ones has the same record between mongodb and node. No error info found in broker log. oo-admin-chk can pass after scale-up finished.

for i in `seq 1 5`; do curl -k -H 'Accept: application/xml' --user bmeng:123123 https://localhost/broker/rest/domains/bmengdev/applications/sapp1/events -XPOST -d event=scale-up & done

    <message>
      <severity>error</severity>
      <text>Application is currently busy performing another operation. Please try again in a minute.</text>
      <exit-code>171</exit-code>
      <field nil="true"></field>
    </message>