Bug 847150 - httpd_singular stacks up and times out C9 app creation requests
httpd_singular stacks up and times out C9 app creation requests
Status: CLOSED CURRENTRELEASE
Product: OpenShift Origin
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Rob Millner
libra bugs
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-09 17:17 EDT by Rob Millner
Modified: 2015-05-14 18:57 EDT (History)
2 users (show)

See Also:
Fixed In Version: devenv_1971
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-09-17 17:30:06 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Rob Millner 2012-08-09 17:17:14 EDT
Description of problem:
Operations creates 1000 nodes at a time for C9.  This stacks up a huge number of requests on the httpd_singular lock and causes timeouts.

Version-Release number of selected component (if applicable):


How reproducible:
Whenever creating huge numbers of nodes at a time.


Steps to Reproduce:
1. Configure for C9
2. Create a thousand apps at a time
3.
  
Actual results:
Fail

Expected results:
It works.

Additional info:

Here's an proposal for httpd_singular to stack up a bunch of graceful requests and only do one.  

This should cause requests to wait at most two iterations for a graceful and create a sharp drop-off on the number of simultaneous graceful requests being processed.

1. Register a request for graceful.
   1.1 atomic unique tmpfile in /dev/shm, empty

2. Grab the singular lock

3. Scan the set of requests and make a list (see #1)
  3.1 A request is an empty request file.
  3.2 If there are requests, then run chk and graceful
  3.3 Copy the results into the request file

4. Read and remove my own request file and return the results

5. Unlock the singular lock
Comment 1 Rob Millner 2012-08-09 19:33:24 EDT
Crankcase pull request #356.
Comment 2 Rob Millner 2012-08-10 14:43:40 EDT
Pull request accepted.
Comment 3 Jianwei Hou 2012-08-13 22:42:17 EDT
Verified on devenv_1975

Steps:
1.Launch an instance and configure for C9
2.Create a 1000 apps at time

Results:
No timeout occurs, the apps were created fast.

Note You need to log in before you can comment on or make changes to this bug.