Bug 970273

Summary: node websocket port concurrency issues
Product: OpenShift Online Reporter: Mike McGrath <mmcgrath>
Component: ContainersAssignee: Mrunal Patel <mpatel>
Status: CLOSED WORKSFORME QA Contact: libra bugs <libra-bugs>
Severity: low Docs Contact:
Priority: medium    
Version: 1.xCC: abhgupta, dmcphers, jhonce, mmcgrath, tcrawley, wdecoste
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-06-11 21:09:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Mike McGrath 2013-06-03 20:57:58 UTC
Description of problem:
While trying to put some load on a test application, we saw that concurrent connections against port 8000 would pile up and generate an error:

DEBUG: Program node /usr/lib/node_modules/openshift-node-web-proxy/bin/web-proxy.js --config /etc/openshift/web-proxy-config.json exited with code 1

DEBUG: Starting child process with 'node /usr/lib/node_modules/openshift-node-web-proxy/bin/web-proxy.js --config /etc/openshift/web-proxy-config.json'
DEBUG: 

DEBUG: /usr/lib/node_modules/openshift-node-web-proxy/bin/web-proxy.js:26

DEBUG: emitter.setMaxListeners(0);

DEBUG: ^

DEBUG: ReferenceError: emitter is not defined
    at Object.<anonymous> (/usr/lib/node_modules/openshift-node-web-proxy/bin/web-proxy.js:26:1)
    at Module._compile (module.js:446:26)
    at Object..js (module.js:464:10)
    at Module.load (module.js:353:31)
    at Function._load (module.js:311:12)
    at Array.0 (module.js:484:10)
    at EventEmitter._tickCallback (node.js:190:38)



We tried a few things With the load tests against 8000 things started off fine but over time the requests would get slower and slower.  This seems to be an issue similar to Apache's "MaxClients".

When running the same test over port 80, requests worked fine and response times were 1-2 seconds.

Even when responses to port 8000 were slow because of this predicted backlog, port 80 was fine.

Comment 1 Mike McGrath 2013-06-03 21:09:38 UTC
Sorry that error above is inaccurate, this is the actual error:


==> supervisor.log <==
DEBUG: (node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.

DEBUG: Trace: 
    at Socket.<anonymous> (events.js:139:15)
    at ClientRequest.<anonymous> (/usr/lib/node_modules/openshift-node-web-proxy/lib/proxy/ProxyServer.js:254:12)
    at ClientRequest.emit (events.js:88:20)
    at Array.0 (http.js:1346:9)
    at EventEmitter._tickCallback (node.js:190:38)

Comment 2 Dan McPherson 2014-03-12 20:21:08 UTC
Is this still an issue?

Comment 3 Mrunal Patel 2014-03-24 22:47:58 UTC
I tested ab -n 10000 -c 100 on a php app a few times without seeing any crashes.

Comment 4 Mrunal Patel 2014-03-26 16:38:27 UTC
Tested with jboss as well without seeing any crashes.