Bug 429304 - client uses excessive memory in performance tests
client uses excessive memory in performance tests
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp (Show other bugs)
beta
All Linux
urgent Severity high
: ---
: ---
Assigned To: Gordon Sim
Kim van der Riet
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-01-18 11:53 EST by Alan Conway
Modified: 2012-12-07 12:46 EST (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Alan Conway 2008-01-18 11:53:04 EST
Description of problem:

Heavy-load testing with 16 clients cause memory to run out on an 8G machine.)
Clearly the clients are consuming too much memory.

Probable solution:

Clients have an unbounded queue for outgoing messages/frames. This means that a
client sending a large number of messages with no delay (such as the perftest
client) may enqueue *every* message it has before anything gets sent to the network.

The outgoing queue needs to be size-bounded and block when the total size
buffered frames exceed N bytes of data. (Note it should *not* be count-bounded
since frames can be of variable size, the goal is to limit memory consumption
not frame production) 

The limit N can be hard coded for now, ultimately it should be configurable but
we need a client-side configuration mechanism for that.

Note this is completely independent of AMQP flow control. AMQP flow control is
designed to protect the broker, this is designed to protect the client's address
space.
Comment 1 Gordon Sim 2008-04-30 10:26:27 EDT
I have added some bounds checking to the client. Only application threads will
block (the io thread needs to send back some controls at times and that never
blocks).

The bounds are specified as a multiple of the max-frame-size and defaults to 2.

Previously I could not run perftests with large counts (anything over 200k on my
laptop) without the client running out of memory. Now I can run with counts of
several million messages. 
Comment 2 Mike Bonnet 2008-05-16 11:08:02 EDT
qpidc-0.2.656926-1.el5 and qpidd-0.2.656926-1.el5 have been pushed to the staging repo for testing

Note You need to log in before you can comment on or make changes to this bug.