Red Hat Bugzilla – Bug 429304
client uses excessive memory in performance tests
Last modified: 2012-12-07 12:46:04 EST
Description of problem:
Heavy-load testing with 16 clients cause memory to run out on an 8G machine.)
Clearly the clients are consuming too much memory.
Clients have an unbounded queue for outgoing messages/frames. This means that a
client sending a large number of messages with no delay (such as the perftest
client) may enqueue *every* message it has before anything gets sent to the network.
The outgoing queue needs to be size-bounded and block when the total size
buffered frames exceed N bytes of data. (Note it should *not* be count-bounded
since frames can be of variable size, the goal is to limit memory consumption
not frame production)
The limit N can be hard coded for now, ultimately it should be configurable but
we need a client-side configuration mechanism for that.
Note this is completely independent of AMQP flow control. AMQP flow control is
designed to protect the broker, this is designed to protect the client's address
I have added some bounds checking to the client. Only application threads will
block (the io thread needs to send back some controls at times and that never
The bounds are specified as a multiple of the max-frame-size and defaults to 2.
Previously I could not run perftests with large counts (anything over 200k on my
laptop) without the client running out of memory. Now I can run with counts of
several million messages.
qpidc-0.2.656926-1.el5 and qpidd-0.2.656926-1.el5 have been pushed to the staging repo for testing