Bug 429304 - client uses excessive memory in performance tests
Summary: client uses excessive memory in performance tests
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp   
(Show other bugs)
Version: beta
Hardware: All Linux
urgent
high
Target Milestone: ---
: ---
Assignee: Gordon Sim
QA Contact: Kim van der Riet
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-01-18 16:53 UTC by Alan Conway
Modified: 2012-12-07 17:46 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

Description Alan Conway 2008-01-18 16:53:04 UTC
Description of problem:

Heavy-load testing with 16 clients cause memory to run out on an 8G machine.)
Clearly the clients are consuming too much memory.

Probable solution:

Clients have an unbounded queue for outgoing messages/frames. This means that a
client sending a large number of messages with no delay (such as the perftest
client) may enqueue *every* message it has before anything gets sent to the network.

The outgoing queue needs to be size-bounded and block when the total size
buffered frames exceed N bytes of data. (Note it should *not* be count-bounded
since frames can be of variable size, the goal is to limit memory consumption
not frame production) 

The limit N can be hard coded for now, ultimately it should be configurable but
we need a client-side configuration mechanism for that.

Note this is completely independent of AMQP flow control. AMQP flow control is
designed to protect the broker, this is designed to protect the client's address
space.

Comment 1 Gordon Sim 2008-04-30 14:26:27 UTC
I have added some bounds checking to the client. Only application threads will
block (the io thread needs to send back some controls at times and that never
blocks).

The bounds are specified as a multiple of the max-frame-size and defaults to 2.

Previously I could not run perftests with large counts (anything over 200k on my
laptop) without the client running out of memory. Now I can run with counts of
several million messages. 

Comment 2 Mike Bonnet 2008-05-16 15:08:02 UTC
qpidc-0.2.656926-1.el5 and qpidd-0.2.656926-1.el5 have been pushed to the staging repo for testing


Note You need to log in before you can comment on or make changes to this bug.