Description of problem:
If the 32-bit qpid broker has a queue with ~3GB of messages in it, trying to send something more to this queue results in std::bad_alloc error from broker. The client returns the connection error:
warning Connection [127.0.0.1:59573-127.0.0.1:5672] closed
Failed to connect (reconnect disabled)
It seems that 32-bit broker is unable to allocate memory somewhere above 3GB.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Start up the 32-bit broker, create a queue with capacity more than 3GB.
2. Start a client which sends lots and lots of messages to this queue.
3. When broker's memory allocation reaches 3GB, it will unable to receive any more messages.
Broker throws std::bad_alloc error and does not accept any more messages.
Broker should be able to accept a message from a client in case that there is some free memory left on the machine and the target queue did not reach its capacity limit.
I tried to run a few test scenarios, such as:
1. Create one queue with a big capacity (4-5 GB). Try to send about 3.5 GB of messages to it. Messages were of a various sizes, from 1KB up to 100MB.
2. Create a number of queues (from 2 to 10) with big capacity. Try to distribute about 3.5 GB of messages among them.
The result was always the same: as soon as broker's memory consumption reached some value around 3GB, it was unable to receive messages any more.
So, either there is a bug in qpid broker, or there is a limitation of its capacity which cannot be overrun in 32-bit environment, in which case such a limitation should be properly described in user documentation.
Created attachment 535946 [details]
Script to reproduce the bug.
Just untar it, change to bug_bad_alloc directory and type "make run".
Created attachment 535947 [details]
qpid broker log with std::bad_alloc error.
This is in part fundamental; it's exhausting process virtual memory. Is there a better way to handle it?
I'm inclined to seek a better log message (if possible) and a documentation fix.
I concur with Justin - I don't think we can do much in the case that std::bad_alloc is thrown - it does mean that we can allocate no more memory. And in this case it almost certainly that we've run out of virtual address space.
We might be able to produce a log message before exiting, but we might not. The log subsystem itself needs to allocate memory.
I think that this can only be solved with documentation.
We may need to figure out safe maxima for the memory size that can be used.
I don't think this is an actual bug, I propose closing it "won't fix."