The initial OpenStack integration only makes a single VM creation request at a time. This clearly doesn't scale in systems which may be attempting to schedule dozens or hundreds of recipes in parallel.
This needs to be changed so the requests can be dispatched in parallel.
- the current MySQL client library is known to be incompatible with gevent, so that can't be used to provide asynchronous IO for the requests
- we've had issues with MySQL deadlocks when attempting to run recipe state updates in parallel, so we're wary of switching back to that configuration
- use the backported concurrent.futures module from Python 3: https://pypi.python.org/pypi/futures
This provides a nice abstraction for submitting work to a thread pool as callables and then waiting for the results: https://docs.python.org/dev/library/concurrent.futures.html#threadpoolexecutor-example
Alternatively, we could use the threading and Queue modules directly, but I think the futures abstraction is a better choice.
I have tried using concurrent.futures to dispatch multiple requests to OpenStack. Everything seems working fine when I run five jobs in my testing environment. I can see that the requests were dispatched asynchronously to OpenStack.
Note: python-futures is not available on RHEL6 so we have to build it by ourselves.
Beaker 24.0 has been released.