Please check http://gerrit.ovirt.org/#/c/19744/ for a possible solution. Roy - the reject policy is run in your own thread, but only after queue is exhausted. I think we should test it under load, but problematic ordering will not be worse than it is today.
Looking at previous comments there are two solutions: 1. Configurable number of threads and queue size. Solution proposed above. 2. Make IVdsServer implementation async. As the first one seems to be ongoing I would like to suggest solution for second one. There is work in progress to have AMQP communication which will introduce async communication. This change can be implemented to provide async behavior for AMQP and http VdsServers.
ovirt 3.4.0 alpha has been released
Bug was verified on RHEVM version 3.4.1-0.23.el6ev OS Version: RHEL - 6Server - 6.5.0.1.el6 Kernel Version: 2.6.32 - 431.el6.x86_64 KVM Version: 0.12.1.2 - 2.415.el6_5.9 LIBVIRT Version: libvirt-0.10.2-29.el6_5.7 VDSM Version: vdsm-4.14.7-2.el6ev The environment: 1 REAL DC - 1 FAKE DC 1 REAL Cluster - 1 FAKE Cluster 1 REAL SD - 10 FAKE SDs 50 REAL VMs (4CPU - 2G RAM) 1 REAL HOST - 50 Fake HOSTS Created 80 LUNs on XtreamIO iSCSI storage. Each FAKE SD has 8 LUNs. "The thread pool is out of limit" didn't appear. See Bug 1106572 - [scale] [storage] ConnectStorageServer failed - The thread pool is out of limit (engine finish its thread pool) for more details
all these bugs were fixed and released as part of 3.4.1, and weren't closed since they weren't included in any errata. closing as current release.