Description of problem:

Heketi should have the intelligence to select the nodes based on available hardware resources as well along with the disk space. This is basically required for any solution (Aplo here) which requires the cluster to be expanded for scaling out volumes as needed. 

Consider a cluster of 4 RHGS nodes created using heketi-cli is running out of resources or has almost reached the max memory limit that is allocated while scaling out more than 280 volumes. So in order to meet the current demand for more volumes, we have expanded the same cluster by adding a few more nodes. However, at present, any new volume create requests are still randomly picking up the older nodes as well (because it has the required free space) even though it doesn't have enogh hardware resources (memory/cpu) to process the request, In this scenario, the subsequent volume creation requests times out and eventually fails. So the new nodes used in expanding the cluster are not being effectively used here.

Heketi should have the intelligence built in to handle similar situations so that it supports scaling out volumes.

Version-Release number of selected component (if applicable):
# heketi --version
Heketi v1.3.0-dev-12-g2f52279