Bug 1712001

As glance-api service is fully independent and stateless the different nodes do not know nor care if they are operating alone or in a cluster. To address some of the distributed models we need to change that and build understanding between the nodes that there is others in same environments that might have different needs or resources available.

Main concerns to address at the beginning are:

0) Cache management
   - For improved usability we need to move some of the caching functionality from node-local tooling to the API. To be able to manage pre-caching and listing cached images in distributed or HA-environments the glance-api nodes needs to be able to talk to eachother so that the commands does not need to be targeted to individual nodes behind loadbalancers.

1) Image Import
   - Currently if "glance-local" import method is availble the glance-api nodes needs to share common filesystem or user needs to (be able) target individual nodes so they can issue the import call to the node that has the image staged. Once the api nodes can communicate with eachother the work can be done by node that has access to the data regardless if it received the original Rest API call.

2) Copying Image data between image stores
   - The call to copy a image to another store in the environment might not be possible from the node that receives the Rest API call. To implement asynchronous copy operation we need, yet again, be able to have the nodes communicating with each other to ensure that the work can be done by node that has access to both, source and destination.

Implementation:

Utilizing RabbitMQ as that is already deployment dependency and is proven technology for such purpose. Unfortunately oslo_messaging does not provide all needed Rabbit features to utilize it in this work so we will need to work directly with available clients and build the boilerplating orselves.