Bug 1041030

Cloned from launchpad blueprint https://blueprints.launchpad.net/nova/+spec/no-db-scheduler.

Description:

Current way how work Nova.scheduler is not optimal there is the way to make it more scalable & flexible.

New scheduler design doc: https://etherpad.openstack.org/p/scheduler-design-proposal

More details about limitations of current scheduler you can see here:
https://docs.google.com/a/mirantis.com/document/d/1_DRv7it_mwalEZzLy5WO92TJcummpmWL4NWsWf0UWiQ/edit

There is a community effort to make 1 scheduler lib or 1 scheduler as a Service, to reduce amount of duplication of code.

During the H cycle we tried to move all common parts of scheduler to oslo.scheduler:
https://blueprints.launchpad.net/oslo/+spec/oslo-scheduler

But we faced a lot of problems. The main problems was, that host_states are deeply connected with db.models that represent host states. And scheduler use directly projects db.

The second thing that we faced was the bad performance of scheduler. The main reason is the we are storing & updating all data in DB + on each request we are fetching all data from DB.

So as a solution we try to resolve these 2 problems:
1) Each scheduler should store all host_states locally (in some object)
2) There should be mechanism that sync all host_states in different backends
(using memcahced or sqla backends, to store updates)
This is required because another way are fanouts RPC calls, that doesn't scale..
3) There should be namespaces in host_states (to store in the same time data from different services) and be able to use filters from different projects in the same time.
4) host_state is just JSON like object.
5) Scheduler should have RPC calls to update/get host states + nova.manger, nova.api should call them instead calling nova.db.api

Our goal is to make queue of next patches:
1) Add new db.models to implement backend of synchronizer
2) Add synchronizer
3) Switch Scheduler to use synchronizer + add new RPC calls to scheduler + call from compute.manager it
4) Implement in Scheduler all required RPC calls to get/list host states
5) Remove compute_nodes table, and use scheduler in db.api as implementation
6) Remove comupte_* methods from db.api and use directly scheduler

This is the first that will make Scheduler faster, flexible & much more independent from Nova code.
Which will make unification of scheduler (or making scheduler aaS much simpler)

Specification URL (additional information):

https://etherpad.openstack.org/p/scheduler-design-proposal