Cloned from launchpad blueprint https://blueprints.launchpad.net/cinder/+spec/filtering-weighing-with-driver-supplied-functions.

Description:

We have been using filter scheduler since Grizzly.  While the default filters and weigher certainly do the job and help scheduler make good decision and rule out unqualified back-ends, there are also some situation when some filters/weighers become useless.  For example, due to thin-provisioning or other advanced data manipulating mechanisms back-ends may have, the 'free capacity' becomes less useful we thought it would be. 

What's more, different backends have different metrics that are actually useful for them, and only the manufacturer really knows what these are, which makes it hard or almost impossible to create a global filter for these kind of metrics.
 
For example, a foomatic super array might have an absolute maximum capacity of 1000 volumes, and might get some performance degredation if more than 75% of total space is actually used. Also it has a max volume size of 500 gig. This sort of detail is likely to be highly variable between vendors, and even between different arrays of from the same vendor.

During HongKong design summit sessions, some idea came up from RedHat engineer.  He mentioned that Cinder could allow each backend to return two new functions when reporting capabilities/status, and those will be used for filtering and weighing.  

    FILTER FUNCTION and GOODNESS FUNCTION, together with any values needed by these functions.
    FILTER FUNCTION is a function that evaluates to True or False, and must evaluate to True for a specific volume in order for that backend to be considered a candidate for this volume.
    GOODNESS FUNCTION is a function that evaluates to a value from one to one hundred, indicating how much of a good fit this backend thinks it is for a specific volume. 100 means 'I'm ideal' 50 means 'if nobody better is about' and 10 'really only as a last resort'.  (Better guidance on these values can be provided)
    In both cases, the function has access to details of the volume, the type, and the last reported stats from the array, e.g. volume.size type.min_iops stats.capacity
To give a concrete example for our foomatic super array, it might return:
    {
        "Backend name": "Foomatic super-array 2000";
        "Filter funtion": "(stats.n_vols < 1000) AND (stats.free_space <= volume.size) AND (volume.size <= 500)";
        "Goodness function": "(stats.percent_capacity_used < 75 ? 50 + (stats.percent_capacity_used / 2) : (stats.percent_capacity_used / 2))";
        "percent_capacity_used": 25;
        "free_space": 2575;
        "n_vols": 217;
        ...
    }

The exact operations supported by the function can be decided by the implementer, but a good start would be <, >, =, <=, >=, min(X,Y,...), max(X,Y,...), +, -, *, /, (X?Y:Z), abs()

With that, scheduler is able to share some internal wisdom of each back-ends to deal with vendor specific metrics or common metrics that vendors tend to interpret differently.  Meanwhile, scheduler also have filters and weighers that does global filtering/weighing.  So schedulers takes two set of filters and weighers to make optimal placement decisions. 

The workflow will be:
 1) back-end drivers periodically reports status and functions to scheduler;
 2) scheduler updates its cache for status and functions;
 3) when new request comes, scheduler filters back-ends using filter-chain like this:

  INPUT (back-end status) ->  [BACK-END_SPECIFIC_FILTER, Global_Filters(availability_zone_filter, retry_filter, json_filter, etc)]
                                                     |
                                                    v
            [BACK-END_SPECIFIC_WEIGHTER, Global_Weigher]  ->   Normalized Weights for candidates -> Winner takes the job



Specification URL (additional information):

https://etherpad.openstack.org/p/cinder-driver-goodness-reporting