Bug 1194532

Cloned from launchpad blueprint https://blueprints.launchpad.net/sahara/+spec/edp-job-types-endpoint.

Description:

The current /jobs/config-hints/<job_type> endpoint is not adequate for providing configuration hints for job types because it does not take a cluster or plugin argument. This has been a problem for a long time, since the config hints need to be specific to a particular plugin type and even the version (often synonymous with the hadoop version). Additionally, there is not a way currently for a user to query a Sahara instance to find out what kinds of jobs its supports.

To solve both of these issues, we can add a new job-types endpoint that will report the following information to the user:

All of the job types that can be run on that Sahara instance
   For each job type, the plugins that support it
     For each plugin, the versions of that plugin that support it with (optionally) configuration hints for that version

With this endpoint, a user can know exactly which types of jobs can be run and what the choices are for a cluster deployment to run those jobs (what image is needed, what templates do I need to use or create, etc)

Proposed JSON output from the endpoint would look something like this (for a cluster with only the hdp plugin enabled and without config hints)

{
    "job_types": {
        "Hive": [
            {
                "description": "The Hortonworks Sahara plugin automates the deployment of the Hortonworks Data Platform (HDP) on OpenStack.", 
                "name": "hdp", 
                "title": "Hortonworks Data Platform", 
                "versions": {
                    "1.3.2": {}, 
                    "2.0.6": {}
                }
            }
        ], 
        "Java": [
            {
                "description": "The Hortonworks Sahara plugin automates the deployment of the Hortonworks Data Platform (HDP) on OpenStack.", 
                "name": "hdp", 
                "title": "Hortonworks Data Platform", 
                "versions": {
                    "1.3.2": {}, 
                    "2.0.6": {}
                }
            }
        ], 
        ... and more job types here ...
    }
}

Specification URL (additional information):

None