Bug 1262777

Summary: Sahara error creating cluster with HDP 2.0.6
Product: Red Hat OpenStack Reporter: Edu Alcaniz <ealcaniz>
Component: openstack-saharaAssignee: Elise Gafford <egafford>
Status: CLOSED NOTABUG QA Contact: Luigi Toscano <ltoscano>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0 (Kilo)CC: ealcaniz, matt, mimccune, pkshiras, yeylon
Target Milestone: ---   
Target Release: 8.0 (Liberty)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-09-14 10:46:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Edu Alcaniz 2015-09-14 10:30:27 UTC 
Description of problem:
Error creating cluster with HDP version 2.0.6

sahara cluster-create --json cluster_create-hdp.json 

ERROR: Cluster is missing a service: YARN
Error ID: 33c7d728-bd47-453e-96cb-0ef03775b56a


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.Create Node group template -- missing PROCESSES 
Ensure the existence of a JOBTRACKER should any TASKTRACKER be deployed to the cluster -- if you follow http://docs.openstack.org/developer/sahara/userdoc/hdp_plugin.html
??? its available with Vanilla plugins but not in HDP
2. Create cluster template
3. Create cluster


1. 

[root@sahara ~(keystone_admin)]# cat ng_worker_template_create-hdp.json 
{
    "name": "worker-hdp-206-tmpl",
    "flavor_id": "2",
    "plugin_name": "hdp",
    "hadoop_version": "2.0.6",
    "node_processes": ["DATANODE"],
    "auto_security_group": true,
    "floating_ip_pool" : "04946cae-672c-4b34-9f59-3b6066cdef0f"

}

[root@sahara ~(keystone_admin)]# cat ng_master_template_create-hdp.json 
{
    "name": "master-hdp-206-tmpl",
    "flavor_id": "2",
    "plugin_name": "hdp",
    "hadoop_version": "2.0.6",
    "node_processes": ["NAMENODE"],
    "auto_security_group": true,
    "floating_ip_pool" : "04946cae-672c-4b34-9f59-3b6066cdef0f"
}


2.

[root@sahara ~(keystone_admin)]# cat cluster_template_create-hdp.json 
{
    "name": "cluster-hdp-206-template",
    "plugin_name": "hdp",
    "hadoop_version": "2.0.6",
    "node_groups": [
        {
            "name": "master",
            "node_group_template_id": "61490289-3a18-415b-8a49-bce29155b792",
            "count": 1
        },
        {
            "name": "workers",
            "node_group_template_id": "8d6d0be4-0fc5-4640-908e-1a95f78dff93",
            "count": 2
        }
    ]
}



3. 
[root@sahara ~(keystone_admin)]# cat cluster_create-hdp.json 
{
    "name": "cluster-hdp-1",
    "plugin_name": "hdp",
    "hadoop_version": "2.0.6",
    "cluster_template_id" : "971f9d96-7f41-4512-84f2-f50c1ffa1608",
    "user_keypair_id": "sahara-key",
    "default_image_id": "3b398a98-3869-4718-b1df-ac71e1cb7bea",
    "neutron_management_network": "c7982086-7fde-4572-9d8e-a43fc71c3b6f"
}



Actual results:


Expected results:


Additional info:
Comment 4 Luigi Toscano 2015-09-14 10:46:33 UTC 
The cluster is incomplete, you need much more services than a NAMENODE and a DATANODE.
The information about validation on the upstream doc are outdated/for HDP 1.x (please file a bug *upstream* for that). JobTracker and TaskTracker are Hadoop 1.x only.



A more complete example of HDP 2.0.6 is available as template used for integration tests:
http://git.openstack.org/cgit/openstack/sahara/tree/etc/scenario/sahara-ci/hdp-2.0.6.yaml?h=stable/kilo


Instead of starting from scratch, you can create a set of "default templates" with sane values. See here for more details:
http://git.openstack.org/cgit/openstack/sahara/tree/sahara/db/templates/README.rst?h=stable/kilo