Bug 1262777 - Sahara error creating cluster with HDP 2.0.6
Summary: Sahara error creating cluster with HDP 2.0.6
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-sahara
Version: 7.0 (Kilo)
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
: 8.0 (Liberty)
Assignee: Elise Gafford
QA Contact: Luigi Toscano
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-09-14 10:30 UTC by Edu Alcaniz
Modified: 2015-09-14 10:52 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-09-14 10:46:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Edu Alcaniz 2015-09-14 10:30:27 UTC
Description of problem:
Error creating cluster with HDP version 2.0.6

sahara cluster-create --json cluster_create-hdp.json 

ERROR: Cluster is missing a service: YARN
Error ID: 33c7d728-bd47-453e-96cb-0ef03775b56a


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.Create Node group template -- missing PROCESSES 
Ensure the existence of a JOBTRACKER should any TASKTRACKER be deployed to the cluster -- if you follow http://docs.openstack.org/developer/sahara/userdoc/hdp_plugin.html
??? its available with Vanilla plugins but not in HDP
2. Create cluster template
3. Create cluster


1. 

[root@sahara ~(keystone_admin)]# cat ng_worker_template_create-hdp.json 
{
    "name": "worker-hdp-206-tmpl",
    "flavor_id": "2",
    "plugin_name": "hdp",
    "hadoop_version": "2.0.6",
    "node_processes": ["DATANODE"],
    "auto_security_group": true,
    "floating_ip_pool" : "04946cae-672c-4b34-9f59-3b6066cdef0f"

}

[root@sahara ~(keystone_admin)]# cat ng_master_template_create-hdp.json 
{
    "name": "master-hdp-206-tmpl",
    "flavor_id": "2",
    "plugin_name": "hdp",
    "hadoop_version": "2.0.6",
    "node_processes": ["NAMENODE"],
    "auto_security_group": true,
    "floating_ip_pool" : "04946cae-672c-4b34-9f59-3b6066cdef0f"
}


2.

[root@sahara ~(keystone_admin)]# cat cluster_template_create-hdp.json 
{
    "name": "cluster-hdp-206-template",
    "plugin_name": "hdp",
    "hadoop_version": "2.0.6",
    "node_groups": [
        {
            "name": "master",
            "node_group_template_id": "61490289-3a18-415b-8a49-bce29155b792",
            "count": 1
        },
        {
            "name": "workers",
            "node_group_template_id": "8d6d0be4-0fc5-4640-908e-1a95f78dff93",
            "count": 2
        }
    ]
}



3. 
[root@sahara ~(keystone_admin)]# cat cluster_create-hdp.json 
{
    "name": "cluster-hdp-1",
    "plugin_name": "hdp",
    "hadoop_version": "2.0.6",
    "cluster_template_id" : "971f9d96-7f41-4512-84f2-f50c1ffa1608",
    "user_keypair_id": "sahara-key",
    "default_image_id": "3b398a98-3869-4718-b1df-ac71e1cb7bea",
    "neutron_management_network": "c7982086-7fde-4572-9d8e-a43fc71c3b6f"
}



Actual results:


Expected results:


Additional info:

Comment 4 Luigi Toscano 2015-09-14 10:46:33 UTC
The cluster is incomplete, you need much more services than a NAMENODE and a DATANODE.
The information about validation on the upstream doc are outdated/for HDP 1.x (please file a bug *upstream* for that). JobTracker and TaskTracker are Hadoop 1.x only.



A more complete example of HDP 2.0.6 is available as template used for integration tests:
http://git.openstack.org/cgit/openstack/sahara/tree/etc/scenario/sahara-ci/hdp-2.0.6.yaml?h=stable/kilo


Instead of starting from scratch, you can create a set of "default templates" with sane values. See here for more details:
http://git.openstack.org/cgit/openstack/sahara/tree/sahara/db/templates/README.rst?h=stable/kilo


Note You need to log in before you can comment on or make changes to this bug.